Tom Riddle did not think highly of either parent, but he seemed to value his mother slightly more, and siding with his Muggle father would have been beyond his emotional capabilities.
You think I was going to use my filthy Muggle father's name forever? I, in whose veins runs the blood of Salazar Slytherin himself, through my mother's side? I, keep the name of a foul, common Muggle, who abandoned me even before I was born, just because he found out his wife was a witch?
Emphasis mine. Simply being a Muggle and leaving his abusive witch wife was enough for Voldemort to side against him. As for Merope Gaunt, even as a child Tom writes her off just for dying.
"My mother can't have been magical, or she wouldn't have died," said Tom, more to himself than Dumbledore.
Furthermore, Tom was raised in an era where men were not typically viewed as victims in cases of sexual assault. Voldemort extensively researched his family history and might have guessed his father had been magically ensnared, but to recognize Tom Sr. as a victim goes against Voldemort's character. If he knew Tom Riddle Sr. was taken against his will, it would not have changed Voldemort's contempt for his Muggle parentage.