Much of the back story is made clear in the books. Viserys has always been arrogant and cruel toward his sister. Some of it is probably hereditary. The Targarians had a long "tradition" of marrying brother to sister, which accounts for their distinctive hair and eye color, and occasional cases of madness. Viserys' father Aeyres Targarian was called "the mad king" and for very good reasons. He was abscessed with fire, he abused his wife, and he burned Ned Stark's father and brother alive.
So you have a boy whose father was a king and a paranoid schizophrenic, who is a product of a long history of incest, who lost his parents at a young age, and who is raised by strangers on the stories about how he is really a rightful king of a vast kingdom, and how that kingdom was stolen from him. Oh, and unlike Daenerys, he is not very bright. Also, Viserys is resentful that he does not get to marry Daenerys himself, as is the custom of his family.
On the other hand Daenerys puts up with him until the very end for the same reason children put up with abusive parents. She was born after her father was killed by Jaime Lannister, and her mother died in childbirth. Viserys is the only family she has, and the only family she has ever known.
And, yes, you do have to keep in mind that all the children are older in the series than in the books. In the book Denny is only 13 when she is married to Khal Drogo.
Also, do not forget that much of what is "regular" today was not at all regular during the medieval times.