His last, comforting thought before he fell asleep was that even if the Potters were involved, there was no reason for them to come near him and Mrs. Dursley. The Potters knew very well what he and Petunia thought about them and their kind. . . .
Vernon Dursley knew who James Potter was; he was the husband of his sister-in-law.
“So?” snapped Mrs. Dursley.
“Well, I just thought . . . maybe . . . it was something to do with . . . you know . . . her crowd.”
“Now, you listen here, boy,” he snarled, “I accept there’s something strange about you, probably nothing a good beating wouldn’t have cured — and as for all this about your parents, well, they were weirdos, no denying it, and the world’s better off without them in my opinion — asked for all they got, getting mixed up with these wizarding types — just what I expected, always knew they’d come to a sticky end —”
Had Vernon Dursley and James Potter ever met?
We know that Vernon dislikes wizards a lot, but this seems to imply that he had met James Potter before and didn't like him (insults highlighted in bold!).