Actually what Hagrid says In H.P.S.S is this
"It's them who should be sorry! I knew yeh weren't gettin' yer letters but I never thought ya wouldn't even know about Hogwarts, for crying out loud! Did you never wonder where your parents learned it all?"
Blockquote
So no... They knew Harry was NOT getting to read his letters. How they knew this was because Harry never replied. Also the letters are magical. Showing up in eggs and such, Hagrid Bewitched them.
Be grateful if yeh didn't mention that ter anyone at Hogwarts. he said "I'm not supposed to do magic, strictly speaking I was allowed to do a bit ter follow yeh an' get your letters to yeh and stuff--- one o' the reasons I was so keen to take on the job.
H.P.S.S chapter 4 page 45
So the letters themselves could be marked as unread by the person they were meant for.