The way that the Ministry detects underage magic is with The Trace, but it only applies to people under the age of 17. Hagrid is over the age of 17, so he doesn't have the Trace on him. Therefore the only way the Ministry would know is if someone reported him, which the Dursleys are unlikely to.
"The Trace, the Trace!" said Mad-Eye impatiently. "The charm that
detects magical activity around under-seventeens, the way the Ministry
finds out about underage magic! If you, or anyone around you, casts a
spell to get you out of here, Thicknesse is going to know about it,
and so will the Death Eaters.
"We can't wait for the Trace to break,
because the moment you turn seventeen you'll lose all the protection
your mother gave you. ..."
The Deathly Hallows, Chapter 4
As for when Hagrid was with Harry in Philosopher's Stone, he was given special dispensation to use magic to retrieve Harry Potter, but was barred from doing so once he had him. So the time when he's retrieving him, the Trace was expected to be activated a few times as necessary:
"How did you get here?" Harry asked, looking around for another boat.
"Flew," said Hagrid.
"Yeah --- but we'll go back in this. Not s'pposed ter use magic now
I've got yeh."
Philospher's Stone, Chapter 5 (Bold mine)
If he's not supposed to use magic now, he was allowed to before. Since he continues using it, only stipulating that Harry not tell, he must expect that any further triggers of the Trace will be covered under Dumbledore's permission or, as BMWurm suggests in the comments, it may be assumed that it is simple uncontrollable magic. They don't go after Harry earlier when he accidentally makes the glass disappear in the snake enclosure.