I think another crucial question is - how did Hagrid get to Godric's Hollow in the first place? The journey couldn't have taken him long, since he was the first to arrive. In the books there is no mention of Snape coming into the house and finding the dead Potters, as it was shown in the movie. Instead, Hagrid claims he was the first there, and that he personally rescued Harry from the ruins of the house (Sirius arrived shortly after that on his motorbike). If both Hagrid and Sirius traveled there very slowly, then Harry would've spent many, many hours (or even days) alone in the destroyed house, which I don't believe.
Which makes me think Hagrid must have used a means of transport that would have gotten him there reasonably quickly - the Knight Bus, a Portkey, or even the Floo network (Godric's Hollow is a wizarding village, so at least one of the magical inhabitants must have a Floo-connected fireplace at home, which Hagrid could have used). Anything else would be too slow, and I cannot imagine that when Dumbledore was sending Hagrid to fetch Harry, he told him: "There is no need to hurry, just take your time, it's no problem if you get there next day, the (hungry, thirsty and scared) baby can wait..." That sounds rather cold, even from someone like Dumbledore.
I'm also still not entirely convinced that Hagrid is unable to Apparate. We never see him do so, but when the Dursleys were hiding with Harry on that island in the middle of the sea in Book One, he somehow managed to transport himself there without any visible magical means (no magic motorbike, no broomstick etc.) So maybe he can Apparate, but his skills are rather mediocre, so he only uses it in cases of emergency (the Potters' death definitely would have been one). But since he's not very confident in his Apparating skills, he wouldn't dare to Apparate again with Harry - risking his own health is one thing, risking the health of a little baby quite another. So the motorbike would come in handy.
But all in all, when contemplating why Hagrid didn't use the same method of transport which he used to get to Godric's Hollow in the first place to leave the village again with Harry, we have to take into account that he wasn't the only person who behaved in an absolutely nonsensensical way that fateful Halloween night 1981.
For example Sirius Black. He found out that his friend Peter, the Potters' Secret Keeper, was missing. Worried for the safety of James and Lily, he immediately set out to their house in Godric's Hollow - on a motorbike... When your friends are in danger and you're coming to their aid, time is a very important factor. Any other wizard would have chosen the quickest possible way - Apparition (which would have transported him there within seconds) instead of a lengthy motorbike ride (which seems to take several hours; or at the very least minutes, if he was somewhere close to their location).
The way Rowling had described the events of Halloween 1981, several things do not make sense and there are some obvious plot holes. For example, how did every wizard in Britain learn so quickly that Voldemort was gone? Sure, the press. But how did they know what exactly happened at the Potters' house, when there were no surviving witnesses apart from a little baby boy? Who couldn't have been questioned (even were he capable of speech), since he was with Hagrid the whole time, and we never hear Hagrid mention that he has stopped at an Aurors Office for several hours on his way to Privet Drive...
The reason why Hagrid chose to use Sirius' motorbike is sadly one of those plot holes. The best explanation probably is that at that point in the series Rowling had not yet developed all means of transport used in the wizarding world as we know them now, and so a flying motorbike made sense then. But after she has developed several much quicker transport methods, it stopped to make sense.