> But I dared not go where other humans were plentiful, for I knew the Aurors were still abroad and searching for me.
~GoF, page 567

He was scared of being discovered, and no one crossed his path until Quirrell.

It also seems that possession in this case means sharing a body vs. taking control, as Quirrell acted as his own agent even while "possessed" by Voldemort.

It was coincidence that Quirrell made his trip right before Harry entered Hogwarts.