After Voldemort's disappearance, some wizards secretly stayed allied to him (as seen in Goblet of Fire when they immediately come to him after he returns in the graveyard). There were very few, however, that showed up. Some more were broken out of Azkaban the following year, but it couldn't have been more than a handful of faithful supporters.
Then, with this limited number of followers, within 3 years Voldemort manages to have people allied to his cause in some of the highest positions in the wizarding community, including:
- Headteacher of Hogwarts (or so he thought)
- Head of Magical Law Enforcement
- Minister of Magic (pretty much the highest position a wizard/witch can have as far as I know)
This is not including the vast number of his followers that worked under these people, or were working to further his cause anywhere else (other teachers, snatchers, others within the ministry etc.).
So how did he manage to garner so much support so quickly, and in such influential positions within the wizarding infrastructure?
Was he even nearly this powerful the first time that he was around?