Something that I find interesting about Voldemort is how predictable he is! In all of the books apart from book 6, he attacks at the end of the school year! Why do all of his attacks happen around this time of the year?
Philosopher's (Sorcerer's) Stone
Professor Quirrell is captured by Voldemort while on holiday. The first time he can attempt to take the Stone is when it is placed in Hogwarts at the start of the year. It takes him a year to work out how to get past the obstacles.
Chamber of Secrets
Malfoy places the Horcrux on Ginny during the summer holidays in Diagon Alley. She first uses the diary at school and it takes Voldemort's horcrux a year to gain enough power to trigger the events that lead to encounter in the titular chamber.
Prisoner of Azkaban
Voldemort doesn't attack at the end of the year.
Goblet of Fire
Voldemort specifically wants to kill Harry during the events of the Triwizard Final so it looks like an accident and he can stay in the shadows. He also needs the school term to start to put his fake Moody in place.
Order of the Phoenix
Voldemort only finds out about the link between himself and Harry later in the year and uses it to trick Harry into thinking he has captured Sirius.
It takes Malfoy nearly a year to repair the Vanishing Cabinet, he cannot start the task until the beginning of the school term.
The battle takes place in May, so technically not at the end of the school year.
As we can see plans seem to need to start at the beginning of the year (to get items or people into Hogwarts) or timed to avoid suspicion (Goblet of Fire). We know Voldemort cannot touch Harry at the Dursley's house, so before or just after term is the best time to attack.
One question we could ask is "Why does it always take Voldemort's minions a year to do their assigned task?"
A popular fan theory (for years 5 upward when Voldemort is alive and actively gunning for Harry) is that the attacks are usually around exam period, for the reason that both students and teachers are at their busiest and most stressed.
e.g. Would Harry have charged off to the MoM without thinking if it were a cool autumn day months before any stress and exams? As it happens, it's halfway through History of Magic, on a hot summers day.
Another example would be sixth year - Draco is presumably able to spend more time tinkering with the cabinet when everybody is too busy revising to notice/care where he is, well, everyone but Harry of course.
Not by any means a comprehensive theory, but something to consider.