Voldemort couldn't kill Harry as long as he called the Dursleys home. Why didn't he just murder the Dursleys? This would prevent Harry from being able to live there.
The traditional fanon answer is "there were blood wards protecting the house". It would not, however, have stopped Voldemort (or his Death Eaters) from attacking the Dursleys when they were out (as proved by Umbridge's dementor attack).
That said, Voldemort would have needed to figure out 1) that Harry lived at the Dursleys (which itself might not have been common knowledge), and 2) that the blood protection was tied to that fact (that's assuming it was, which we only have Dumbledore's word on, IIRC, so that might not even be true).
And after the ritual at the end of Harry's fourth year, Voldemort, specifically, had little to worry about from the blood protection anyway.
The Dursleys were safe by the same form of magic that kept Harry safe. However, all of them were only safe within the bounds of their home, till Harry came of age(i.e., he turned 17). Harry or any one of the Dursleys could have been attacked outside their home. Thus, when Harry was about to turn 17, the Order of Phoenix took the Dursleys some place safe. On that night, Harry and his protective guard were attacked as soon as they crossed the 'boundary' that protected the house, since they could not be attacked within that boundary. It was the house which had the protection.
Harry Potter and the Half Blood Prince:
Chapter 3: Will and Won't
..but Dumbledore raised his finger for silence, a silence which fell as though he had struck Uncle Vernon dumb. “The magic I evoked fifteen years ago means that Harry has powerful protection while he can still call this house ‘home.’ However miserable he has been here, however unwelcome, however badlytreated, you have at least, grudgingly, allowed him houseroom. This magic will cease to operate the moment that Harry turns seventeen; in other words, at the moment he becomes a man. I ask only this: that you allow Harry to return, once more, to this house, before his seventeenth birthday, which will ensure that the protection continues until that time.”
Harry Potter and the Deathly Hallows:
Chapter 3: The Dursleys Departing
“You claim,” said Uncle Vernon, starting to pace yet again, “that this Lord Thing—”
“—Voldemort,” said Harry impatiently, “and we’ve been through this about a hundred times already. This isn’t a claim, it’s fact, Dumbledore told you last year, and Kingsley and Mr. Weasley—”...
...“—Kingsley and Mr. Weasley explained it all as well,”Harry pressed on remorselessly. "Once I’m seventeen, the protective charm that keeps me safe will break, and that exposes you as well as me..."
What would he achieve with this?
According to Dumbledore:
The magic I evoked fifteen years ago means that Harry has powerful protection while he can still call this house ‘home.’
Harry and Aunt Marge are the only living relatives of the Dursleys. So if the Dursleys are killed, then Harry will inherit the house or part of it.
Technically he will be still able to call the house his 'home' and the protection will stay.
There is also the possibility that Vernon and Petunia Dursley created a will that explicitly transfers the full ownership of the house to Aunt Marge in case they are all dead. In this case the Order of the Phoenix will have hard time persuading Aunt Marge to invite Harry to stay until he is seventeen.
Still I find this unlikely. Even the Dursleys, as horrible as they are, would not plan for what happens AFTER the death of their only child.