It seems that Arthur Weasley and Kingsley Shacklebolt (and probably others in the Order of the Phoenix) believed that Voldermort would be able to get at Harry when he turns seventeen and the protective charm that Dumbledore put on the house would break.

In the Deathly Hallows (page 33 of Scholastic paperback), Harry is telling the Dursleys:

Once I'm seventeen, the protective charm that keeps me safe will break, and that exposes you as well as me. The Order is sure Voldemort will target you...

Also, earlier, Snape is discussing with Voldemort and the Death Eaters about the plans to move Harry. While Snape doesn't mention the Dursleys or Privet Drive by name, he does imply that he (or Voldemort already) knows the location. Snape tells Voldemort (Deathly Hallows, page 3 of the Scholastic version):

My Lord, the Order of the Phoenix intends to move Harry Potter from his current place of safety on Saturday next, at nightfall (emphasis mine)

It is unclear how much Voldermort and the Death Eaters knew about the Dursleys themselves. But obviously they knew Harry was at Privet drive. When Harry and the others left, the Death Eaters were waiting. As Harry told Mrs. Weasley (Deathly Hallows, page 68 of the Scholastic version):

"The Death Eaters were waiting for us," Harry told her. "We were surrounded the moment we took off..."

It seems that Arthur Weasley and Kingsley Shacklebolt (and probably others in the Order of the Phoenix) believed that Voldemort would be able to get at Harry when he turns seventeen and the protective charm that Dumbledore put on the house would break.

In the Deathly Hallows (page 33 of Scholastic paperback), Harry is telling the Dursleys:

Once I’m seventeen, the protective charm that keeps me safe will break, and that exposes you as well as me. The Order is sure Voldemort will target you…

Also, earlier, Snape is discussing with Voldemort and the Death Eaters about the plans to move Harry. While Snape doesn’t mention the Dursleys or Privet Drive by name, he does imply that he (or Voldemort already) knows the location. Snape tells Voldemort (Deathly Hallows, page 3 of the Scholastic version):

My Lord, the Order of the Phoenix intends to move Harry Potter from his current place of safety on Saturday next, at nightfall (emphasis mine)

It is unclear how much Voldemort and the Death Eaters knew about the Dursleys themselves. But obviously they knew Harry was at Privet drive. When Harry and the others left, the Death Eaters were waiting. As Harry told Mrs. Weasley (Deathly Hallows, page 68 of the Scholastic version):

“The Death Eaters were waiting for us,” Harry told her. “We were surrounded the moment we took off…”