According to JKR, gnomes are only found in the homes of wizards. No specific explanation is given as to why this is.
Q. Why are the gnomes bad? What do they do?
JKR: Gnomes eat the roots of your plants, and make little heaps of earth, like moles do. They are also a bit of a giveaway that
wizards live in a house.
Scholastic: Interview With J. K. Rowling
As to their other detrimental effects, the Wiki description of the Harry Potter and the Philosopher's Stone video game indicates that in large numbers, the gnomes can become emboldened and quite vicious
Rolanda Hooch: "No flying today, Mr Potter. There are gnomes burrowing into the pitch."
Harry Potter: "Why would that stop anyone flying?"
Rolanda Hooch: "Have you tried to land a broomstick with fifteen gnomes snapping at your ankles? I thought not. Run along now."