Some do, but some also infest Muggles.
Certain magical pests are either created by or attracted to magic, so therefore affect only wizard dwellings. Other magical pests do indeed infest and affect Muggles, but they are simply ignorant to the source of their troubles.
Pests that only exist around magic:
Ashwinders:
Ashwinders are created when magical fires are left to burn without being checked for too long.
“The Ashwinder is created when a magical fire2 is allowed to burn unchecked for too long.”
- Fantastic Beasts and Where to Find Them
A magical fire is a fire to which a magical substance like Floo powder is added.
“2 Any fire to which a magical substance such as Floo powder has been added.”
- Fantastic Beasts and Where to Find Them
As these types of fires would only exist in wizarding homes, Muggles not having any access to magical substances, Ashwinders would not infest Muggle homes.
Ghouls:
Ghouls live in attics or barns owned by wizards. There is a department in the Ministry that removes ghouls in places that will next be owned by Muggles.
“The ghoul, though ugly, is not a particularly dangerous creature. It resembles a somewhat slimy, buck-toothed ogre, and generally resides in attics or barns belonging to wizards, where it eats spiders and moths. It moans and occasionally throws objects around, but is essentially simple-minded and will, at worst, growl alarmingly at anyone who stumbles across it. A Ghoul Task Force exists at the Department for the Regulation and Control of Magical Creatures to remove ghouls from dwellings that have passed into Muggle hands, but in wizarding families the ghoul often becomes a talking point or even a family pet.”
- Fantastic Beasts and Where to Find Them
They only seem to choose wizarding dwellings to begin living in, and if the dwelling later comes into Muggle hands, the Ministry removes the ghoul first.
Gnomes:
While there is nothing in the actual series indicating that gnomes only infest wizards’ homes, JKR said in an interview that they are a sign wizards live in a house.
Why are the gnomes bad? What do they do?
JKR: Gnomes eat the roots of your plants, and make little heaps of earth, like moles do. They are also a bit of a giveaway that wizards live in a house.
- Scholastic: Interview With J. K. Rowling
For them to be a sign that wizards live in a house, that therefore means they do not infest Muggle dwellings.
Pests that live around Muggles as well:
Billywigs:
Billywigs are stinging insects that do go near Muggles, but are too fast for Muggles to notice them.
“The Billywig is an insect native to Australia. It is around half an inch long and a vivid sapphire blue, although its speed is such that it is rarely noticed by Muggles and often not by wizards until they have been stung.”
- Fantastic Beasts and Where to Find Them
It is not mentioned if they sting Muggles as well as wizards.
Chizpurfles:
Chizpurfles are a problem mainly to wizards because they are attracted to and feed off of magic.
“Chizpurfles are small parasites up to a twentieth of an inch high, crablike in appearance, with large fangs. They are attracted by magic and may infest the fur and feathers of such creatures as Crups and Augureys. They will also enter wizard dwellings and attack magical objects such as wands, gradually gnawing their way through to the magical core, or else settle in dirty cauldrons, where they will gorge upon any lingering drops of potion.6”
- Fantastic Beasts and Where to Find Them
However, if they cannot find magic, they will affect Muggle electrical items.
“6 In the absence of magic, Chizpurfles have been known to attack electrical objects from within (for a fuller understanding of what electricity is, see Home Life and Social Habits of British Muggles, Wilhelm Wigworthy, Little Red Books, 1987). Chizpurfle infestations explain the puzzling failure of many relatively new Muggle electrical artifacts.”
- Fantastic Beasts and Where to Find Them
So, Chizpurfles are mainly but not exclusively a wizarding problem - they do affect Muggles at times.
Knarls:
Knarls are magical creatures that are similar to hedgehogs, except that they are highly suspicious of offered food and will attack the garden of those who offer it food. Knarls have often caused Muggle children to be accused of vandalism by the mess they create.
“The Knarl (northern Europe and America) is usually mistaken for a hedgehog by Muggles. The two species are indeed indistinguishable except for one important behavioural difference: If food is left out in the garden for a hedgehog, it will accept and enjoy the gift; if food is offered to a Knarl, on the other hand, it will assume that the householder is attempting to lure it into a trap and will savage that householder’s garden plants or garden ornaments. Many a Muggle child has been accused of vandalism when an offended Knarl was the real culprit.”
- Fantastic Beasts and Where to Find Them
That Muggle children are often blamed for a Knarl’s mess shows that Knarls do affect Muggle gardens as well.
Pests that it is unclear if they infest Muggles.
Bundimuns:
Bundimuns infest houses, but it is not mentioned whether they exclusively infest wizarding houses or may infest Muggle houses as well.
“Bundimuns are found worldwide. Skilled at creeping under floorboards and behind skirting boards, they infest houses. The presence of a Bundimun is usually announced by a foul stench of decay. The Bundimun oozes a secretion which rots away the very foundations of the dwelling in which it is found.
The Bundimun at rest resembles a patch of greenish fungus with eyes, though when alarmed it will scuttle away on its numerous spindly legs. It feeds on dirt. Scouring charms will rid a house of an infestation of Bundimuns, though if they have been allowed to grow too large, the Department for the Regulation and Control of Magical Creatures (Pest Sub-Division) should be contacted before the house collapses. Diluted Bundimun secretion is used in certain magical cleaning fluids.”
- Fantastic Beasts and Where to Find Them
Nothing is mentioned about Bundimuns that indicate whether they can ever be found in Muggle homes.
Glumbumbles:
Glumbumbles can infest beehives, but it is not mentioned if these are only beehives owned by wizards or Muggle-owned beehives as well.
“The Glumbumble (northern Europe) is a grey, furry-bodied flying insect that produces melancholy-inducing treacle, which is used as an antidote to the hysteria produced by eating Alihotsy leaves. It has been known to infest beehives, with disastrous effects on the honey. Glumbumbles nest in dark and secluded places such as hollow trees and caves. They feed on nettles.”
- Fantastic Beasts and Where to Find Them
They are known to ruin honey, but it is not said whether this includes honey produced by Muggle beehives.
Horklumps:
Horklumps infest gardens, but it is unclear if they infest Muggle gardens as well.
“The Horklump comes from Scandinavia but is now widespread throughout northern Europe. It resembles a fleshy, pinkish mushroom covered in sparse, wiry black bristles. A prodigious breeder, the Horklump will cover an average garden in a matter of days. It spreads sinewy tentacles rather than roots into the ground to search for its preferred food of earthworms. The Horklump is a favourite delicacy of gnomes but otherwise has no discernible use.”
- Fantastic Beasts and Where to Find Them
It is mentioned that gnomes, a wizarding-only pest, eat Horklumps, but that does not necessarily mean that Horklumps are also a wizarding-only pest.
Nogtails:
Nogtails infest farms and blight them, but it is not said if this is a problem exclusive to wizards.
“Nogtails are demons found in rural areas right across Europe, Russia, and America. They resemble stunted piglets with long legs, thick, stubby tails, and narrow black eyes. The Nogtail will creep into a sty and suckle an ordinary sow alongside her own young. The longer the Nogtail is left undetected and the bigger it grows, the longer the blight on the farm into which it has entered.
The Nogtail is exceptionally fast and difficult to catch, though if chased beyond the boundaries of a farm by a pure white dog, it will never return. The Department for the Regulation and Control of Magical Creatures (Pest Sub-Division) keeps a dozen albino bloodhounds for this purpose.”
- Fantastic Beasts and Where to Find Them
Nothing on the Nogtail indicates whether it ever affects Muggles or not.