This is covered more in-depth in the comic. See Issue 83. It's disgusting and makes people physically sick from the stench. It also plays havoc on your focus and makes people on edge.
Ever smelled a small dead animal? How pungent and awful even a roadkill can be? Imagine that on a larger scale and being literally inside it. That's why.
SPOILER ALERT if you click on the link and are not caught up.
EDIT: The YouTube video of the comic issue was taken down. The relevant information is in the spoiler block below.
Douglas is "covering" them by continuing to shoot wildly and Rick tries to yell out to him to stop shooting. Douglas is attracting the undead, gets bitten several times, and he stops firing. Rick asks if everyone is okay and then asks Carl. When he does not answer, Rick looks down at Carl, only to discover that Carl has been shot in the eye.
NEW EDIT - 08-31-15
This info pertains only to the comic so far, but is relevant to the question. Spoilers!
There is now, in the comic, a huge group of people who dress up in the flesh of the dead, and walk among them. Their disguise protects them. It should also be noted that for all practical reasons, these people, or at least their leader, Alpha, seems to be insane or at least a proponent of radical devolution. They wear the flesh of the dead to walk among them, but also seem to lose what it means to be human. Or at least a fairly regular person. These "Whisperers" seem to be further away from Rick and his group than the band Right Said Fred is away from Slayer.