To answer this question it's necessary to dispel one common misconception: Middle-earth is actually an incredibly high-magic setting: just look at all the magic items that the Fellowship are laden-down with, the constant references to spells, the magical realms they pass through, and so forth.
There's further discussion (and evidence) of that here, but for the purposes of this question it's just necessary to first establish that "Middle-earth is low-magic" is quite false, so I'll just give you a quote and refer you to the full article for more information.
J.R.R. Tolkien created huge heaping treasures of magic items, piled them up, and set dragons to guard them. He raised entire cities around groups of Elven and Dwarven craftsmen who minted, built, constructed, manufactured, conjured, or otherwise produced factory-style volumes of magical cloaks, lanterns, musical instruments, weapons, armor, and maybe even shoes. You cannot find a story about Middle-earth where Tolkien doesn't have someone do something magical. He has sleep spells, disease spells, spells of far-farsightedness, curses, counter-curses, healing magic, and Elves can run on snow.
So the assertion that "as Middle Earth is a very low-magic setting, gunpowder would not be made obsolete by magic, as magic hardly exists at all" is instantly disproven, because:
- Middle-earth is actually a very high-magic setting,
- Gunpowder can easily be made obsolete (and even unnecessary) by magic because,
- Magic exists around every corner and under every stone.
Despite this it seems valid to still ask "why isn't gunpowder more common", but there needn't be a significant reason for this. The ancient Chinese had gunpowder, fireworks and blasting devices, but yet it wasn't hugely common there either - it was invented entirely by accident, required specialized alchemical knowledge, handling of volatile and dangerous ingredients, and its use as a weapon was a later development.
The History Stack Exchange site has a relevant question and accepted answer about the use of gunpowder in ancient and medieval history that's worth reading in this context.
Edit: 19th April 2014
The user vsz has raised some objections to this answer and it's necessary to clarify. In actual fact there are two (if not more) different types of "magic" in Tolkien's work, but Tolkien - confusingly - used the word "magic" for both of them.
Tolkien clearly expresses this in Letters 131 (written before LotR was published) and 155 (written after), so I'm going to quote from both of them here; first of all his acknowledgement of this confusion.
I have not used 'magic' consistently... (Letter 131)
I am afraid I have been far too casual about 'magic' and especially the use of the word... (Letter 155)
In Letter 131 he defines the two types:
and so to the Machine (or Magic). By the last I intend all use of external plans or devices (apparatus) instead of development of the inherent inner powers or talents — or even the use of these talents with the corrupted motive of dominating: bulldozing the real world, or coercing other wills. The Machine is our more obvious modern form though more closely related to Magic than is usually recognized.
...the Elves are there (in my tales) to demonstrate the difference. Their 'magic' is Art, delivered from many of its human limitations: more effortless, more quick, more complete (product, and vision in unflawed correspondence). And its object is Art not Power, sub-creation not domination and tyrannous re-forming of Creation.
In the latter quote note especially the words "demonstrate the difference".
Letter 155 again draws the distinction, and particularly notes that the types are not necessarily differentiated into "good" or "evil":
...some would say that there is a latent distinction such as once was called the distinction between magia and goeteia ... but magia could be, was, held good (per se), and goeteia bad. Neither is, in this tale, good or bad (per se), but only by motive or purpose or use. Both sides use both, but with different motives. The supremely bad motive is domination of other 'free' wills. The Enemy's operations are by no means all goetic deceits, but 'magic' that produces real effects in the physical world. But his magia he uses to bulldoze both people and things, and his goeteia to terrify and subjugate. Their magia the Elves and Gandalf use (sparingly): a magia, producing real results (like fire in a wet faggot) for specific beneficent purposes. Their goetic effects are entirely artistic and not intended to deceive: they never deceive Elves (but may deceive or bewilder unaware Men) since the difference is to them as clear as the difference to us between fiction, painting, and sculpture, and 'life'.
When discussing magic in Middle-earth it's extremely important to acknowledge that both types exist. When speaking of "magic as Art" you're not speaking of the entirety of "magic" in Middle-earth: you're only speaking of one type. When speaking of all of the magic items that the Fellowship are laden down with, you're speaking of the other type. The existence of one type doesn't prevent the existence of the other type, and there is no contradiction.