I think that this and even more so this link explain the issue quite well.
To the latter I might add that an axe makes for a much better chopping weapon than a sword, making it more usable for the rank-and-file soldier, especially given dwarves' archetypical stature, lack of fine dexterity and any willingness for exquisite swordplay.
There are other circumstances as well. Armored target? Axe/falchion/morningstar is better than a general sword. Unarmored target? Go fetch your scimitar. Unprofessional/clumsy/heavily armored wielder? Forget about sword, axe is easier to use. Funds are limited? Forget about sword, it costs a fortune.
Side note: Tolkien's dwarves wielded swords just fine sometimes (Torin, for example).
EDIT: as per request, I include the quote from the second link in the post:
Useful for chopping wood and heads alike, axes are often pressed into service as weapons. An axe's wide, cleaving head is effective against armor, making it a slower and more powerful weapon than a sword, though not quite as massive as a hammernote . Smaller axes, such as the tomahawk or the francisca, can be used as throwing weapons. In certain media, they may even return to the user's hand like a boomerang, though they have the tendency to stick in enemies if they score a hit.
Historically, due to being somewhat easier to manufacture, their general familiarity to the peasant population and mobility (real-life combat axes were small and rarely weighed more than one kilo), they used to be the close combat Weapon of Choice of the common foot soldiers, as opposed to the officers and nobility, who wore swords (which were generally too expensive for commoners). While military weapons were often illegal for peasants to own, axes are tools, and avoid the prohibition. Franks used them extensively (lending their name to the francisca), and nowadays axes are associated with dwarves, Orcs, Vikings (and other members of proud warrior races), barbarians, medieval executioners, woodcutters, firefighters, and psychopaths (giving Ax Crazy its name and meaning). If the Big Guy of a Five-Man Band isn't using a hammer or his bare fists, he'll generally use a gigantic axe as a weapon. For extra comedy, the Cute Bruiser may be given a head-chopper bigger than the rest of her body.
May also include halberds and other poleaxes, which combine the chopping power of an axe with the range of a Blade on a Stick.