As it was originally tagged LoTR dwarfs:
Lord of The Rings
In The War of the Jewels , Tolkien wrote:
"no Man nor Elf has ever seen a beardless Dwarf - unless he were shaven in mockery, and would then be more like to die of shame... For the Naugrim have beards from the beginning of their lives, male and female alike..."
And to add now that the tag has been removed:
Nearly every writer has their own idea of Dwarfs, Dwarves and Mountain Dwellers! So to find reference to the way each idea of a Dwarf is born, would be madness, however, I'll give it a bash from the settings that I have seen, read etc.
No mention in the lore about Dwarf babies, there is a miniature of a Dwarven female villager holding a baby, however the face of the wee thing can't be seen and these were only made in the 1980s.
I don't recall any mention of babies at all in Discworld. The only reason for mentioning anything in the books is about Women distinguishing themselves as such in society rather than just being "A Dwarf".
D&D, Forgotten Realms
Generally, I think it's a matter of interpretation. If you're reading/watching something based on Tolkien's version of Dwarfs then one should assume that they are born with beards. Otherwise, in stories where it's not necessary for Dwarfs to have beards and some are known to trim, style or completely shave off their beard; it can be assumed that there isn't much emphasis on the beardyness and it's doubtful that they are born with them.