From what I understand, The Lord of the Rings is written as the mythic past of our own world. We are constantly reminded that the race of man is becoming more and more prominent. At the end of The Return of the King the Elves leave Middle-earth to the humans. But what about the other mythical races, primarily the Hobbits and the Dwarves? Did they die out?
According to the prologue of the Lord of the Rings under the section Concerning Hobbits, the Hobbits are still with us today, but "now they avoid us with dismay and are becoming hard to find."
In Appendix A of the Lord of the Rings under section III, Durin's Folk, the following is mentioned as a possible reason for the lack of Dwarves:
There's no mention of the ultimate fate of the Dwarves, but it's easy to infer that Dwarves could not maintain a positive population growth rate.
First of all, to clarify a misconception:
No. LOTR setting is a totally different version of our world called "Arda", NOT "past".
Here's a supporting Tolkien quote I found on one forum:
So, dwarves and hobbits didn't "exist in the past" as per LOTR - they existed in different "mythology"
Also, as far as proposed reconciliation of hobbits and modern history, Encyclopedia of Arda has to say this:
Although nothing is said in LotR, HoME 12 (The Peoples of Middle Earth) has this to say about the eventual fate of the Dwarves:
A further revision brings the text to:
Both of these passages are from original drafts of the "Durin's Folk" appendix, and while neither were taken up in the published text (and so their status as a canon or final decision on the matter must be considered dubious) they do imply that the Dwarves did eventually die out.
What was taken up in the published text was the concept of "Durin VII (and last)" who was a descendant of Dain Ironfoot and lived sometime in the Fourth Age, which seems to indicate that at least some of the concept of the Dwarves dying out was retained.
That however is not everything, for the Silmarillion also has this to say:
So even assuming that the draft texts were never explicitly abandoned (but rather omitted for reasons of space, forgetfulness, or whatever) and assuming that the Dwarves did die out, then they will return at the End of Time and help rebuild the world.
One final passage, this time from Of The Rings of Power and the Third Age (published in the Silmarillion) may be mentioned:
Although not a conclusive statement, this can also be read as supporting the theory that the Dwarves died out, and when all the evidence is taken together it seems at least quite likely that this is what actually happened.
As I've understood it, with reference to the Dawn of Man / the Fourth Age, it seems that their supreme being (Eru) saw that Men would take over the world, so he willingly removed his mythical races from Middle Earth and delivered them to Valinor.