The most iconic of Middle-earth questions:
Who or what was Tom Bombadil?
Was he an elf? Was he akin to Gandalf, and one of the wizardly stock? Was he something… else?
Tom Bombadil wasn't an elf: Frodo and Strider/Aragorn both know enough about elves at the point Tom Bombadil is introduced to recognize them on sight, and they both act and behave as though he was something foreign and strange.
Tom Bombadil was not a wizard (a member of the Istari): there were only five sent to protect and guide the races of Middle Earth during the Third Age: Saruman the White, Gandalf the Grey, Radagast the Brown, and the two wizards Alatar and Pallando.
However, Tom Bombadil displays several curious characteristics that can provide some guidance as to what he is or if he fits into the rest of the Middle Earth mythology:
One theory is that he's part of the so-called Ainur: direct representatives of Ilúvatar (the creator). Depending on how you look at it, Ainur are either gods and demigods or something similar to angels; but pantheism, where the Ainur are merely different aspects of Ilúvatar, more closely describes the mythology of Middle Earth.
The Ainur are broken into two groups:
There's speculation that he's a Maia, as he appears to be immortal, isn't a named Vala, and could feasibly be the agent of Yavanna (Vala of nature). However, other Maiar we're exposed to seem to be fully capable of succumbing to the One Ring's power in a way that Tom Bombadil isn't.
Other theories include that he's an incarnation of a Vala. However, it's not clear the Valar take incarnations: throughout Tolkien's works, Valar, when they want to interact, send agents or interact directly as themselves.
In reality, Tolkien never defines what Tom Bombadil is: not in The Lord of the Rings or in The Adventures of Tom Bombadil. The closest thing to a definition that Tolkien gives of Tom is that he's extremely old, and simply is.
To this end, the third and final theory, that Tom Bombadil is an incarnation of Ilúvatar himself, takes hold. Ilúvatar would definitely not be subject to the corruption of the One Ring, would be far more powerful and older than even the most powerful of the Ainur, and care little for the day-to-day struggles of the mortal and immortal races. Finally, the characteristic that Tom Bombadill simply is is similar to how Yahweh, the god of Abraham and Moses, characterizes himself: I am.
But it's best not to think of it too deeply. Tolkien was loath to create allegories and allusions, saying of The Lord of the Rings:
As for any inner meaning or 'message', it has in the intention of the author none. It is neither allegorical nor topical. [...]
But I cordially dislike allegory in all its manifestations, and have done so since I grew old and wary enough to detect its presence. I much prefer history, true or feigned, with all its varied applicability to the thought and experience of readers.
Out of universe, Tolkien has two remarks published in Letters that explain who Bombadil is meant to be; the first in Letter 19:
Tom Bombadil, the spirit of the (vanishing) Oxford and Berkshire countryside...
And the second in Letter 144, first the background (excerpting quite heavily):
Tom Bombadil is not an important person – to the narrative ... he represents something that I feel important, though I would not be prepared to analyze the feeling precisely ... the story is cast in terms of a good side, and a bad side ... but both sides ... want a measure of control.
And then Tom's role in all of this:
But if you have, as it were taken 'a vow of poverty', renounced control, and take your delight in things for themselves without reference to yourself, watching, observing, and to some extent knowing, then the question of the rights and wrongs of power and control might become utterly meaningless to you, and the means of power quite valueless. It is a natural pacifist view, which always arises in the mind when there is a war.
In universe, we know he's not Eru from Letter 153, first of all from the intro:
He also cited the description of Bombadil by Goldberry: 'He is.' Hastings said that this seemed to imply that Bombadil was God.
Then Tolkien's rebuttal (I've left out a lot here and refer you to Letters for the full discussion):
As for Tom Bombadil, I really do think you are being too serious, besides missing the point. (Again the words used are by Goldberry and Tom not me as a commentator) ... Frodo has asked not 'what is Tom Bombadil' but 'Who is he' ... Only the firstperson (of worlds or anything) can be unique. If you say he is there must be more than one, and created (sub) existence is implied. I can say 'he is' of Winston Churchill as well as of Tom Bombadil, surely?
Aside from the "enigma" quote in Letter 144, there is another source for who Bombadil may be, and that's given in the published Silmarillion, chapter 2, which itself was derived from a late essay called Of the Ents and the Eagles:
Behold! When the Children awake, then the thought of Yavanna will awake also, and it will summon spirits from afar, and they will go among the kelvar and the olvar, and some will dwell therein, and be held in reverence...
This is IMO a credible source. Tom is evidently a nature spirit, and this source explains the origin of nature spirits in Middle-earth (note also that only "some will dwell therein"; evidently there are others that won't).
The only thing left out is Tom's claim to be "Eldest", and here I'm going to refer to a passage from Letter 153 which I quoted earlier and will quote again:
the words used are by Goldberry and Tom not me as a commentator.
This is important, and it must be emphasised: for all of the claims made about Tom, Goldberry is saying them, Tom is saying them, but Tolkien is not. These are words spoken and claims made by characters in Tolkien's world, not by Tolkien himself. This is a point that Tolkien returns to in the context of Treebeard, also in Letter 153:
Treebeard is a character in my story, not me; and though he has a great memory and some earthy wisdom, he is not one of the Wise, and there is quite a lot he does not know or understand.
There's a clear distinction being made here, and that is that words spoken or claims made by a character in the story should not be seen as a definitive authorial statement, and the same should be borne in mind when considering Tom.
Here is a pretty in depth essay on the possible identity of Tom Bombadil, and it also has a lot of interesting links: Who is Tom Bombadil? by Gene Hargrove.
The conclusion is that, if Tom is to be more than an enigma, Tom is either a nature spirit, or is Aule, one of the Valar(or a manifestation thereof). His mastery over the ring and lack of a desire to possess earthly power or things is offered in support of the latter position. After a lenghty examination, the bottom line taht the essay offers is this:
If Tom is Aule, however, there is a moral dimension, indeed, a heightened one, for Tom's appearance in the story, although only a "comment," serves as a sharp and clear contrast to the two evil Maiar, Sauron and Saruman, both of whom were once his servants before turning to evil and darkness. Unlike their former master, these two followed the ways of Melkor, envy, jealousy, excessive pride, and the desire to possess and control. As Tolkien explained to his proofreader, Tom's role was to show that there were things beyond and unconcerned with domination and control. On the surface, this view of Tom seems to make him unrelated to all other things and events in Middle-earth - indeed, anomalous. As Aule, however, Tom is not beyond and unconcerned anomalously, but rather is located at the core of morality as it existed in Middle-earth, as the ultimate exemplification of the proper moral stance toward power, pride, and possession. In fact, in terms of the moral traits that most fascinated Tolkien both as an author and as a scholar, Tom Bombadil is Tolkien's moral ideal.
I thought it was particularly interesting that while Tolkien wanted to leave some enigmas in his writing, he also probably had an explanation for himself since he liked to reconcile any "errors". So the fact that Tom Bombadil was conceived as a separate story and later integrated into LotR doesn't mean that he didn't have an idea of how Tom fits into the overall universe.
I believe, rather fancifully, that Bombadil is simply the spirit of middle earth manifest as a sentient, magical being. He might have acted as guardian of the ancient places and as these places diminished so did Bombadil's influence. However, he will always be as unassailable and unknowable as the very earth itself!
It strikes me as a surprise that Oldest and Fatherless: The Terrible Secret of Tom Bombadil, one of the more interesting fan theories about Bombadil -- written by a certain livejournal user, km_515 -- isn't mentioned here yet. So, here it goes. All in all, this obviously unofficial theory says Bombadil is an ancient, almost timeless evil rivaling Sauron & Co., biding its time until it can reclaim what it once ruled. Let me give you a quote:
The spell that binds Bombadil to his narrow and cursed country was put in place centuries ago by the Valar to protect men and elves. It may last a few decades more, perhaps a few generations of hobbit lives. But when the last elf has gone from the havens and the last spells of rings and wizards unravel, then it will be gone. And Iarwain Ben-Adar, Oldest and Fatherless, who was ruler of the darkness in Middle Earth before Sauron was, before Morgoth set foot there, before the first rising of the sun, will come into his inheritance again. And one dark night the old trees will march westward into the Shire to feed their ancient hatred. And Bombadil will dance down amongst them, clad in his true shape at last, singing his incomprehensible rhymes as the trees mutter their curses and the black and terrible Barrow-Wights dance and gibber around him. And he will be smiling.
Do click the link and read the whole thing for the rather intriguing reasoning. Keep in mind though, that its author also writes:
Do I think that Tolkien planned things in this way? Not at all, but I find it an interesting speculation.
None of the other answers mention this, but I feel it's important to note that Tom Bombadil was originally a Dutch doll belonging to one of Tolkien's children, Michael. One day Michael threw it in the toilet, but it was rescued and it became the protagonist of a story fragment Tolkien invented for his children. He later wrote a poem about him called ‘The Adventures of Tom Bombadil’ (1934), long before the writing of the Lord of the Rings began.
tba tokiengateway.net – Tom Bombadil
tba Who or what was Tom Bombadil?
And even in a mythical Age there must be some enigmas, as there always are. Tom Bombadil is one (intentionally).
That said, I think we can show what Bombadil is not.
Tom is not a man, hobbit, elf, or dwarf
...or any other sort of "normal" creature.
The ring had no effect on him.
Tom is not a Maia
Gandalf was also a Maia but he was tempted by the Ring. Likewise, Saruman, the Balrog, and Sauron were able to seduced by the Darkness.
In contrast, Tom has little interest in the affairs or "politics" of Middle Earth.
Tom is not a Valar
As far as it is told, the Valar live exclusively in Valinor. After the destruction of the Two Lamps, the Valar never interacted directly with the affairs of Middle Earth. And after Númenóreans reached Valinor, it was removed from the rest of Arda.
Tom is not Eru
Tom was not all-powerful, at least according to the Rivendell elves:
"But in any case," said Glorfindel, "to send the Ring to him would only postpone the day of evil. He is far away. We could not now take it back to him, unguessed, unmarked by any spy. And even if we could, soon or late the Lord of the Rings would learn of its hiding place and would bend all his power towards it. Could that power be defied by Bombadil alone? I think not. I think that in the end, if all else is conquered, Bombadil will fall, Last as he was First; and then Night will come."
-Fellowship of the Ring
Of course, Glorfindel could be wrong.
But Tom as Eru would be rather inconsistent with other manifestations of divine beings. Generally, the more powerful the being, the further removed from Middle Earth he is. The wizards (Maia) could not contend with Sauron directly. The more powerful Valar -- Melkor included -- did not live in Middle Earth at all.
Though Tom and his origins may be deliberately an enigma, that doesn't mean there aren't clues painting a vague idea.
Recall that Glorfindel's said he would be "Last as he was First".
In the same meeting, Galdor says,
Glorfindel, I think, is right. Power to defy our Enemy is not in him, unless such power is in the earth itself.
So the clues:
Tom Bombadil is an incarnation of Arda, or perhaps a steward of Arda, or of Middle Earth. He's a "Mother Earth"/"Father Time" figure of sorts. We can't know exactly, since it is supposed to a mystery. But that's the gist. Human emotions have no more effect on him than they do on a tree, or rock. He is as old or nearly as old as the Earth itself, he enjoys nature, and he cares for it. And he has been there for a very, very long time.
EDIT: I found this essay on Tom Bombadil by Gene Hargrove, which describes him similarly, as a Pan-like "nature god".
Tolkien - in one of his letters - specifically stated that Bombadill is not Illuvatar. There is a reference in Tolkien's earlier writings to nature spirits (Fays, Leprawns) and such, and my own feeling is that Bombadill is one of these - a being as old as the world whose purpose may occasionally intersect those of the Valar (and in this case Sauron), but who works out his own task under Eru and takes little part in the struggles of Elves, Men and demons. Tom's remit is to do with the Earth itself and his power and remit are dealt according to that. He is outside the 'politics' of the peoples of the Earth, but not immune should Sauron crush nature itself. Whilst not a maia undoubtedly Yavanna (as instructed by Eru) would know of Bombadill, his fellows and their modus operandii. One of the mischevious folk; seemingly anarchic and a law unto themselves, but built into the fabric of the world since its beginnings by Eru (or a byproduct of the dissonnances between Melkor's discord and Eru's harmony in the Great Music, but still extant by the sovereign will of Eru Himself and given purpose by Him).
Bombadil has to be a Maia. He himself said he was alive before the Big People. Who are elves and men. Tolkien said Bombadil wasn't Eru in his letters. There was nothing else in the Silmarillion before the elves besides Eru, the Valar, and the Maiar.
Eldest, that's what I am. Mark my words, my friends: Tom was here before the rivers and the trees; Tom remembers the first raindrop and the first acorn. He made paths before the Big People, and saw the little People arriving. He was here before the Kings and the graves and the Barrow-wights. When the Elves passed westward, Tom was here already, before the seas were bent. He knew the dark under the stars when it was fearless - before the Dark Lord came from Outside.
~ The Fellowship of the Ring
I would like to add a theory: Tom Bombadil is Tolkien's ego.
From a psychological standpoint, Tolkien delved deep into his unconscious to create the story. He didn't direct it intentionally, per se, but let it tell itself.
Even here, though, Tolkien('s ego) is still master, and could do anything. In order to leave the Hobbit/LotR universe as is he could not get too involved, but he could visit it, and it would have no power over him. He could also help when asked. Though he takes no action on his own.
I had this in mind when listening to the book and found it to make sense to me.
Clearly Tolkien's intention was to have Tom be a mystery, but if we are allowing an interpretation outside of Tolkien's intention, I have a theory to suggest that I maintain accords with everything in the texts, and which no one else here has yet suggested: Tom was Iluvatar's prototype, which Iluvatar created in Arda for practice before he created the Children. He has a form akin to that of Men or Elves, but not all their features, or frailties. He is the eldest of all things created in Arda, and is tied to it, but is unique. Goldberry and her father the River and similar creatures may be spirits summoned by the awakened thought of Yavanna, but Tom is distinct. His power comes mainly from being the eldest; he was created before all the rules of nature were finalized and stands a little outside them.
I think Tom is an embodiment of the music of the Ainur. All his power derives from song - he has a song for old man willow, and one for the Barrow Wight. 'Tom's songs are stronger songs...' When Goldberry is asked who he is she answer 'He is....' This is an echo of Iluvatar's "Ea' when he creates Middle earth from the music of the Ainur.
Yes, a lot of comments on the right track.He was one of those "lesser and greater spirits"/ "other orders" sent into Arda. He is the Green Man, Pan, Kernennos (Herne). Eru is the Universal Creator. Arda is just one amongst millions of worlds created in Ea The Universe. Eru has created lesser Gods and each world has a life-spirit/Earth-God supported by elementals, just as Eru in Heaven is supported by the Ainur in their two orders, Valar & Maiar. Goldberry is one of those elementals (water). Eru created lesser Gods (Earth-bound-Demi-Gods), Ainur (Archangels/His Executive Cabinet), Maiar (Angels/The Divine Administration/Government/Bureaucracy)and other orders of Agents/Operatives (Istari etc) as agents of creation. Certain of them tasked with the agency of sub-creation in different levels of power/authority. Eru has delegated/decentralised some of his creative powers in greater and lesser amounts according to the roles and capacities of his subjects, this trickling down to humans who, although it is limited have been gifted with free will/agency to act as agents of God's Creation.