While reading this answer on who/what Tom Bombadil was, I started wondering what Tolkien's intention for him was and why he was created the way he was? So, specifically
So, it's not about his purpose in the book and how he helps the plot (or not, which is answered here) but why Tolkien included him anyway.
Tolkien said that Bombadil represented a sort of passive pacifism, which was important to represent in the story but couldn't play much of a role in the actual plot.
From Tolkien's Letters, letter #144:
Tom Bombadil is not an important person – to the narrative. I suppose he has some importance as a 'comment'. I mean, I do not really write like that: he is just an invention (who first appeared in the Oxford Magazine about 1933), and he represents something that I feel important, though I would not be prepared to analyze the feeling precisely. I would not, however, have left him in, if he did not have some kind of function. I might put it this way. The story is cast in terms of a good side, and a bad side, beauty against ruthless ugliness, tyranny against kingship, moderated freedom with consent against compulsion that has long lost any object save mere power, and so on; but both sides in some degree, conservative or destructive, want a measure of control. but if you have, as it were taken 'a vow of poverty', renounced control, and take your delight in things for themselves without reference to yourself, watching, observing, and to some extent knowing, then the question of the rights and wrongs of power and control might become utterly meaningless to you, and the means of power quite valueless. It is a natural pacifist view, which always arises in the mind when there is a war. But the view of Rivendell seems to be that it is an excellent thing to have represented, but that there are in fact things with which it cannot cope; and upon which its existence nonetheless depends. Ultimately only the victory of the West will allow Bombadil to continue, or even to survive. Nothing would be left for him in the world of Sauron.
In Letter #20, Tolkien also reveals the out-of-universe inspiration for Bombadil:
Do you think Tom Bombadil, the spirit of the (vanishing) Oxford and Berkshire countryside, could be made into the hero of a story? Or is he, as I suspect, fully enshrined in the enclosed verses?
Note: this answer is copied from my answer to the same question on another site.
