Why did Gandalf go out of time?

Question

Given that the Ainur, like the Elves, were bound to the universe as long as it lasted, only Men being able to escape (the Gift of Men), why on the death of his Earthly form did Gandalf leave time and get sent back by Eru, rather than simply travel to the Halls of Mandos for healing and reclothing in Valinor?

'Naked I was sent back — for a brief time, until my task is done'. Sent back by whom, and whence? Not by the 'gods' whose business is only with this embodied world and its time; for he passed 'out of thought and time'. Naked is alas! unclear. It was meant just literally, 'unclothed like a child' (not disincarnate), and so ready to receive the white robes of the highest. Galadriel's power is not divine, and his healing in Lórien is meant to be no more than physical healing and refreshment."

[The Letters of J.R.R. Tolkien, (#156)]

Answer 1

We don't know precisely, but it was related to his embodiment.

Death in Middle-earth

In Tolkien's legendarium, death has a very particular meaning: it's the severance of the spirit (fëa) and body (hröa). This is true of both Elves and Men; in fact, without special knowledge, the death of an Elf looks identical to the death of a Man:

[I]n the days when the minds of the Eldalië were young, and not yet fully awake death among them seemed to differ little from the death of Men.

[...]

It was in Aman that they learned of Manwë that each fëa was imperishable within the life of Arda, and that its fate was to inhabit Arda to its end.

^{_{History of Middle-earth X Morgoth's Ring Part 2 "The Second Phase" Chapter 3: "Laws and Customs Among the Eldar" Of Death and the Severance of Fëa and Hröndo [> Hröa]}}

Of course this definition doesn't make sense when discussing beings like the Maiar, because their bodies aren't critical parts of their being:

[T]heir shape comes of their knowledge of the visible World, rather than of the World itself; and they need it not, save only as we use raiment, and yet we may be naked and suffer no loss of our being.

^{_{The Silmarillion I Ainulindalë}}

So when we say that the Valar and Maiar can't die, that's what we mean.

What about the Istari?

The wizards were different:

For with the consent of Eru they [the Valar] sent members of their own high order, but clad in bodies of as of Men, real and not feigned, but subject to the fears and pains and weariness of earth, able to hunger and thirst and be slain; though because of their noble spirits they did not die, and aged only by the cares and labours of many long years.

^{_{Unfinished Tales Part 4 Chapter II: The Istari}}

He phrases this differently in Letter 156:

By 'incarnate' I mean [the Istari] were embodied in physical bodies capable of pain, and weariness, and of afflicting the spirit with physical fear, and of being 'killed', though supported by the angelic spirit they might endure long, and only show slowly the wearing of care and labour.

^{_{The Letters of J.R.R> Tolkien 156: To Robert Murray, SJ (Draft). 4 November 1954}}

The explicit identification of Gandalf as an "incarnate" is interesting, because that's what Elves and Men are. The implication, then, is that the angelic spirit of Olórin the Maia was somehow tethered to the body of Gandalf. The precise mechanics of how this works is unclear, and we don't have a good idea of what would have happened to Olórin if Gandalf had not been resurrected, but there is something fundamentally different between the two beings. Death means something different to Gandalf than to Olórin, though we don't know what.

Okay, so why did Gandalf leave Eä?

This may be a controversial opinion, but I don't think he did. Consider how Gandalf describes his death:

I threw down my enemy, and he fell from the high place and broke the mountain-side where he smote it in his ruin. Then darkness took me; and I strayed out of thought and time, and I wandered far on roads that I will not tell.

^{_{The Two Towers Book III Chapter 5: "The White Rider"}}

When people suggest that Gandalf left Eä upon his death, they usually point to the line "I strayed out of thought and time"¹. However, I find it interesting how the word "time" is spelt: it's not capitalized. Now consider two excerpts from The Silmarillion:

Vairë the Weaver is [Mandos'] spouse, who weaves all things that have ever been in Time into her storied webs, and the halls of Mandos that ever widen as the ages pass are clothed with them.

^{_{The Silmarillion II Valaquenta}}

And:

[T]he sons of Men die indeed, and leave the world; wherefore they are called the Guests, or the Strangers. Death is their fate, the gift of Ilúvatar, which as Time wears even the Powers shall envy.

^{_{The Silmarillion III Quenta Silmarillion Chapter 1: "Of the Beginning of Days"}}

Note how "time" is capitalized here. It seems to me that "Time", in Tolkien, is a reference to the material universe; thus I submit that there's a difference between Gandalf saying "I strayed out of thought and time" and "I strayed out of thought and Time." I don't think it's a coincidence that he chose the former.

Finally, consider the death of Saruman:

To the dismay of those that stood by, about the body of Saruman a grey mist gathered, and rising slowly to a great height like smoke from a fire, as a pale shrouded figure it loomed over the Hill. For a moment it wavered, looking to the West; but out of the West came a cold wind, and it bent away, and with a sigh dissolved into nothing.

^{_{Return of the King Book VI Chapter 8: "The Scouring of the Shire"}}

Since Gandalf and Saruman are the same class of being (incarnate angels, as Tolkien puts it), presumably the rules governing what "death" means to them are similar. Indeed, the main difference between their deaths are the circumstances. And the implication given by Saruman's death is that he wanted to return to Valinor², but was forbidden, most likely due to his many failures as an agent of the Valar.

So, to (finally) answer the question: why did Gandalf leave the universe when he died?

I don't think he did.

---

¹ You could perhaps also make a case for "I wandered far on roads that I will not tell", but I don't find that particularly compelling. Gandalf is vague throughout telling his story, and ascribing special meaning to one instance of vagueness seems troubling to me.

² Indeed, Tolkien says that the desire to return home was a principle motivator of all Istari:

[T]hough they knew whence they came the memory of the Blessed Realm was to them a vision from afar off, for which (so long as they remained true to their mission) they yearned exceedingly. Thus by enduring of free will the pangs of exile and the deceits of Sauron they might redress the evils of that time.

^{_{Unfinished Tales Part 4 Chapter II: The Istari}}

Answer 2

The only answer is found in the same letter you quoted. Here is a larger portion of the letter:

"For in his condition it was for him a sacrifice to perish on the Bridge in defence of his companions, less perhaps than a mortal Man or Hobbit, since he had a far greater inner power than they; but also more, since it was a humbling and abnegation of himself in confirmity to 'the Rules': for all he could know at that moment he was the only person who could direct the resistance to Sauron successfully, and all his mission was in vain. He was handing over to the Authority that ordained the Rules, and giving up personal hope of success.

...So Gandalf sacrificed himself, was accepted, and enhanced, and returned. 'Yes, that was the name. I was Gandalf.' Of course, he remains similar in personality and idiosyncrasy, but both his wisdom and power are much greater. When he speaks he commands attention; the old Gandalf could not have dealt so with Théoden, nor with Saruman. He is still under the obligation of concealing his power and teaching rather than forcing or dominating wills, but where the physical powers of the Enemy are too great for the good will of the opposers to be effective he can act in emergency as an 'angel' - no more violently than the release of St. Peter from prison....

Gandalf really 'died', and was changed: for that seems to me the only real cheating, to represent anything that can be called 'death' as making no difference... He was sent by a mere prudent plan of the angelic Valar or govenors; but Authority had taken up this plan and enlarged it, at the moment of its failure. 'Naked I was sent back - for a brief time, until my task is done'. Sent back by whom, and whence? Not by the 'gods' whose business is only with this embodied world and its time; for he passed 'out of thought and time'. Naked is alas! unclear. It was meant just literally, 'unclothed like a child' (not disincarnate), and so ready to receive the white robes of the highest. Galadriel's power is not divine, and his healing in Lórien is meant to be no more than physical healing and refreshment."

• The Letters of J.R.R. Tolkien, #156

The "Authority" (always capitalized) of whom Tolkien speaks can be no one other than Eru Ilúvatar. In this case, as in all others, when Eru exercises His power, it is because this is how He has determined that things should be. Eru sent Gandalf back to Middle-earth because He always intended to do so. This was always His plan. Gandalf is the one chosen by Eru to oversee the destruction of Sauron, and Gandalf's decision to sacrifice himself to save the Fellowship is the act that made him worthy of resurrection in Eru's eyes.

This is a function of Tolkien's deeply held religious convictions. To him, Eru is essentially another name for the Christian god. It probably wouldn't make sense to Tolkien to ask why god does what he does. Gandalf's resurrection was what Tolkien referred to as a "eucatastrophe":

For it I coined the word 'eucatastrophe': the sudden happy turn in a story which pierces you with a joy that brings tears (which I argued it is the highest function of fairy-stories to produce). And I was there led to the view that it produces its peculiar effect because it is a sudden glimpse of Truth, your whole nature chained in material cause and effect, the chain of death, feels a sudden relief as if a major limb out of joint had suddenly snapped back. It perceives – if the story has literary 'truth' on the second plane (for which see the essay) – that this is indeed how things really do work in the Great World for which our nature is made. And I concluded by saying that the Resurrection was the greatest 'eucatastrophe' possible in the greatest Fairy Story – and produces that essential emotion: Christian joy which produces tears because it is qualitatively so like sorrow, because it comes from those places where Joy and Sorrow are at one, reconciled, as selfishness and altruism are lost in Love.

• ibid, #89