We don't know precisely, but it was related to his embodiment.
Death in Middle-earth
In Tolkien's legendarium, death has a very particular meaning: it's the severance of the spirit (fëa) and body (hröa). This is true of both Elves and Men; in fact, without special knowledge, the death of an Elf looks identical to the death of a Man:
[I]n the days when the minds of the Eldalië were young, and not yet fully awake death among them seemed to differ little from the death of Men.
It was in Aman that they learned of Manwë that each fëa was imperishable within the life of Arda, and that its fate was to inhabit Arda to its end.
History of Middle-earth X Morgoth's Ring Part 2 "The Second Phase" Chapter 3: "Laws and Customs Among the Eldar" Of Death and the Severance of Fëa and Hröndo [> Hröa]
Of course this definition doesn't make sense when discussing beings like the Maiar, because their bodies aren't critical parts of their being:
[T]heir shape comes of their knowledge of the visible World, rather than of the World itself; and they need it not, save only as we use raiment, and yet we may be naked and suffer no loss of our being.
The Silmarillion I Ainulindalë
So when we say that the Valar and Maiar can't die, that's what we mean.
What about the Istari?
The wizards were different:
For with the consent of Eru they [the Valar] sent members of their own high order, but clad in bodies of as of Men, real and not feigned, but subject to the fears and pains and weariness of earth, able to hunger and thirst and be slain; though because of their noble spirits they did not die, and aged only by the cares and labours of many long years.
Unfinished Tales Part 4 Chapter II: The Istari
He phrases this differently in Letter 156:
By 'incarnate' I mean [the Istari] were embodied in physical bodies capable of pain, and weariness, and of afflicting the spirit with physical fear, and of being 'killed', though supported by the angelic spirit they might endure long, and only show slowly the wearing of care and labour.
The Letters of J.R.R> Tolkien 156: To Robert Murray, SJ (Draft). 4 November 1954
The explicit identification of Gandalf as an "incarnate" is interesting, because that's what Elves and Men are. The implication, then, is that the angelic spirit of Olórin the Maia was somehow tethered to the body of Gandalf. The precise mechanics of how this works is unclear, and we don't have a good idea of what would have happened to Olórin if Gandalf had not been resurrected, but there is something fundamentally different between the two beings. Death means something different to Gandalf than to Olórin, though we don't know what.
Okay, so why did Gandalf leave Eä?
This may be a controversial opinion, but I don't think he did. Consider how Gandalf describes his death:
I threw down my enemy, and he fell from the high place and broke the mountain-side where he smote it in his ruin. Then darkness took me; and I strayed out of thought and time, and I wandered far on roads that I will not tell.
The Two Towers Book III Chapter 5: "The White Rider"
When people suggest that Gandalf left Eä upon his death, they usually point to the line "I strayed out of thought and time"1. However, I find it interesting how the word "time" is spelt: it's not capitalized. Now consider two excerpts from The Silmarillion:
Vairë the Weaver is [Mandos'] spouse, who weaves all things that have ever been in Time into her storied webs, and the halls of Mandos that ever widen as the ages pass are clothed with them.
The Silmarillion II Valaquenta
[T]he sons of Men die indeed, and leave the world; wherefore they are called the Guests, or the Strangers. Death is their fate, the gift of Ilúvatar, which as Time wears even the Powers shall envy.
The Silmarillion III Quenta Silmarillion Chapter 1: "Of the Beginning of Days"
Note how "time" is capitalized here. It seems to me that "Time", in Tolkien, is a reference to the material universe; thus I submit that there's a difference between Gandalf saying "I strayed out of thought and time" and "I strayed out of thought and Time." I don't think it's a coincidence that he chose the former.
Finally, consider the death of Saruman:
To the dismay of those that stood by, about the body of Saruman a grey mist gathered, and rising slowly to a great height like smoke from a fire, as a pale shrouded figure it loomed over the Hill. For a moment it wavered, looking to the West; but out of the West came a cold wind, and it bent away, and with a sigh dissolved into nothing.
Return of the King Book VI Chapter 8: "The Scouring of the Shire"
Since Gandalf and Saruman are the same class of being (incarnate angels, as Tolkien puts it), presumably the rules governing what "death" means to them are similar. Indeed, the main difference between their deaths are the circumstances. And the implication given by Saruman's death is that he wanted to return to Valinor2, but was forbidden, most likely due to his many failures as an agent of the Valar.
So, to (finally) answer the question: why did Gandalf leave the universe when he died?
I don't think he did.
1 You could perhaps also make a case for "I wandered far on roads that I will not tell", but I don't find that particularly compelling. Gandalf is vague throughout telling his story, and ascribing special meaning to one instance of vagueness seems troubling to me.
2 Indeed, Tolkien says that the desire to return home was a principle motivator of all Istari:
[T]hough they knew whence they came the memory of the Blessed Realm was to them a vision from afar off, for which (so long as they remained true to their mission) they yearned exceedingly. Thus by enduring of free will the pangs of exile and the deceits of Sauron they might redress the evils of that time.
Unfinished Tales Part 4 Chapter II: The Istari