Sauron also kept his ring when his body got destroyed. Tolkien says that people shouldn't "boggle" at how a spirit can keep hold of a ring.
In October of 1958, Tolkien wrote a long letter to Dr Rhona Beare (then a student at Exeter University, responding to a list of twelve questions she sent him on behalf of a group of "fellow-enthusiasts for The Lord of the Rings"). One of her questions was "How could Ar-Pharazôn defeat Sauron when Sauron had the One Ring?" In Tolkien's answer he confirms that Sauron brought the ring to Númenor. He then talks about Sauron's destruction there and says to not "boggle" at how his spirit carried it off.
Sauron was first defeated by a ‘miracle’: a direct action of God the Creator, changing the fashion of the world, when appealed to by Manwë: see III p. 317. Though reduced to ‘a spirit of hatred borne on a dark wind’, I do not think one need boggle at this spirit carrying off the One Ring, upon which his power of dominating minds now largely depended. That Sauron was not himself destroyed in the anger of the One is not my fault: the problem of evil, and its apparent toleration, is a permanent one for all who concern themselves with our world. The indestructibility of spirits with free wills, even by the Creator of them, is also an inevitable feature, if one either believes in their existence, or feigns it in a story.
Letter to Rhona Beare, 14 October 1958, Letter of JRR Tolkien #211
Tolkien does not seem to have ever explained how a spirit could carry a ring, but he explicitly says that it's what happened, so I think it would make sense that Gandalf's spirit also kept his ring.