The name Gandalf was lifted directly from the Dvergatal, or "Catalogue of Dwarves", in the Völuspá. You may recognise some of the other names here too:
Alþjófr, Dvalinn,
Nár ok Náinn,
Nípingr, Dáinn,
Bífurr, Báfurr,
Bömburr, Nóri,
Án ok Ánarr,
Ái, Mjöðvitnir.
Veigr ok Gandalfr,
Vindalfr, Þráinn,
Þekkr ok Þorinn,
Þrór, Litr ok Vitr,
Nár ok Nýráðr,
nú hefi ek dverga,
Reginn ok Ráðsviðr,
rétt um talða.
Fíli, Kíli,
Fundinn, Náli,
Hepti, Víli,
Hanarr, Svíorr,
Nár ok Náinn,
Nípingr, Dáinn,
(Where "Þ" = English "Th", therefore Þráinn = Thráinn, Þorinn = Thorinn, Þrór = Thrór)
This was openly acknowledged by Tolkien in a December 1937 letter, referenced in the foreword to History of Middle-earth 6.
I don’t much approve of The Hobbit myself, preferring my own mythology (which is just touched on) with its consistent nomenclature – Elrond, Gondolin and Esgaroth have escaped out of it – and organized history, to this rabble of Eddaic-named dwarves out of Volüspá, newfangled hobbits and gollums (invented in an idle hour) and Anglo-Saxon runes.
What is particularly interesting about this is that, in the original drafts of the Hobbit, Gandalf was actually the name of the lead Dwarf, whereas the wizard was named Bladorthin, and it was only in the course of composition that the names changed: first of all Gandalf was dropped from the lead Dwarf, but the wizard remained Bladorthin for a short while, only picking up the name Gandalf a couple of chapters later. This is all covered in the History of the Hobbit.
The name "Gandalf" itself is just Old Norse for "Elf with a Magic wand", and Tolkien used the same meaning in his own works, quoting here from the Istari material in Unfinished Tales:
Mostly he journeyed unwearingly on foot, leaning on a staff; and so he was called among Men of the North Gandalf, “the Elf of the Wand”. For they deemed him (though in error, as has been said) to be of Elven-kind, since he would at times works wonders among them, loving especially the beauty of fire; and yet such marvels he wrought mostly for mirth and delight, and desired not that any should hold him in awe or take his counsels out of fear.
Obviously a more appropriate name for a wizard than for a Dwarf!