Your question is from the movie and not the writings of tolkien.
Perhaps the screenwriters of the movie ment to infer that the necromancer could not be of the Istari, as were Gandalf, Saruman, and Radagast. in the books Gandalf the grey mentions that Saruman was the expert in ring lore. the smoke ring incident in "Unfinished Tales" brings about Sarumans conflict with gandalf long before it is mentioned in LOTR in the Council of Elrond chapter of The Fellowship of the Ring. Saruman was the "head of the Order " and was compromised by his will being corrupted by Sauron filling his head with Propaganda Views of the size of the armed might of the Barad-dur armies.Saruman was covering for Sauron, with the Palenthari Saruman knew who was occupying Tol-Brandur. Galadrael already smelled a rat in Saruman and wanted Gandalf to lead The White Council.Samwise wanting to see "Elf magic", the spells on the blades of the swords of the hobitts from the barrow downs shows that magic wasnt limited to the istari (the blade that a hobit knifed the head ringwraith is proof that humans could do magic and spells
(see:On the Barrow Downs section of the fellowship .