It's possible; Tolkien wrote conflicting notes on the etymology of the name "Incánus", which are published in Unfinished Tales in the essay "The Istari".
Firstly, in a note written sometime before 1966, Tolkien wrote (bold is my emphasis, italic is Tolkien's):
[The passage quoted in the question] is the only evidence that survives for his having extended his travels further South. Aragorn claims to have penetrated "the far countries of Rhûn and Harad where the stars are strange" (The Fellowship of the Ring II 2). It need not be supposed that Gandalf did so. [...] Harad "South" is thus a vague term, and although before its downfall Men of Númenor had explored the coasts of Middle-earth far southward, their settlements beyond Umbar had been absorbed, or being made by men already in Númenor corrupted by Sauron had become hostile and parts of Sauron's dominions. But the southern regions in touch with Gondor (and called by men of Gondor simply Harad "South", Near or Far) were probably both more convertible to the "Resistance," and also places where Sauron was most busy in the Third Age, since it was a source to him of man-power most readily used against Gondor. Into these regions Gandalf may well have journeyed in the earlier days of his labours.
The name Incánus is apparently 'alien', that is neither Westron, nor Elvish (Sindarin or Quenya), nor explicable by the surviving tongues of Northern Men. A note in the Thain's Book says that it is a form adapted to Quenya of a word in the language of the Haradrim meaning simply 'North-spy'.
Second, in 1967, he wrote a different history of the name (again, bold is my emphasis):
It is very unclear what was meant by "in the South." Gandalf disclaimed ever visiting "the East," but actually he appears to have confined his journeys and guardianship to the western lands, inhabited by Elves and peoples in general hostile to Sauron. At any rate it seems unlikely that he ever journeyed or stayed long enough in the Harad (or Far Harad!) to have there acquired a special name in any of the alien languages of those little known regions. The South should thus mean Gondor (at its widest those lands under suzerainty of Gondor at the height of its power).
Gandalf, it is said in the Tale of Years, appeared in the West early in the eleventh century of the Third Age. If we assume that he first visited Gondor, sufficiently often and for long enough to acquire a name or names there - say in the reign of Atanatar Alcarin, about 1800 years before the War of the Ring - it would be possible to take Incánus as a Quenya name devised for him which later become obsolete, and was remembered only by the learned.
Is it likely? That's hard to say. Tolkien makes the very good point that there wasn't much point in Gandalf hanging around the Haradrim, who are pretty much under Sauron's dominion from the beginning1. However, there is an argument to be made that Gandalf could have travelled to South Gondor, a territory claimed by both Gondor and the Haradrim. Looking at modern examples of disputed territories2, it's not inconceivable that Gondorians in that region could have had some loanwords from the Haradrim language.
Although speculation, this is a possible explanation for how Gandalf could have had a name with a Haradrim origin while never actually travelling amongst the Haradrim people.
1 The racial underpinnings of that opinion, as well as rectifying it with Tolkien's Catholic belief that no being is beyond redemption, is something I leave to others
2 I'm not naming any; please, no flame wars