Taking it as written, it is the West that is forgotten. It's a similar construction to "blade that was broken."
I took it to mean the majority of regular folk, hobbits, non-noble dwarves, and men of Gondor and Rohan, and all those who live beyond the edges of Tolkein's map, do not know about the Undying Lands. (Although most of the characters we become intimate with: wizards, kings, elves, and more-sophisticated hobbits, have not forgotten the West and know its importance, it "is forgotton" by most folk living at the end of the Third Age.)
To me it also hints at Gandalf's understanding that such knowledge will eventually be completely forgotten by all who remain in Middle Earth. The decline and fall that is a strong theme of Tolkien's work.
And if you take the tales of Middle Earth at their stated (by the author) purpose, as millennia-old tales of a mythic Britain, he's talking about our days, long after all the Elves left, and dwarves and hobbits faded or merged with Men.
No doubt I'm reading too much into such a brief expression.