No, this is correct
The Grey Havens are where Círdan maintains the Elven ships that travel over the Sea. This is also mentioned a few times in the text; for example (emphasis mine):
'And I've heard tell that Elves are moving west. They do say they are going to the harbours, out away beyond the White Towers.' Sam waved his arm vaguely: neither he nor any of them knew how far it was to the Sea, past the old towers beyond the western borders of the Shire. But it was an old tradition that away over there stood the Grey Havens, from which at times elven-ships set sail, never to return.
Fellowship of the Ring Book I Chapter 2: "The Shadow of the Past"
It's possible that you're confusing the Grey Havens with Alqualondë, the haven of the Teleri on the shores of Aman:
[M]any pearls they won for themselves from the sea, and their halls were of pearl, and of pearl were the mansions of Olwë at Alqualondë, the Haven of the Swans, lit with many lamps. For that was their city, and the haven of their ships; and those were made in the likeness of swans, with beaks of gold and eyes of gold and jet.
The Silmarillion III Quenta Silmarillion Chapter 5: "Of Eldamar and the Princes of the Eldalië"
Alqualondë is indeed on a separate continent from Middle-earth, though this is only true until the fall of Númenor in the middle of the Second Age; after that Aman is removed from the world, and is more akin to a separate planet rather than a separate continent. But these two havens are not the same place.
You may also be getting confused by Falas, Círdan's base of operations in the First Age, which is also described as "Havens":
[T]here came also great numbers of the Grey-elves, wanderers of the woods of Beleriand and folk of the Havens, with Círdan their lord.
The Silmarillion III Quenta Silmarillion Chapter 13: "Of the Return of the Noldor"
The Havens of Falas were in Beleriand, which was a region in the west of Middle-earth; while not a continent, it was indeed rather far from the Shire, but Beleriand (and therefore Falas) were destroyed at the end of the First Age, and sank into the sea.