I do not believe Tolkien addressed this in his writings. However, it is extremely likely that lore of their ancestry passed down from father to son (or daughter) in the line of the Kings of Gondor. It is also possible that old scripts1 were written which retained this information.
The Kings were definitely not "afraid" or "mistrustful" of Elves. In TA1974 King Eärnil sends his son Eärnur to the aid of the North Kingdom. They arrive via the Grey Havens, which is Elvish territory.
But when Eärnur came to the Grey Havens there was joy and great wonder among both Elves and Men.
Afterwards the army of Gondor goes to battle alongside the Elves of Círdan and Elves from Rivendell under the command of Glorfindel, an Elf.
But the Host of the West came down on him out of the Hills of Evendim, and there was a great battle on the plain between Nenuial and the North Downs. The forces of Angmar were already giving way and retreating towards Fornost when the main body of the horsemen that had passed round the hills came down from the north and scattered them in a great rout. Then the Witch-king, with all that he could gather from the wreck, fled northwards, seeking his own land of Angmar. Before he could gain the shelter of Carn Dûm the cavalry of Gondor overtook him with Eärnur riding at their head. At the same time a force under Glorfindel the Elf-lord came up out of Rivendell. Then so utterly was Angmar defeated that not a man nor an orc of that realm remained west of the Mountains.
The fact that the army of Gondor was allowed into the Havens shows that there must have been communication between Gondor and the Elves of the Grey Havens.
Moving on to the Stewards and the time of the War of the Ring. Denethor knew what Imladris was: Rivendell in the old tongue. It's pretty clear cut that Denethor got his knowledge somewhere: either from his fathers before him or (more likely) from the library of Gondor.
'In that dream I thought the eastern sky grew dark and there was a growing thunder, but in the West a pale light lingered, and out of it I heard a voice, remote but clear, crying:
Seek for the Sword that was broken:
In Imladris it dwells;
There shall be counsels taken
Stronger than Morgul-spells.
There shall be shown a token
That Doom is near at hand,
For Isildur's Bane shall waken,
And the Halfling forth shall stand.
Of these words we could understand little, and we spoke to our father, Denethor, Lord of Minas Tirith, wise in the lore of Gondor. This only would he say, that Imladris was of old the name among the Elves of a far northern dale, where Elrond the Halfelven dwelt, greatest of lore-masters. Therefore my brother, seeing how desperate was our need, was eager to heed the dream and seek for Imladris; but since the way was full of doubt and danger, I took the journey upon myself. Loth was my father to give me leave, and long have I wandered by roads forgotten, seeking the house of Elrond, of which many had heard, but few knew where it lay.'
So there would've been some old tome2 that describes what was Rivendell in Gondor for Denethor to know what it is. And if the Stewards had access to these tomes, surely the Kings before them did as well.
Finally, the Kings of Gondor were proud of their ancestry and knew who Ar-Pharazôn the Golden was. It is likely that the Kings who were learned in lore and proud of their ancestry knew who their ancestors were and who they were originated from: Elros Tar-Minyatur, brother of Elrond.
'The loss of Umbar was grievous to Gondor, not only because the realm was diminished in the south and its hold upon the Men of the Harad was loosened, but because it was there that Ar-Pharazôn the Golden, last King of Númenor, had landed and humbled the might of Sauron. Though great evil had come after, even the followers of Elendil remembered with pride the coming of the great host of Ar-Pharazôn out of the deeps of the Sea; and on the highest hill of the headland above the Haven they had set a great white pillar as a monument.
1 & 2: The Akallabêth written by Elendil is likely one of these tomes. That is assuming that it still existed in Gondor.