like did he just know they were there or sense there were people around his domain? like how exactly did he find out about the Faithful survivors of the Downfall of Numenor?.
The Numenoreans had cities and havens in Middle-Earth before Sauron was ever brought to Numenor. In fact, his arrogance regarding the Numenoreans was part of the reason Ar-Pharazon decided to go to war with him.
And sitting upon his carven throne in the city of Armenelos in the glory of his power, he brooded darkly, thinking of war. For he had learned in Middle-earth of the strength of the realm of Sauron, and of his hatred of Westernesse. And now there came to him the masters of ships and captains returning out of the East, and they reported that Sauron was putting forth his might, since Ar-Pharazôn had gone back from Middle-earth, and he was pressing down upon the cities by the coasts; and he had taken now the title of King of Men, and declared his purpose to drive the Númenoreans into the sea, and destroy even Númenor, if that might be.
Naturally, the enormous army that Ar-Pharazon mustered had to land at a port, not just somewhere in the wilderness.
But the fleet came at last to that place that was called Umbar, where was the mighty haven of the Númenoreans that no hand had wrought.
So he certainly would have known Numenoreans survived the Downfall in general even beforehand, since not all of them lived in Numenor. And certainly we might suspect that some of the Faithful lived in Middle-Earth to escape from the persecution of Sauron and Ar-Pharazon.
A specific haven of the Faithful is called out as being founded even before Sauron was taken to Numenor:
2350 Pelargir is built. It becomes the chief haven of the Faithful Númenoreans.
(LotR, Appendix B)
It is not until 3262 that Sauron is taken to Numenor, and in 3319 that the we have Downfall of Numenor, in which Sauron perishes. In 3320, Arnor and Gondor are founded and Sauron returns to Mordor. But note it is more than a hundred years before Sauron attacks the Numenoreans in Exile:
3429 Sauron attacks Gondor, takes Minas Ithil and burns the White Tree. Isildur escapes down Anduin and goes to Elendil in the North. Anárion defends Minas Anor and Osgiliath.
(LotR, Appendix B)
So he had plenty of time to learn about the kingdoms on his doorstep in the intervening period, if he did not already know about them. Which given Pelargir had been around for more than thousand years at this point, you'd think he already would!
I would suspect that Sauron assumed the leaders of the Faithful survived anyway, since they rebelled against the King and Sauron and went aboard their ships prior to the Downfall:
For Elendil had remained in Rómenna, refusing the summons of the King when he set forth to war; and avoiding the soldiers of Sauron that came to seize him and drag him to the fires of the Temple, he went aboard his ship and stood off from the shore, waiting on the time.
They weren't exactly hiding: they openly set up the kingdoms of Arnor and Gondor.