SHIELD were a publicly known and legal organisation. That is no longer the case.
SHIELD was disbanded and shut down after the events of Captain America: The Winter Soldier, which revealed to us that Hydra had been growing within SHIELD for many decades. Since then, the version of SHIELD that we see in show has been classified as a rogue organisation, mostly due the fact that Hydra attacked the United Nations in season two of Agents of SHIELD while pretending to be SHIELD.
Thie situation is explained in dialouge in "Laws of Nature", the first episode of season three of Agents of SHIELD. At one point, the leader of the ATCU, Rosalind Price, and the Director of SHIELD, Phil Coulson, discuss the secrets that they know about each other, causing Rosalind to reveal that she knows all about Colsoun's death and revival through the TAHITI program. Her exact line is;
In the eyes of the public, SHIELD no longer exists. And you were killed long before the agency you work for was pronounced dead.
Rosalind Price, Agents of SHIELD episode "Laws of Nature"
This would seem to confirm that despite the on-going SHIELD activity that we have seen in Agents of SHIELD, the general population are not aware that SHIELD continue to operate, at least legally.