Were the people of the galaxy not concerned that Palplatine had named a Sith Lord as his number 2 in the Empire?
Vader was mostly a troubleshooter and assassin for the Emperor, as noted in comments to this answer. He probably was not in the public eye, nor was he much involved in the day-to-day politics and administration of the Empire. As mentioned by Tarkin, this was done by the Senate with its bureaucracy, and later by regional governors:
TARKIN: The Imperial Senate will no longer be of any concern to us. I have just received word that the Emperor has dissolved the council permanently. The last remnants of the Old Republic have been swept away forever.
TAGGE: But that's impossible! How will the Emperor maintain control without the bureaucracy?
TARKIN: The regional governors now have direct control over their territories. Fear will keep the local systems in line. Fear of this battle station.
By the time of Episode IV, it is known that Vader claims to be a Force user, although many do not believe the Force really exists:
ADMIRAL MOTTI: Don't try to frighten us with your sorcerer's ways, Lord Vader. Your sad devotion to that ancient Jedi religion has not helped you conjure up the stolen data tapes, or given you enough clairvoyance to find the rebels' hidden fortress...
[Vader makes a pinching motion and Motti starts choking]
Vader exhibits all the obvious characteristics of a Sith, including use of the prefix/title "Darth". But Motti does not mention the Sith, and instead refers to Vader as a Jedi. This is an indication that the Sith were not generally known.
It is possible that it suited the Emperor to tell part of the truth, that Vader had once been a Jedi but turned against his order. Interestingly enough, this is much the same as what Obi-Wan tells Luke:
OBI-WAN KENOBI: A young Jedi named Darth Vader, who was a pupil of mine until he turned to evil, helped the Empire hunt down and destroy the Jedi knights.
It's also notable that Obi-Wan doesn't expect Luke to know what the Sith are, or consider it worth explaining to him.
Some people might remember the history of the Sith and make the connection to Darth Vader, but judging by Motti's remarks and other circumstantial evidence, this was not a matter of great public concern.
(All quotes from Episode IV)