Many of the Jedi that survived order 66 decided to simply renounce their powers as well being a Jedi and live in obscurity, anonymity and to never tell of what they were before order 66.
There are several New Jedi Order books that deal with the Jedi that survived the purge and attempted to re-organize themselves into an effective fighting force. Vader encountered one of these Jedi just a couple of weeks after order 66. He fought her and she nearly defeated him. Things that he had been able to do before being cut down by Obi-Wan on Geonosis and being encased in his armor, he could scarcely do. He knew this particular female Jedi and she was only recently granted the title of Jedi Knight. But he could not understand why she was giving him so much trouble during their fight.
Though Vader eventually defeated her.. it was only later when he realized that because of all of the damage to his body, the loss of his legs, the damage to his lungs, eyes, ears, tongue, skin and other trauma, that his connection to the force was greatly reduced, and along with it, his lightsaber skills.
The Skywalker line had twice the potential of Palpatine.. but after Anakins damage, his power decreased to only 80% of that of Palpatine, which was about the same power as Darth Maul had.
Both Vader and Palpatine knew that Vader would never measure up to the potential that he had prior to his horrific injuries. Though Vader did adapt and get most of his lightsaber skill back, his force powers, even as impressive as they still were, were less than half of what they were before his defeat at the hands of Obi-wan.
Of course most of the Jedi were wiped out via order 66. Some became regular merchants, while others became bodyguards, assassins, or simply hid themselves as regular folk, moved to far distant corners of the galaxy and tried to blend in as much as possible and live a normal life away from the empire and prayed that Vader, other dark Jedi or the storm troopers didn't come for them.