The Night's Watch receives recruits from a number of sources including volunteers, criminals, and outcasts. What happens if a recruit either fails the training or refuses to take the oath after completing training? Can outcasts, like Janos Slynt, leave and then attempt to quietly live their lives, or perhaps leave Westeros? Could Snow and Tarly simply have gone home? What about the criminals, would they have been transported back to the dungeons they came from?
Volunteers are apparently free to leave at any time before they take their oath. Until Tarly took his oath, he was free to leave the Night's Watch - but stayed out of fear of what his father might do if he were to leave. Being the coward he was, it would likely have never occured that he could somehow escape his father's reach.
Criminals, on the other hand, are sentenced to the Night's Watch instead of life imprisonment or execution. If they do not take up the vow, their original sentence under the judgment of the King (or their liege-lord) remains - and would be conducted summarily by the Night's Watch if the criminal failed to take the vow, or was caught running from the Wall.
As for "outcasts", like Janos Slynt - well, this is really just a special of case of criminal. Another example, would be Jorah Mormont - Eddard Stark would have either had Mormont executed or committed to the Night's Watch for the crime of dealing in slaves. Jorah was able to escape into exile. Slynt, being in the disfavour of the Hand, and sentenced to the Night's Watch, would also have been forced to choose exile as he would not be safe from the Hand's justice anywhere in Westeros.
One last point I realised I didn't cover was your question about "failing in training" - there are many jobs other than patrolling and ranging - the men of the Night's Watch need feeding (and cleaning up after), and arming (and maintainence of weapons), and clothing (and laundry). There are kennels and stables, and buildings needing repair. Any recruit unsuitable for ranging and patrolling will be given one of the other jobs so that a more able man is free to be a Ranger.
Adding up to HorusKol's answer, recruits who came to the Watch out of their free will can leave at any point they want before they take their vows.
“You have learned the words of the vow. Think carefully before you say them, for once you have taken the black, there is no turning back. The penalty for desertion is death.” The Old Bear paused for a moment before he said, “Are there any among you who wish to leave our company? If so, go now, and no one shall think the less of you.”
No one moved.
AGOT - JON VI
The night before his induction, Jon contemplated quitting and going back.
Once he swore his vow, the Wall would be his home until he was old as Maester Aemon. "I have not sworn yet," he muttered. He was no outlaw, bound to take the black or pay the penalty for his crimes. He had come here freely, and he might leave freely … until he said the words. He need only ride on, and he could leave it all behind. By the time the moon was full again, he would be back in Winterfell with his brothers.
AGOT - JON V
And one day later, Jon was going to leave when they assigned him to the order of Stewards instead of the Rangers.
Marsh turned his smile on Jon. “Lord Commander Mormont has requested you for his personal steward, Jon. You’ll sleep in a cell beneath his chambers, in the Lord Commander’s tower.”
“And what will my duties be?” Jon asked sharply. “Will I serve the Lord Commander’s meals, help him fasten his clothes, fetch hot water for his bath?”
“Certainly.” Marsh frowned at Jon’s tone. “And you will run his messages, keep a fire burning in his chambers, change his sheets and blankets daily, and do all else that the Lord Commander might require of YOU.”
“Do you take me for a servant?”
“No,” Maester Aemon said, from the back of the sept. Clydas helped him stand. “We took you for a man of Night’s Watch... but perhaps we were wrong in that.”
It was all Jon could do to stop himself from walking out. Was he supposed to churn butter and sew doublets like a girl for the rest of his days? “May I go?” he asked stiffly.
“As you wish,” Bowen Marsh responded.
AGOT - JON VI
Criminals and Traitors however arrive to atone for their crimes. If they refuse to atone, they will logically have to face the legal ramifications of their crimes.
There is no such thing as failing the training. The training never ends until the Master at Arms decides it has ended or the recruit dies.
“Our Lord Commander has given the training of recruits into the hands of Ser Alliser Thorne,” the maester said gently. “Only he may say when a boy is ready to swear his vow, as you surely know. Why then come to me?”
AGOT - JON V
Chett could stand no more. “I’ve seen this fat boy in the common hall,” he said. “He is a pig, and a hopeless craven as well, if what you say is true.”
“Maybe it is so,” Maester Aemon said. “Tell me, Chett, what would you have us do with such a boy?”
“Leave him where he is,” Chett said. “The Wall is no place for the weak. Let him train until he is ready, no matter how many years that takes. Ser Alliser shall make a man of him or kill him, as the gods will.”
AGOT - JON V
Of course a recruit can call quits at any time during the training. Depending on his condition of arrival, he may walk free or held accountable for his crimes.
As this answer demonstrates, the training for the Night's Watch, like fights in Fight Club, will go on as long as it has to. It is not clear what the status of the criminals are during this period. After the training is complete, but before saying the words
Mormont stood before the altar, the rainbow shining on his broad bald head. "You came to us outlaws," he began, "poachers, rapers, debtors, killers, and thieves. You came to us children. You came to us alone, in chains, with neither friends nor honor. You came to us rich, and you came to us poor. Some of you bear the names of proud houses. Others have only bastards' names, or no names at all. It makes no matter. All that is past now. On the Wall, we are all one house.
This to me implies that the crimes are in the past prior to saying the oath. In essence, that completing the training is considered sufficient punishment for those offered the choice of joining the Night's Watch.
That said, it is a little confusing since Maester Luwin tells Greyjoy
"Don't go home. Join the Night's Watch. Once a man has taken the black, he is beyond reach of the law. All his past crimes are forgiven."
which seems to imply that you have to join (say the oath to be forgiven).
One argument against the idea that completing the training would be enough to forgive past crimes is that criminals would then opt for this. This thesis however, argues that military style boot camps were more desirable than prison, but less desirable than county jail. Further, the argument is made that military style boot camp could be used in lieu of imprisonment. While the punishments in Westeros are more severe than in the US, the training on the wall is also pretty severe. Potentially, even if the criminals left after completing the indefinitely long training, the nobles may feel that they have served there time.
There is also the additional benefit that the criminals see a 700 ft tall, 300 mile long wall while being given 3 meals a day and forming close bonds. A lot of people enjoy serving in the military.