Ned even comments on this to Bran himself when they are talking about Gared. He mentions that the man is a deserter and that is why he is to be executed not because he ran away from some White Walkers.
His lord father smiled. "Old Nan has been telling you stories again. In truth, the man was an oathbreaker, a deserter from the Night's Watch. No man is more dangerous. The deserter knows his life is forfeit if he is taken, so he will not flinch from any crime, no matter how vile. But you mistake me. The question was not why the man had to die, but why I must do it."
A Game of Thrones, Bran I
Chett later thinks the same that it is the act of desertion which is why they are executed.
And at home they'll know you for deserters and lop off your fool heads, thought Chett. There was no leaving the Night's Watch, once you said your words. Anywhere in the Seven Kingdoms, they'd take you and kill you.
A Storm of Swords, Prologue
Later this is commented on again when talking about Mance but his situation is obviously a bit more complicated.
Jon gave her a weary look. "Mance is a deserter from the Night's Watch. The penalty for that is death. If the Watch had taken him, he would have been hanged by now, but he's the king's captive, and no one knows the king's mind but the red woman."
A Storm of Swords, Samwell IV
Lastly, it's worth looking into the oath to see why they are executed upon desertion and not by fleeing an enemy during a mission.
Night gathers, and now my watch begins. It shall not end until my death. I shall take no wife, hold no lands, father no children. I shall wear no crowns and win no glory. I shall live and die at my post. I am the sword in the darkness. I am the watcher on the walls. I am the fire that burns against the cold, the light that brings the dawn, the horn that wakes the sleepers, the shield that guards the realms of men. I pledge my life and honor to the Night's Watch, for this night and all the nights to come.
As you can see it's clear from the oath that the punishment for desertion is death and that by returning to Castle Black, and hence the Watch, he would not have been killed.