Heads would roll.
The Starks has a rule that the man who passes the sentence should swing the sword:
You send hired knives to kill a fourteen-year-old girl and still quibble about honor?" He pushed back his chair and stood. "Do it yourself, Robert. The man who passes the sentence should swing the sword. Look her in the eyes before you kill her. See her tears, hear her last words. You owe her that much at least."
A Game of Thrones - Eddard VIII
This is the key idea here, although Eddard could've left the sword for Robb to pass the sentence, Eddard was still the Warden of the North and of higher rank than Robb (evidently). However, now as the Hand of the King he would certainly have need of the sword, and as such he would be swinging the sword himself, as opposed to letting the King's Headsman swing the sword.
This is highlighted when Lady is sacrificed for Nymeria's attack of Joffrey. Note this is also confirmation that it was Eddard that brought it down to King's Landing and not that it was taken later.
"Send for Ilyn Payne."
"No," Ned said. "Jory, take the girls back to their rooms and bring me Ice." The words tasted of bile in his throat, but he forced them out. "If it must be done, I will do it."
Cersei Lannister regarded him suspiciously. "You, Stark? Is this some trick? Why would you do such a thing?"
A Game of Thrones - Eddard III
Eddard had indeed would've had time to use it during his time in office, had is death not come in such an untimely manner. While he would not have personally executed Gregor, he had commanded those he sent to execute him (thanks @Aegon), he set the tone for his later command over his position as Hand.
In the name of Robert of the House Baratheon, the First of his Name, King of the Andals and the Rhoynar and the First Men, Lord of the Seven Kingdoms and Protector of the Realm, by the word of Eddard of the House Stark, his Hand, I charge you to ride to the westlands with all haste, to cross the Red Fork of the Trident under the king's flag, and there bring the king's justice to the false knight Gregor Clegane, and to all those who shared in his crimes. I denounce him, and attaint him, and strip him of all rank and titles, of all lands and incomes and holdings, and do sentence him to death. May the gods take pity on his soul
A Game of Thrones - Eddard XI
The sword was also ceremonial, and it is likely that Eddard would have had times where he'd have need of it.
Asked if Ned ever used Ice in battle. George points out it was a greatsword, very large and cumbersome, a ceremonial sword for beheading people more than a fighting sword...
So Spake Martin