In the episode, The Wolf and the Lion, following conversation took place:
BARRISTAN: Life is strange. Not so many years ago we fought as enemies at the Trident.
NED: I'm glad we never met on the field, Ser Barristan ... as is my wife. I don't think the widow's life would suit her.
BARRISTAN: (chuckles) You're too modest. I've seen you cut down a dozen great knights.
NED: My father once told me you were the best he'd ever seen. I never knew the man to be wrong about matters of combat.
I think this speaks to Ned's skill as a swordsman. Also, dozens, perhaps hundreds of Knights and thousands of soldiers died during Robert's rebellion - the definitely were not all great, as evidenced by their deaths and Ned Stark's survival.