From what I can tell of Jaime Lannister, he is a good swordsman, because he has fought numerous battles and lived to tell the tale.
What I can't find is any evidence of him beating anyone in a duel with anyone actually good with a sword. All the one-on-one fights he has taken part in with anyone who is particularly skilled with a sword have been inconclusive.
- Big Belly Ben of the Kingswood Brotherhood: He saves Lord Sumner Crakehall, but Ben escapes.
- The Smiling Knight of the Kingswood Brotherhood: He managed to hold him back until Ser Arthur Dayne managed to take over and kill him.
- Ned Stark: One of Jaime's guards manages to intervene by attacking Ned before the fight is over.
- Brienne of Tarth: Their fight is interrupted my Vargo Hoat (Locke in the TV series).
There might be more, but I can't find anything about them on the wiki.
He might have won a tourney melée or two in his youth, but from what I can tell most of the better knights joust, so I doubt he would have bested anyone noteworthy in a sword fight during a tourney.
And he was only given the position of Kingsguard by Aerys because he wanted to make sure that Tywin's heir couldn't inherit his lands.
Every other scenario I can think of that he has fought in throughout the timeline of the books/tv show have been in a group, like the Battle of the Whispering Wood and when he attempted to escape from Riverrun (books).
Of course, now that he has lost his right hand it is likely that he will never be good again, but with his left hand he can't even compete with Ser Ilyn Payne/Bronn.
I have no doubt that Jaime was good, as he is an heir of one of the greater houses, so would have had the best training, but where did he gain this fearsome reputation of being one of the best? Was it simply because he became a Kingsguard so young?
EDIT: Thanks to the comment of Nika G for pointing out the feats of Barristan the Bold, this is the exact kind of thing I'm looking for.
There's not much in the White Book about Jaime, and that's kind of what I mean. Whilst he has shown his bravery (protecting Brienne from being killed) and ferocity (trying to kill Robb after he had already lost the Battle of the Whispering Wood), we haven't had much proof of his supposed legendary skill.
I don't doubt that Jaime is actually a good swordsman, he has cut down knights left, right and center. But they were all less experienced/not as well trained, that's nothing more than many others have done. People like Ned Stark, Jon Snow & Stannis Baratheon are all considered good swordsmen, but none of them are called the best.
I'm saying that Jaime hasn't done much to claim the mantle of the best (or one of) other than being appointed the youngest Kingsguard in history, which he didn't get on battle merits alone. I'm asking if his reputation might have originated from somewhere other than that, or if people had given him this accolade simply from seeing him fight (like Barristan Selmy).