You misremember. I just searched through my kindle edition of book 1-4, and there is really only a single mention of Rhaegar as "the last dragon", of Ser Jorah Mormont to Daenerys. However, this makes a big impression on Dany, and she remembers it several times.
So it's really only the opinion of one man, and he dismisses Viserys as "less than a shadow of a snake" because, well, Viserys. As for Dany, she is at that point just a young girl who's been sold off to a barbarian chieftain. Besides, Ser Jorah has feelings for her that wouldn't fit well with seeing her as a dragon, and that does become an issue later on.