He's clearly using hyperbole. If there had been a thousand avatars in the last ten thousand years then the average lifespan per avatar would be just ten years.
In my answer to; How many Avatars have there been? I pointed out that in the Avatar episode "Southern Air Temple" we see a chamber containing approximately 130 statues representing prior incarnations of the Avatar. These are tiered in 7 ascending rows plus a further 51 statues at floor level giving a grand total of 181 previous avatars.
In the film "Avatar : The Last Airbender" we see the same temple, but this time with 6 tiers, each containing 30 status (in 5 blocks of 6) plus a larger statue of the most recent avatar; Roku. Again, this gives a grand total of 181 statues.
Given the similarity in numbers, it seems likely that this figure is intended to be the canon figure.
If we assume that the average mortality rate of an avatar is 53 and that the next avatar is born at the moment of the previous avatar's death then there would have been 180(ish) avatars in 10,000 years prior to the events of "The Last Airbender" (10,000 / 54 = 180ish)
On further consideration, it may be that he's describing the fact that he's had to master each element (e.g. of four) each time he's reborn in which case he would have had to learn how to bend the elements a grand total of 724 times, close enough to a thousand that the quote could make some sense.