What’s the history of the dragons and how they became extinct?
I noticed in an episode I watched last night Tywin says:
When I was Hand of the King under your father's predecessor, the skulls of all the Targaryen dragons were kept in this room. And the skull of the last of them was right here. It was the size of an apple.
Spoilers for the books, of course:
Each generation of dragons became smaller, sicklier and less able to reproduce, until they eventually died out. There are strong hints that this is related to the disappearance of magic from the world at the same time, and that the rebirth of dragons has been correlated (it's unclear which way the causal relationship goes) with the return of magic.
In the books it is mentioned several times that dragons do not grow as large when they are kept in captivity. Mike gives a good example of this in his answer. Also:
Drogon is the largest of the three living dragons. Although I believe he was always the largest dragon, it is worth noting that in Meereen he was roaming free instead of being locked up under one of the pyramids.
As to their extinction, Marwyn hints in AFFC that:
The Citadel was behind the death of the last dragons and that they have been actively trying to build a world bereft of magic
All that said, it is still not explained whether:
The disappearance of magic caused the dragons to die, or if it was the other way around.
It would also be good to point out that a lot of them died during the Dance of Dragons (Not the 5th book of the series -- which was called A Dance With Dragons-- , but the civil war that happened during the Targaryen rule). This civil war was on GRRM's novella The Princess and The Queen, or, The Blacks and The Greens, part of the Anthology Dangerous Women.
The dragons that died during the civil war were:
There weren't many of them to begin with, and as you can see, quite a lot perished on that civil war (129 - 131 AC)
Most dragons, and Valyrians, perished during the Doom of Valyria. Those that survived were those residing on Dragonstone: House Targaryen. Originally a minor house of the Valyrian Freehold, the Targaryens suddenly found themselves the only survivors of their race and masters of the three remaining dragons:
The last dragon of House Targaryen (and supposedly in the world) died approximately 150 years before the events of A Song of Ice and Fire. Most of the dragons died in combat (with the majority coming from the Targaryen civil war called the Dance of Dragons). Each successive generation of dragons was smaller, sicklier, and shorter-lived than the previous.
It is believed that the deaths of the dragons coincided with magic leaving the world. There is also evidence that (AFFC spoiler)
the Citadel
was behind the death and demise of the last dragons.
