TL;DR : Noble dragonriders may not have been exclusively Targaryen but all were Valyrian. Furthermore, there have been some bastard dragonriders, called dragonseeds, of unknown origin.
Long answer :
There is a little bit of speculation in the answer, as it is not etched in stone that one can ride a dragon only if he is valyrian, but as there is a limited amount of dragons and dragonriders, we know individually each dragonriders (at least in post-doom era)
As far as I know, all known noble dragonriders where Valyrian. There are also some bastards who became dragonriders, called dragonseeds ( bastards of House Targaryen as well as House Velaryon), even though we are not 100% sure of their origin.
Of course, the Targaryen are the most famous dragonriders, but there are others as well.
- Jaenara Belaerys flew over Sothoryos with his dragon.
- Aurion, Emperor of Valyria, is a famous dragonlord that survived the Doom of Valyria, proclaimed himself Emperor of Valyria, and tried to reestablish the Freehold.
- House Velaryon, A Valyrian noble House who, like the Targaryan, left Valyria before the Doom. They are allied with Targaryens, and there have been a number of dragonriders from this house.
- Dragonseeds, alleged Valyrian bastards who became dragonriders during the Dance of the Dragons.
The only potential non-Valyrian dragonriders are the dragonseeds, because we don't really know the lineage of those bastards.
Usually, the term of dragonseed is used for Targaryan and Velaryon bastards, but the term was also used to call the dragonriders during the dance of the dragons.
Long story short, there was a civil war between Targaryens, and the black (Rhaenyra Targaryen's loyalists) had more dragons than dragonriders. Prince Jacaerys Velaryon offered lands, riches, and knighthood to any such "seeds" that could master a dragon. A lot of men tried, a lot of men failed.
Those who succeeded were known as dragonseeds, but we don't know for sure if they were all Targaryens/ Velaryons or if it's their alleged Valyrian blood which enabled them to ride a dragon. Another thing to note is that not all of them had Valyran traits (Silver-gold hair, purple eyes etc.).
The best example is Nettles : She is described as a "brown-skinned girl with black hair and brown eyes", not your typical Targaryen.
So, to sum up, all noble dragonriders were valyrian, and bastards dragonriders were supposed to be Valyrian as well, but not all of them had strong valyrian traits, so in conclusion:
- Only Valyrians can master the dragons.
- Valyrians don't need to be pure-blooded to accomplish that feat.