This series is mostly fantasy fiction, till the end bit. I think I read this story in the mid to late 90's possibly early 2000's
The stones or gems or, the thing they use to gain the powers in the first place are alien in nature and sentient.
There is a man who uses a staff as a focus. He has power of both Water and Air, and using both he can control the weather. There was another, at one time, who controlled both Earth and Fire, but this combination drove him crazy and the weather user had to stop the fire user.
However, the fire user had a son, a weakling with a frail body who (I think) was being forced to take up his fathers powers, to gain the strength and knowledge to be able to control the Earth power, however if I recall correctly, he did not want the boy learning Fire, as he would lose control just like his father did.
As his focus, he eventually took up a sword, which was uncommon for an elemental user, but was told that as long as he holds it, and stores his power within it, it can realistically be whatever he wants it to be.
I dont really remember the meat of the trilogy, just the beginning, bits of the middle, and the end.
In the end, he decided to take on the power of fire, but this indeed proved too much for him to handle and the power was going to go lose. The weather user said that if he lost control, he would break the sword which housed his power, and the boy would die. While struggling to regain control, he ended up falling on more of the gem things, I think taking 12-24 or something into his body instead of the typical 2-4 people use. This was WAY WAY to much and easily overpowered him, and the weather user broke the sword, but all of those gems were sentient, and repaired the damage. To avoid losing control, the boy threatened to kill himself, which would kill all the beings as well. They tried to call his bluff, but the boy was not bluffing. Faced with either dying or becoming subservient, they chose the later, and the boy became the most powerful elemental user of all time (though he still only had the knowledge of Earth and Fire, I always assumed he could do more now)
The book ended with him coming back, unscathed, and meeting with the weather man again, before heading out.
I think the boy had a girlfriend too? Unsure.