I read the books about 15 years ago, borrowed from a local library in Germany (books were in German, but I assume they where translation of an English book series).
It was a book series set in a high fantasy world. The protagonist had to set out to gather a number of stones/crystals that where associated with the elements (Fire/Water/Earth/Air). I think each book was themed after one of the crystals (I seem to mostly remember the blue cover of the water book). There might have been more books after the four initial ones.
I remember the protagonist at one point losing one of the crystals again.
I think after each journey the protagonist returned to a special place / sanctuary (maybe an island?).
At some point of the story the protagonist, enters a "cursed" / deathly cave system, that has some kind of yellow / black ancient markings, which from the descriptions can be identified as the nuclear power sign (suggesting that the cave system kills via radiation).
I think there was a map of a coastal region, that could be identified as part of North America, but with different places.
While looking for matches, I came across the Shannara series(es) by Terry Brooks which seems to fit part of what I remember (post-apocalyptic fantasy world), but none of the books seem to match the crystal/stone gathering quest.