First, let me say, I know people are going to say "It's just a comic/movie franchise, you have to suspend disbelief". Please bear with me.
X-Men is a little different than other franchises.
We have the concepts of Homo superior/ homo mutandis / mutantur / homo sapiens supreme and the X-Gene / mutant gene / X-Factor chromosome / X-Factor / etc. And these are very central to the plotlines.
In other words, there is the attempt to explain the multitude of powers arising through a biological explanation. Maybe it's just me, but that is qualitatively different than a lot of other origin stories, like 'he's an alien' or 'he can do magic'. (Yes, I realize people will give examples of other superheroes/villains where a biological explanation is attempted, but this question is about X-Men, and it's probably different because in X-Men it is literally in the DNA.)
But here we have all kinds of powers (turning to diamond, telekenesis, opening portals to other dimensions, teleportation, controlling weather, etc.) that clearly can't have a biological basis (at least in our universe).
So, what I'm probably not explaining very well, is that the idea of the impossible arising from an impossible explanation isn't the same as the impossible arising from a supposed plausible explanation (genes). So it's only a partial explanation. Is there then a part of the explanation that tries to close the gap?