> surely, in an infinite multiverse, there will be at least one, if not an infinite number of dimensions/universes in which everything is almost exactly the same, with only small differences, for example the sky is a slightly different colour

Maybe, maybe not. We have no idea what rules the MCU multiverse operates under, and I'm sure the writers don't want to be too specific about it until they come to actually write stories that depend on those rules.

One such story that features "the" Dark Dimension is [*What If... Doctor Strange Lost His Heart Instead of His Hands?*](https://marvelcinematicuniverse.fandom.com/wiki/What_If..._Doctor_Strange_Lost_His_Heart_Instead_of_His_Hands%3F) This features an alternate universe, where Doctor Strange does some... inadvisable things with the Time Stone. We see the Doctor Strange of this universe confront and defeat Dormammu in the Dark Dimension, much as our Doctor Strange did in his movie.

As such, presumably each universe portrayed in *What If...?* has its own version of the Dark Dimension, as we didn't see two Doctors Strange in there bargaining Dormammu right in his big ugly face.