It is my understanding that each universe has its own Infinity Stones and they only work within their respective universes.
Dormammu exists within a universe where he controls every bit of energy and matter. He goes from universe to universe to draw more energy into his own.*
When he entered into the universe the Doctor Strange movie was in, there was a portal that merged the 2 universes that Strange used to enter the Dark Dimension and confront Dormammu.
Now, the basic assumption here is that the word dimension is used synonymously with universe.
How come Doctor Strange was able to use an Infinity Stone from his own universe in the Dark Dimension? Likewise, shouldn't Dormammu wield nearly infinite power by possessing the Infinity Stones of his own universe?
* Overlooking the fact that I am sure he tried to invade a particular universe and, unbeknownst to him, he was met with an alternate version of himself, only nicer, friendlier, and had a penchant for showtunes.