In the Avengers Academy's Fear Itself crossover it's claimed "the arrogance of [Pym]! To fill [underspace] with naught but his home." Is underspace solely the domain of Pym with his Pym particles? It seems unusual for there to be only one way to access a dimension so that one man alone can control it.
The Infinite Avengers Mansion is found in Underspace and would appear to fill the entire realm of Underspace, but that would not be true. Their mansion would appear to be an infinite subset of the infinite realm of Underspace. So Underspace is not completely filled with the Avenger's mansion, it only seems to from the INSIDE of the mansion.
As to whether only the Avengers live there, Pym Particles act as a specific key to the dimensional realm called the Underspace from Earth. Pym does not have the only access (Thor was able to open portals there) but since Pym is the primary discoverer and currently the only producer of Pym Particles on Earth, he would be the only person spending time or allowing access there.
Is Underspace completely empty? Perhaps. Think of Underspace as a universe with the same potential as our universe. So there may be others living there, but are so far away (relatively speaking) the Avengers have not run into them yet. If the Avengers Infinite Mansion spends enough time there, they will eventually run into the natives. Count on it. Anyone remember the Negative Zone and how empty it appeared once upon a time...