This is really hard to answer. We have seen universes without other heroes, such as the Sony movies that (Spoiler for SM NWH)
just got merged with the MCU, and both Spider-Man were confused by the mention of the Avengers and other heroes beside Spider-Man, as well as Venom being confused at the mention of other Superheroes
so we absolutely have universes with only Spider-Man. (We can probably safely assume, that the movies all exist in the Marvel Comic Universe somewhere, as most shows and games also get to exist in the Universe).
Edit: As of the release of Doctor Strange: Multiverse of Madness, this part also gets more complicated, as the MCU main universe is now officially called 616, which is the comic main universe, however, there are comics that are not part of comic 616 about the MCU and other Marvel movies. At the time of first writing this answer, it was already mentioned in Far From Home that the MCU universe is 616, but as the whole multiverse stick in that movie turned out to be bogus made up by Mysterio and his gang, i assumed that was also completely made up by him. So there is evidence the MCU is part of the comic multiverse, but there is conflicting information from the MCU itself.
However, there are also universes where Spider-Man does not exist, like most shows that do not feature Spider-Man as the main character, as well as some older Marvel movies that are inside their own universe.
If we talk comics the answers gets a little bit more complicated and less clear. Here, in the Spider-Verse event, we learn that the multiverse is held together by the Master-Weaver weaving the Web of Life and Destiny and that Spider-Totems (of which the current comics equivalent of Madam Web is one) are kind of a constant. BUT a lot of Spiders were already killed by the inheritors, leaving their universe without a Spider-Totem. So while in here, we learn every universe HAD a Spider-Totem...not every universe still has a Spider-Totem anymore.
However, later on we get a bit of a different explanation for the multiverse in the comics in the Secret Wars event from 2015. So the question is: does the Master-Weaver REALLY weave all universes? If yes, we can safely assume the Spider-Totem is a constant, while we saw universes that did not have different versions of other heroes. And with a bit of imagination, we can assume that these universes, in which we did not yet see a Spider-Totem, still have one, but they are staying under the radar (like Ezekiel did), are in hiding (like Ezekiel put Silk in hiding), or the universe will eventually have one in the future. If the Master-Weaver however does not Weave all universes, we can assume that the Spider-Totem is kind of special in that regard, but there are universes out there that never had or ever will have a Spider-Totem, just like we saw universes where certain heroes did not exist.