Most mountains in the Lord of the Rings appear to be hollowed out with tunnels or caverns. You have the dwarf mines under the Lonely Mountain and the Iron Hills. The Goblin Caves under the Misty Mountains and the Mines of Moria under the Mountains. The only mountain without tunnels in it is Mount Doom which is a volcano not a mountain. Does Tolkien ever mention a mountain that doesn't have any caves underneath it?
There are mountains that have been named, however it's not clear whether or not they were hollowed out, in Middle-earth.
The Lord of the Rings
The mountains in Middle-earth (as you explained) have either been carved out by dwarves or Orcs. The only quote for orcs mining I managed to find in The Lord of the Rings or The Hobbit was:
You must dig swift and deep, if you wish to hide from Orcs.
The Fellowship of the Ring, II, 6: "Lothlórien"
It seems that mountains in The Lord of the Rings that were mentioned were either with reference to an Orc domain or a Dwarven domain. Because of that there didn't seem to be any need to mention mountains that were not hollowed. Although as suchiuomizu said, it is likely that most mountain ranges only had a few mountains hollowed out. Although it must be mentioned, the Orcs were said to have been able to travel underground across all the Misty Mountains and through the Grey Mountains to Erebor, suggesting they tunnelled a long way through them.
Messengers had passed to and fro between all their cities, colonies and strongholds; for they resolved now to win the dominion of the North. Tidings they had gathered in secret ways; and in all the mountains there was a forging and an arming. Then they marched and gathered by hill and valley, going ever by tunnel or under dark, until around and beneath the great mountain Gundabad of the North, where was their capital, a vast host was assembled ready to sweep down in time of storm unawares upon the South.
The Hobbit: XVII "The Clouds Burst"
There are a few mountains that were never mentioned to have been carved out in the Silmarillion. To save words I'll refer to them as mountain ranges or parts of them.
The first example I can think of is the Mountains around Gondolin. There were only two gates in the mountains of Gondolin. The main gate in Gondolin went out (West?) under the Encircling Mountians. I can only recall one Gate, however Idril asked Tuor to build a tunnel in secret, which went north.
“But at last, as with great labour they came to the very foot of the cliff, they found an opening, as it were the mouth of a tunnel worn in the hard rock by waters flowing from the heart of the mountains. ”
The Unfinished Tales: Of Tuor and his Coming to Gondolin
“Therefore in that time she let prepare a secret way, that should lead down from the city and passing out beneath the surface of the plain issue far beyond the walls, northward of Amon Gwaret”
The Silmarillion: Of the Fall of Gondolin
Although whether this secret path was underneath the mountains or not is unclear.1
There are however a few mentioned that are not in Middle-earth. The Pelori was the mountain range in Aman upon which Manwë had his seat. There's no mention of them having been tunnelled or anything being placed under them. Furthermore, the Valar split the mountains to let light through the range and to the Elves.
“ and therefore a gap was made in the great walls of the Pelóri, and there in a deep valley that ran down to the sea”
The Silmarillion: Of Eldamar and the Princes of the Eldalië
“... of that passage the captains of Angband knew nothing, and thought not that any fugitives would take a path towards the north and the highest parts of the mountains and the nighest to Angband.”
The Silmarillion: Of the Fall of Gondolin
There are about six or seven mountain ranges in the usual maps of (northwestern) Middle-earth depending on if the Iron Hills are counted, and more mountain ranges off the edges of the maps. If each mountain range has 10 to 100 mountains, there should be 60 to 700 mountains in the area of the usual Middle-earth maps.
I believe the Dwarves all lived together on or under Mount Gundabad early in the First Age before splitting up and all the seven races seeking out separate mountains to live in. During the Third Age dragons began attacking the Dwarves and the seven cities of the Dwarves were taken over by dragons. The Dwarves became a wandering people, though no doubt each race established one or two new cities during the remainder of the Third Age. So this makes a total of about 14 to 21 underground cities of the Dwarves, some of them off the maps in eastern mountain ranges, plus Mount Gundabad.
The other main underground species was the orcs. They originated by uncertain means in Morgoth's underground fortress of Utumno in the far north during the First Age. It is unknown if Utumno was inside any mountain or mountains.
Morgoth build the Iron Mountains in a great arc from east to west to guard Utumno. In the west Morgoth build a lesser underground fortress, on the north side of the mountains, and possibly under a plain instead of a mountain.
Utumno and Angband were devastated when Morgoth was captured by the Valar after the discovery of the Elves. When Morgoth returned to Middle Earth he resided in the rebuilt and redelved Angband. Morgoth built a great tunnel under the Iron Mountains from Angband to the southern side. Morgoth also raised a great artificial triple peaked mountain/slag heap/volcano/smokestack/gatehouse called Thangorodrim as a fortification over the mouth of the tunnel.
So Thangorodrim was certainly a very hollowed out artificial mountain. And though Morgoth's orcs had always lived underground, the ones who lived and/or worked in Thangorodrim were the first specifically said to have done so inside any kind of mountain.
During the Second and Third Ages orcs lived in many underground communities. Some of them lived in former Dwarf cities such as Moria and Mount Gundabad. Some of them lived in new Orc cities they carved out of mountains.
We might guess that there were about ten to thirty new orc underground cities in the mountains in the usual maps of (part of) Middle-Earth.
Thus there should have been about 15 to 22 Dwarf underground cities and about 10 to 30 Orc underground cities in various mountains in the maps of (northwestern) Middle-earth. If there are 60 to 700 mountains in those maps then 0.0357 to 0.866 of the mountains within the area of the usual maps of Middle-earth would have had underground Dwarf or Orc cities.