Several factors enter into this:
Rank vs. Position
In the real world, rank is somewhat tied to the position. For instance, infantry companies would usually be commanded by Captains (Army-style rank name, not Navy-style Captains). There were examples of Sergeants commanding companies after lots of casualties, but usually one would send at least a senior Lieutenant. And a company commander who gets promoted to Major could expect a reassignment soon, perhaps to the battalion or brigade staff.
In the series, characters hold the same position over many seasons. Promoting them to higher rank would have required an explanation why they are still at their old job. Note how Kirk's promotion in the movie series made it hard to explain why he was still running the Enterprise.
The Right Stuff
Back in the 60s, all astronauts in the Mercury and Gemini programs were officers. So one assumption in the original series was that all Starfleet crew are academy graduates and officers. The juniormost rank was Ensign.
Then came the next generation and Chief O'Brien. He was called Chief, he acted like a CPO, what was he? One option was to think of him as the Chief of the Transporter Department or something like that. He would have had officer rank. The other option was a messy retcon, and that was ultimately taken.
The assumption that almost everybody is an officer does not leave many ranks to promote in -- Ensign, Lieutenant J.G., Lieutenant, Lieutenant Commander, Commander, Captain. If dramatic necessity means there is just one Captain on the ship (to avoid confusing the viewers), then the others share just five ranks. How many Commanders can you have on a ship like the Enterprise?
Junior Promotions and Taking a Bus
While promotions of command staff would have messed with the established cast of characters, juniors could and did get promotions. Promoting a senior character would require an explanation why they were still on the ship, or them leaving the ship (and show, except for guest appearances).
- Kira Nerys started the show as a major, two steps below Sisko, and ended as a colonel, the same rank as Sisko (but with less seniority in rank). Note that Bajor uses Army-style rank designations while Starfleet uses Navy-style ranks.
- Geordi LaForge started the show as a lieutenant junior grade and got promoted to lieutenant and then lieutenant commander.
So summarized, dramatic necessity made the shows unrealistic in this regard.