In TOS, it was very rare to show two ships shooting at each other in the same shot.
The Enterprise might be seen firing phasers or photon torpedos out of the frame, and then they might cut to the enemey ship being hit by the weapons fire coming from out of the frame.
Or there might be a shot of the enemy ship firing its weapons out of the frame, and then it cuts to a shot of the exterior of the Enterprise being hit by the weapons fire. Or maybe a shot of the interior of the Enterprise bridge, with everything shaking wildly under the force of the hit.
And the stated distances in combat were given as tens of thousands or hundreds of kilometers or miles.
Pete quoted dialog from "The Enterprise Incident", "Elaan of Troyius", and
"Patterns of Force" in his answer.
In "The Changling" the Enterprise is hit by two energy bolts from an unknown source before Spock manages to detect the source:
SPOCK: Unknown, Captain. Nothing within sensor range. (a third bolt approaching) Something now, Captain. Very small. Bearing one two three degrees, mark one eight. Range ninety thousand kilometres.
KIRK: That's our target, Mister Sulu. Prepare photon torpedo.
(The third bolt hits)
SCOTT: Shields still holding, sir, but the drain on the engines is reaching the critical point. Ach, we lost warp manoeuvreing power. Switching to impulse.
SULU: Photon torpedoes armed, sir.
KIRK: Has the target changed location, Mister Spock?
SPOCK: No, sir. Holding steady.
KIRK: Ready photon torpedo number two, Mister Sulu.
SULU: Ready, sir.
KIRK: Fire.
SULU: Torpedo away. (a pause, then a flash) Direct hit.
SPOCK: No effect. Target absorbed full energy of our torpedo.
In "Journey to Babel"
CHEKOV: Aye, sir. Phasers standing by. He's just hovering out there, sir.
KIRK: Looking us over. We're dead as far as he knows.
THELEV: You're baiting him. You're trying to lure him in.
CHEKOV: Here he comes. Range decreasing. Speed dropping close to sublight.
KIRK: Hold your fire, Mister Chekov.
CHEKOV: Phasers locked on target. Range closing. Seventy five thousand kilometres.
KIRK: Fire.
(There's a satisfying flare on the viewscreen.)
CHEKOV: Got him!
In "The Tholian Web" a tholian ship approaches:
SULU: Range two hundred thousand kilometres. Velocity zero point five one C.
(An arrow-head shaped ship has arrived.)
SPOCK: Lieutenant Uhura, signal Red Alert.
UHURA: Aye, sir. Red Alert. All decks, go to Red Alert.
SULU: They've stopped dead, sir. Range ninety thousand kilometres and holding.
Commander Loskene agrees to wait until the next interphase to see what happens, so the Tholian ship should not change its position after the conversation:
Later, when the interphase doesn't seem to happen on schedule:
SULU: Mister Spock, we're being fired upon.
(The ship shakes to the impact of the blasts of energy.)
The Enterprise fires back and damages the Tholian ship. So 90,000 kilometers is within the weapons ranges of both ships. Later another Tholian ship appears.
SPOCK: Distance, Mister Sulu.
SULU: Just beyond phaser range, sir.
Then the two Tholian sihps start spinning a spherical energy web to trap the Enterprise. A web that should have a radius of over 90,000 kilometers to be out of range of the Enterprise's phasers.
The shots of the Tholian web don't make it look like it is over 180,000 kilometers in diameter, but I guess that is due to simple special effects failure and we shouldn't consider it important.
In later Star Trek productions most spacehip battles had the two (or more) space ships seen in the same shot and thus appearing to be be only a kilometer or two from each other.
I put that down to a sort of reverse special effects failure. The budgets for special effects were now high enough that they could afford to show both ships in the same shot together, and thus did so, despite that it made them fight each other at distances much closer than World War Two battleships fought.
One possible in universe explanation could be that force shields were very weak in the era of TOS, so that phasers could penetrate them and damage ships even at distances of tens or hudneds of thosuands of kilometers. But by the era of TNG energy shields were much stronger, and starships had to get within a few kilometers of each other to have any chance of penetrating the shields and damaging the enemy ships.
I have read an article online somewhere saying the out of universe reason was Star trek II: The Wrath of Khan. The final battle took place within the improbably dense Mutara Nebula, where vision was reduced to about a kilometer and sensors were almost totally useless. Thus starships had to be almost touching each other to be able to detect each other and aim their weapons. And later movies and tv shows copied those scenes of starships fighting almost touching because they wanted their battles scenes to look as cool as those in Star trek II: The Wrath of Khan, even though the in universe reason for the ships being so close was not longer applicable.
And if anyone remembers details about that article it would improve my answer.
The influence of the Star Wars movies with spaceships shooting at each other from close range was probably also important.
