This stumps me. It is indeed very hard to understand. I've been a part of both military and civilian units (humanitarian, commercial and one scientific survey group -- as a leader and a follower) in hostile and semi-permissive environments. In either case we practice "immediate action drills". Only in the military case do we practice offensive drills.
The core training and preparation cycle is the same.
In the military case the main place we spend our time is on offensive operation rehearsals and contingency planning/practice. In the civilian case the main place we spend our time is on setup/teardown of the operational site, surveys of the area, operational planning related to whatever the point of the trip is, etc. But in both cases whenever entering an environment where hostility is likely to be encountered we spend a lot of deliberate time practicing immediate action drills.
What are these drills of which you speak?
To understand, we need a slight correction...
s/action/reaction/
...fixed
It is a way for a group to plan its first few moments of tactical reaction long in advance of actual contact. So, for example, if we find ourselves in a near ambush, everyone knows without anyone giving an order that the thing to do is turn and charge the enemy firing line immediately. (Non-fighters are to stick as closely to a fighter as possible; if there is a designated principal, then we form a shell around that guy but the basic maneuver is the same.) The act of being ambushed is the assault order.
The delay required for someone in charge to make up their mind about what to do, make all elements aware of that decision, everyone to actually understand what the order is (by no means certain when the poo is flying), and finally for everyone to start actually moving is a death sentence. This delay is the very reason why surprise attacks like ambushes, raids, and covered flanking movements are so successful.
The immediate action in the case of a far ambush is the opposite. Immediate action in the case of an IED, loss of communications, loss of visibility, break in contact (where two sections of your group become separated), crowd-covered shooters, grenade-in-the-car, etc. are all planned in advanced and rehearsed. This is also true for static installations like your base camp, housing area, dining facility, hotel, etc. They all should have immediate actions planned in advance, though complacency is a hard thing to overcome in some environments (when the gap between the last attack and the present day/locale exceeds some emotionally significant threshold -- and there is never any telling what that threshold is going to be).
Life is always played in hardcore mode: No more Marios
The reason for all this practice is that the defining moments of most fights are the very first moments, and you never get a chance to relive them. Pre-planning your moves in those initial moments prevents unit-wide paralysis while everyone waits on the commander to show up and say "Ah, we're being shot at? Hrm... well... let's see here... lemme check a map and call the boss... Hrm... yep. I think we should shoot back and start moving." -- which is essentially the phenomenon you are puzzled about in the Star Trek universe.
Even in really horrible places the vast majority of the civilian contracts I've taken never involve any actual hostile contact. In those that do maybe we'll be doing something operational for 100 days, and only 1 or 2 or those days see something bad happen (there are rare cases where every day is an adventure, but that is really unusual).
By contrast, in just about every show the Federation gets involved in some sort of hostile action. To think that I spend so much time preparing for events that almost never happen, but the Federation believes that good intentions and an appeal to reason will protect them from things that happen all the freaking time and therefore don't practice is sort of ridiculous. Previously I mentioned the "emotionally significant threshold" of time/distance from the last bad incident. Considering near-death experiences are something like a weekly affair for the main case of a Star Trek series I'm shocked at the level of unpreparedness.
And to tie this off...
I have no freaking clue why this is the case. Perhaps it is because they are "non-combatants" who merely are interested in science. But this falls apart when we consider the various Federation wars and the fact that their "scientific ships" are overflowing with shields, huge phasers, planetary bombardment devices, photon torpedoes, and the like. I don't recall Jacques Cousteau having his ship similarly equipped when I was a child.
A more likely answer is that they have a major doctrine problem, a burdensome political process to live with, and most Federation officers are essentially living in denial. This actually seems plausible, considering all the wacky interplay between politicians, high-ranking Federation officials, foreign officials, and random acts of backstabbing plot-thicknessness encountered throughout the series and the movies. It simply may be the case that the Federation's naval assets could very easily become a dominant tool of belligerence in the galaxy were it to redefine its mission that way, and this might scare the bejesus out of politicians and neighboring factions alike. In this sort of environment in the real world we see military arms redefined as police forces, or as research divisions, or as "internal security units" or whatever, and training doctrine deliberately written in a way that prevents tactical units (and especially the leaders of those units) from experiencing many of the situations necessary to really extend their methods of thought and command to encompass the tactical realm more fully.
The tactical engagements in Star Trek could perhaps be made much more interesting were this taken into account. The Federation was victorious against a Klingon incursion, after all, so its not like they don't know how to do this sort of thing -- perhaps they are politically afraid to, because much like the United States today, it could easily ignite an inner war of ethos between the "Federated Empire" and the "Federated Republic" concepts of what the Federation should be.
In any case, I really think it would be awesome to exploit this duality as a plot device. The point this question brings up about them appearing to be constantly underprepared and late to the punch is a great entry point to this sort of political plotline.