I don't think we know, in TOS, whether they are always recording or not.
It seemed pretty clear to me in the show that Kirk was not surprised that there was a recording, or that it showed him pressing certain buttons. That indicates to me that it is an actual recording, and not a simulation based on recorded data. One possibility though is that the circumstances, such as alert level, might affect how much recording was done, automatically. They could, but might not want to record everything, or at least erase records after a while for unimportant periods, just to reduce the sense of being recorded all the time (I know that would bug me).
In the episode "The Menagerie", there is a trial where an objection is made about the level of recording presented in that episode, which is much more than in "Court Martial" and includes scenes beyond the ship. They say that no ship keeps that level of detailed recordings. But that also of course indicates that they do keep recordings. (As Michael McGriff kindly looked up and posted in comments, the quote is: "KIRK: Screen off. Chris, was that really you on the screen? (flash) That's impossible. Mister Spock, no vessel makes record tapes in that detail, that perfect. What were we watching?")