The ISS is mostly concerned with expelling heat. So if the power went out the instruments would stop giving off heat but the bodies should produce quite a lot and if the ship is only losing heat by IR radiation, this is quite a slow process.
I think it has been made pretty clear in the comments that the rate of heat loss in the vacuum of space is relatively slow and complicated process and would depend on many variables. But I don't think that really matters for this answer. What matters is why it happens the way it does in the Star Trek series.
So to that end, I would make the point that since the first series of ST, due to a lack of realistic experience in star ship adventures, there have been strong parallels made to ship and submarine naval voyages, disasters, adventures and confrontations.
In TOS these parallels were obvious; The eerie sounds of submarine radar and other equipment is very much present on Kirk's bridge. They even have a type of periscope.
Some episodes of Star Trek were even said to be based on naval movies and even real-life naval encounters. The reasons for these parallels are obvious; the vastness of space is reminiscent of the vastness of the oceans with isolated and vulnerable starships replacing equally isolated and vulnerable submarines and early ships where voyages would last from a few weeks to decades. The mystery, the loneliness and the fear were all very much present under and above the ocean.
So we look here to imagine the atmosphere, the anticipation and the consequences of any problems which would arise in such situations. This includes complete power failure. Under the ocean where no heat from the sun reaches, it gets cold quickly. So I think it is just another example of this parallel.
So to summarize, I think this rapid cooling of the ship comes from the connection we see to terrestrial, naval-esque consequences.
BTW... I think the "Space is cold" thing in general could be a misinterpretation of the fact that liquid freezes in a vacuum due to boiling which causes the liquid to rapidly lose heat.
As with everything, key is "Location, location, location".
Every object in space will freeze eventually. Obviously. How long it will take depends mainly on the OTHER sources of heat nearby and inside. In deep space or in the shadow it will be faster, in the sun it might be very slow. Satellites, especially ones requiring low temp to operate properly were basically built so that instrumentation was isolated from the maintenance module, covered in extra thermal isolation and additionally shielded by sun shield to prevent absorption of sun rays. And it worked (see Infrared Space Observatory for example).
When talking about starships in ST universe please remember that they are quite big with quite a lot of surface that gives off heat. Radiators work very well by increasing the size of surface to radiate the excess of the heat. Few of those on the dark side of any space object and you can increase the ratio of radiated heat to absorbed by at least 6-10 to 1. So you can have pretty efficient cooling system on a spaceship and nowhere it said that SF did not use them in starship design.
Now, the crew obviously will be a very significant source of heat, but in order for them to continue living the air must be artificially moved throughout the ship else they will suffocate rather fast. If there is a microgravity condition this will be even more of an issue. Moving air cools everything it passes through/by pretty efficiently. Which also means rather faster than slower. Obviously not to the level of freezing, but definitely to be uncomfortable.
However, Star Trek is not really an s-f show. Thus a lot of issues is just stated, especially since time limit on the show's episode make it impossible to do with great majority of those issue anything other than oversimplify it.