From the official Star Wars Databank:
Hyperdrives allow starships to travel faster than the speed of light, crossing space through the alternate dimension of hyperspace. Large objects in normal space cast “mass shadows” in hyperspace, so hyperspace jumps must be precisely calculated to avoid collisions
Apparently, this maneuver works, but only if you are jumping a massive object.
It is hard to tell how massive something has to be for that to work, but we can chalk it up to plot that the lost ships wouldn't have been massive enough.
In terms of war economy it is easy to see why this tactic isn't widely used by a smaller force as well(as is the case with the Resistance). In terms of battle economy, if you keep throwing big, expensive ships like that at your enemy, you might be able to cause more damage than your ship is worth, but that still doesn't mean you will be able to cause more damage than the ship is worth to you. In such a small force, the loss of a cruiser might be devastating, therefore it might only be thought as an strategy in last resort. This is the basis of utility theory in economy.
In regards to war economy that would have made more sense if done much earlier, as instead of losing 3 ships, they would just have lost the bigger one. But that still wouldn't be a guarantee win, since destroying the Supremacy clearly didn't stop the First Order from attacking Crait, and it might have even pushed the First Order into a fiercer attack.
Remember that the First Order only continued this starvation tactic because they believed the rebels had no escape, and maintaining this tactic would be more efficient than losing more fighters on a long range engagement. By poking it's big ship, they would be risking enraging the quiet lion into an attack.
Edit: A lot of people keep pointing out that the "mass shadows" are cast by the massive objects in realspace, not by the one in hyperspace, and that is true. Still, we can take two things from this:
First, this proves that collisions can indeed occur in hyperspace, and it is plausible to assume that the Supremacy is big enough for that to happen.
Second, we know that an object in hyperspace interacts with the "mass shadow" of an object in realspace, not directly to the object itself.
In my mind this means that when collisions occur, the velocity of impact isn't really as important as the mass of the object, since any damage we are doing here isn't damage "to the actual realspace structures" but more like gravitational damage dealt by our ships mass interacting with this "mass shadow". This would be either by stripping mass shadow away or (as the effect in the movie makes more likely) stripping bits of mass into hyperspace.
This last paragraph is just my theory, and there is no source anywhere that corroborates how a hyperspace collision would happen, as far as we know this is the first time something like this ever took place in canon(thought the bit on mass shadows proves it is possible).
It might seem that I am trying too hard to explain why smaller ships can't do this maneuver, and it is because I am. The size restriction is the only explanation that would afford this tactic not being worth using and therefore it not being established previously.