Introduction
The answer I provide below is based upon my recollection of the movie (RotS) as well as the novelisation of said movie and the Wookieepedia articles that are linked to. Based on these references, I have made a few extrapolations and conclusions.
Answer
It was most likely, in part, a ruse. The order to send the Separatist Leaders to Mustafar originally came from Darth Sidious(AKA Emperor Palpatine), as stated in this article, under "Death" and was passed down to them via General Grievous. The Wookieepedia article on Mustafar states the following, under Clone Wars:
"Toward the end of the Clone Wars, Palpatine would continue to have a
vested interest in the planet, ordering General Grievous to move the
Separatist Council there near the end of the war, apparently for their
own safety, but in reality to gather them together so that Skywalker,
newly anointed as Darth Vader, Palpatine's new apprentice, could kill
them all, and deactivate the droid armies, bringing about the end of
the war."
This implies that the planet wasn't necessarily as well-protected, or rather well-defended as Grievous and Palpatine let on.
It does, however, a bit earlier, state that:
"The main Separatist stronghold was located in a massive industrial
complex located on a fiery cliff bracketed by two huge lava flows.
Collection arms mined lava from the area, while durable industrial
droids worked further afield. Within this facility lay a Separatist
command center, one of the most secure bunkers in the galaxy."
This certainly suggests that it would be no mean feat for an enemy force to conquer the stronghold, but this does not necessarily mean that there is a lot defensive fire-power located on the planet, but merely that a ground force would have an extremely hard time landing and taking the stronghold, given it's location.
It is my belief that sending the Separatist Leaders to Mustafar was a nice way of having them all gathered in the place that, "coincidentally", held the controls to the droid forces deployed across the galaxy. Killing off all Separatist Leaders and shutting down the forces at once was the most effective way to end the war: the alternative being a prolonged cat-and-mouse chase with Vader first hunting down all leaders and then deactivating the droids before new leaders could emerge, or vice versa.
So, where did the money go?
As stated in your question, the cost of securing the Stronghold at Mustafar nearly bankrupted the Trade Federation, and the money had to go somewhere.
Given the difficult location of the Stronghold (amidst a flowing river of lava), it would probably cost a pretty credit to securing the Stronghold from environmental hazards (flailing lightsabers not covered by insurance), but most likely not enough to "almost bankrupt" the Trade Federation.
I would think that the money, as suggested in the comments, went to space-based defenses or that it was simply siphoned into other parts of the war effort, without Gunray's knowledge or approval.
Conclusion
The stated security of Mustafar was, I believe, in part a ruse to gather all Separatist Leaders in the same place, so that Darth Vader could finish them off AND deactivate the droids all at once.
The planet may have had extensive space-based defenses, to explain the incredible cost to the Trade Federation, or this money may simply have been siphoned in to other parts of the War effort.
