I'm going to add to what other have said about the forest being part of the castle grounds, and explain why I think the forest ended up filled with magical creatures, and whether or not it was planned.
Like with all animals, some magical creatures prefer living in forests. Presumably creatures like Bowtruckles can be found all over the world, in places where there are trees of wand quality. I'm not sure about Thestrals, but I assume they too could thrive in other forests. Them living in the Forbidden Forest could therefore just be accidental. There would be less risks posed by Muggles, like deforestation, hunting and picnicking, but otherwise it would be very much a normal forest.
That said, Bowtruckles (and most likely Thestrals too, despite being considered dangerous) doesn't pose as much of a threat to Muggles as other creatures. Dragons for instance (classified as XXXXX), while there as none of those living in the forest as far as I know, are kept in close control by the Ministry. Dragons are only permitted in certain areas (though of course they don't care about that themselves, they fly wherever they want), usually far from Muggles (mentioned either in GoF or Fantastic Beasts I think). Most likely, dragons would be considered too dangerous for them to live in the forbidden forest.
It is possible that similar protected areas exist for other creatures, both in the cases where the creatures are dangerous and in cases where they need protection. The Forbidden Forest could be one such area where endangered animals are kept. One such species is unicorns, which are both dangerous and possibly endangered. The grounds at Hogwarts are magically protected, which would make it harder (but not impossible) for poachers to hunt down magical animals, like the unicorns, to collect potion ingredients without regard to the creature's survival.
Another creature classified as dangerous (XXXX) by the Ministry is centaurs. They have near-human intelligence, at least according to the Ministry, and are very protective of their territory. They would therefore need to live somewhere far from Muggles, or Muggles could end up being hurt. Being intelligent and long lived creatures, they would likely be far more picky with where to live than Thestrals, and would not like being told what to do by humans.
We do not know exactly how the centaurs ended up in the forest. One may assume that, being a magical place, some centaurs might have decided on settling there centuries ago.
Later, the Ministry probably wanted the centaurs to live in a controlled area, while the centaurs would only move voluntarily. They could possibly then later have come to agreement, so that more/most/all centaurs moved to Hogwarts. While less than ideal to have dangerous creatures close to the school, it was probably decided to be the better solution, since it would be easier to keep a watch on them there.
To sum up, I believe a good explanation is that there wasn't an intention for the forest to be filled with magical creatures, it just happened over the course of centuries. Some animals probably lived there before the school opened, some have moved there since either of their own accord or was moved by force by the Ministry. Most of them are either mostly harmless, or dangerous only when angered, which is probably the reason students are supposed to keep out of the woods.
Another interesting observation I've noticed, is that seemingly all creatures I have checked so far that live in the forest are classified as XXXX (with one exception), which means dangerous but not impossible to tame or handle if you are experienced. The exception being the Acromantulas, Aragog's family. If I recall, Aragog was placed there (very deep in the forest) by Hagrid on suggestion from Dumbledore, which I assume wouldn't be popular with parents if they knew. Acrumantulas are considered untameable, and while Hagrid (and probably Dumbledore, being as trusting as he was) assumed that the spiders would keep to themselves, we clearly see on several occasions (CoS and HBP) that while Aragog cares about Hagrid, the spiders in general have no problem attacking/eating students or other humans from the school.