Did Ilúvatar know that Melkor would 'turn bad'? Was it part of his vast design? Tolkien often hints that also Morgoth's evil is part of the grand plan, and that he does the will of Ilúvatar in the long run, despite opposing him. But why did he oppose Ilúvatar in the first place? Why did Ilúvatar make someone who would oppose him? Surely THE God would have complete control and knowledge.
Melkor's "fall" was that he wanted to make new things, that he wanted to go beyond the bounds of his own intrinsic abilities and nature, that he wanted to be more than what he was (there are interesting parallels to this in other "falls" in the Silmarillion & associated works, but they're for another time):
But as the theme progressed, it came into the heart of Melkor to interweave matters of his own imagining that were not in accord with the theme of Iluvatar, for he sought therein to increase the power and glory of the part assigned to himself. To Melkor among the Ainur had been given the greatest gifts of power and knowledge, and he had a share in all the gifts of his brethren. He had gone often alone into the void places seeking the Imperishable Flame; for desire grew hot within him to bring into Being things of his own, and it seemed to him that Iluvatar took no thought for the Void, and he was impatient of its emptiness.
But being alone he had begun to conceive thoughts of his own unlike those of his brethren.
So Melkor's discord was of his own imagining, but did Iluvatar create him with the capability to imagine this discord, or did he come upon it on his own? Let's ask Iluvatar...
This is your minstrelsy; and each of you shall find contained herein, amid the design that I set before you, all those things which it may seem that he himself devised or added. And thou, Melkor, wilt discover all the secret thoughts of thy mind, and wilt perceive that they are but a part of the whole and tributary to its glory.
And (in case we were in any doubt)...
And thou, Melkor, shalt see that no theme may be played that hath not its uttermost source in me, nor can any alter the music in my despite. For he that attempteth this shall prove but mine instrument in the devising of things more wonderful, which he himself hath not imagined.
This makes it clear. It seems to the Ainur as if they devised things on their own, but ultimately even these things that it seems to them are their own doing are a part of the Music as directed by Iluvatar.
Just to reinforce the point; there is only one order of being in Tolkien which has the ability to go beyond the bounds of the music as directed by Iluvatar, and that's Men, and the ability was an explicit gift from Iluvatar:
'...But to the Atani I will give a new gift.'
Therefore to willed that the hearts of Men should seek beyond the world and should find no rest therein; but they should have a virtue to shape their life, amid the powers and chances of the world, beyond the Music of the Ainur, which is as fate to all things else...
Note in particular the phrase "new gift" here; this is something else that no other being has had before.
Not only was Melkor given the greatest gift of power and knowledge of all the Ainur, he also shared in all the abilities of all the others. The Ainur were said to know only the part of the mind of Ilúvatar from where they came, and so he was closest in all things to Ilúvatar himself. Perhaps this is why he was not as content with merely expounding the theme of Ilúvatar..
To the question of why he was tolerated:
“[Ilúvatar: ] 'Seest thou not how here in this little realm in the Deeps of Time Melkor hath made war upon thy province? He hath bethought him of bitter cold immoderate, and yet hath not destroyed the beauty of thy fountains, nor of my clear pools. Behold the snow, and the cunning work of frost! Melkor hath devised heats and fire without restraint, and hath not dried up thy desire nor utterly quelled the music of the sea. Behold rather the height and glory of the clouds, and the everchanging mists; and listen to the fall of rain upon the Earth! And in these clouds thou art drawn nearer to Manwe, thy friend, whom thou lovest.'
J.R.R. Tolkien, The Silmarillion.
(To which Ulmo responded that he had neither anticipated rain nor conceived of snow).
Melkor, "he who arises in might" is as his creator made him. As far as Tolkien allows us to know, Ilúvatar understood Melkor's addition to the design and any changes he made to the works of others, only enhanced their beauty, functionality or overall quality in some way. This was as Ilúvatar decided it to be.
As a creation myth it is almost necessary to have a force which opposes the general thread of the myth, lest there be no source for challenge, growth, development or change in the mythos. In his way Tolkien was ensuring his universe would have similar forces for change taking place through out the work.
Does this inconvenience the creations? Yes.
Does it force them to grow and evolve? Yes.
Do they always make the best decisions? No. If they did, they would have learned nothing from their existence.
Tolkien's work and his characters resemble most creation mythos with the strongest and brightest member of the pantheon having a frustrated itch they cannot scratch. Melkor's was a desire to create life, a power left only to Ilúvatar. Consider Lucifer who also wanted dominion over something that was uniquely his.
I would consider everything that happened, as it should be, as Ilúvatar in his infinite wisdom allowed to take place. Could he change it? Sure. But if he did, what was the point of bringing Creation into existence at all if he was going to micromanage it?