During the episode "Simon & Marcy":
>!Marcy mentions that Simon's features have changed each time he takes the crown off. Simon is very reluctant throughout the episode to put the Ice Crown on, because each time it becomes harder and harder for him take off again and each time he puts it on he keeps a little piece of the Ice King's features in his personality and the way he physically looks. At the end of the episode, the last time he puts the Ice Crown on, he sings a song that reminds him of his previous self. This is the only way he is able to come back to help save Marcy. If you notice that when he takes the Ice Crown off the final time in this episode, he barely resembles Simon at all and looks almost completely like the Ice King.