We know that, although publicly he scorned Gandalf, Saruman copied him and also smoked, and imitated Sauron, as often said in The Lord of the Rings. But I just saw a quote about Radagast, saying that
Radagast is, of course, a worthy wizard, a master of shapes and changes of hue; and he has much lore of herbs and beasts, and birds are especially his friends.
And then I remembered Saruman's cloak that changes colour, and I wonder, maybe Saruman also scoffed about Radagast
"Radagast the Brown! [...] Radagast the Bird-tamer! Radagast the Simple! Radagast the Fool! Yet he had just the wit to play the part that I set him."
but then somehow he got this cloak idea from the "master of changes of hue".
It's just a thought – any more evidence?