Radagast was sent to perform a similar role to Gandalf, but became too distracted by the birds and the beasts of Middle Earth. From Unfinished Tales:
Radagast, the fourth, became enamoured of the many beasts and birds that dwelt in Middle-earth, and forsook Elves and Men, and spent his days among the wild creatures.
The two remaining Istari were Alatar and Pallando, the Blue Wizards. In Unfinished Tales it mentions
Whether they remained in the East, pursuing there the purposes for which they were sent; or perished; or as some hold were ensnared by Sauron and became his servants, is not not known.
In Letter 211 Tolkien expands on this a little more:
I think they went as emissaries to distant regions, East and South, far out of Numenorean range: missionaries to enemy-occupied lands, as it were. What success they had I do not know; but I fear that they failed, as Saruman did, though doubtless in different ways; and I suspect they were founders or beginners of secret cults and 'magic' traditions that outlasted the fall of Sauron.
In Peoples of Middle Earth, Tolkien later wrote that they may have been more successful than otherwise thought, so his thoughts on these two may have changed over the years:
The 'other two' came much earlier, at the same time probably as Glorfindel, when matters became very dangerous in the Second Age. Glorfindel was sent to aid Elrond and was (though not yet said) pre-eminent in the war in Eriador. But the other two Istari were sent for a different purpose. Morinehtar and Rómestámo. Darkness-slayer and East-helper. Their task was to circumvent Sauron: to bring help to the few tribes of Men that had rebelled from Melkor-worship, to stir up rebellion and after his first fall to search out his hiding (in which they failed) and to cause [? dissension and disarray] among the dark East. They must have had very great influence on the history of the Second Age and Third Age in weakening and disarraying the forces of East who would both in the Second Age and Third Age otherwise have outnumbered the West.
That's the extent we know of the Blue Wizards.