Gandalf did wear Narya, as did Galadrial, Nenya and Elrond, Vilya. The three Elf rings were 'safe' to wear and use so long as Sauron didn't possess the One Ring to rule them. They knew he didn't have it because Frodo had it.
They had to pull out all the stops and use any and all means at their disposal to stop Sauron from getting the One Ring. That was the main theme of the story; keep the Ring out of Sauron's hand and destroy it. They were playing a desperate game, balanced on a razor's edge of survival, so keeping their rings of power in their pocket was not an option.
If Sauron finally succeeded in getting his ring back it was all over anyway; the world would be lost.
Gandalf used Narya's powers to inspire others to resist tyranny, domination, and despair, as well as giving himself resistance to the weariness of time. It's obvious that Gandalf used his ring's power to motivate Theoden and Eomer before and during the battle of Helm's Deep. He also rallied Gondor's disheartened, fleeing and disorganized forces when Minas Tirith was under siege and the Witch-king was at the door. Resistance to weariness? That guy covered a lot of ground that couldn't have been 'humanly' traveled in the time he had. Gandalf was the Energizer Bunny because of Narya's powers.
So too did Galadriel wear Nenya, and Elrond, Vilya. They used their rings' powers to maintain 'good in the world' and to assist Frodo and others in support of the desperate struggle to keep the One from Sauron.
Observers seemed to be magically 'blinded' from seeing the rings on their fingers. While Frodo could see them on their fingers by virtue of being the Ring-bearer, others could not. Even Samwise was 'allowed' to perceive Galadriel's ring while in Lórien, though not clearly. He tells her that he "saw a star through your fingers". This tells me that the rings can be disguised from an observer even though they were worn.
Frodo, not being well versed in ring lore, was unaware of the numerous Elven rings' significance. Lots of people wear rings and jewellery. Frodo even spoke of it (Galadriel's ring) later to Aragorn while traveling down the river Anduin after leaving Lórien. Aragorn quickly admonished him to not speak of it outside of Lórien. That means Aragorn too was aware of Galadriel's ring, and probably Gandalf's as well due to their close and longstanding friendship. To his credit, he could keep his mouth shut.
EDIT:
Sauron forged the Ring around the year SA 1600. Soon afterwards, Sauron attempted to use it to subjugate the wielders of the three Elvin rings crafted by Celebrimbor. When Sauron placed the One Ring on his finger, the Elves were immediately aware of him so removed their rings. Sauron marshalled his armies to seize all the Rings of Power by force. The War began in SA 1693.
After causing lots of trouble, Sauron was finally brought to heel in SA 3441 and the Ring was cut from Sauron's hand by Isildur at the end of the Siege of Barad-dûr. Although Sauron's body was destroyed, his spirit remained in a greatly weakened state and unable to take physical form.
Isildur kept the ring until TA 2 when he was killed by an Orc ambush at the Gladden Fields and the Ring was lost in the River Anduin. The Ring remained hidden in the riverbed for almost two and a half millennia until found by a Stoor named Déagol who discovered it while on a fishing trip with his friend and cousin Sméagol. Sméagol took the ring from Déagol. Sméagol soon took up residence beneath the Misty Mountains for nearly five hundred years.
Bilbo finds the Ring while spelunking under the mountain. The Ring stays in the family for 77 years until it gets melted where it was made.
In short; after the Ring was cut from Sauron's hand he never touched it again. For three thousand years the One Ring was not available to him and the three uncorrupted Elven rings were safely worn and used for the good of Middle-earth.