Sometimes there are advantages of being lightly armored. When Bronn fought Ser Vardis Egen as Tyrion Lannister's champion at Eyrie, Bronn was lightly armored and he was able to tire Ser Vardis Egen out. Ser Vardis Egen was dressed in steel from head to heel and found it hard to move.
Ser Vardis Egen was steel from head to heel, encased in heavy plate armor over mail and padded surcoat. Large circular rondels, enameled cream-and-blue in the moon-and-falcon sigil of House Arryn, protected the vulnerable juncture of arm and breast. A skirt of lobstered metal covered him from waist to midthigh, while a solid gorget encircled his throat. Falcon’s wings sprouted from the temples of his helm, and his visor was a pointed metal beak with a narrow slit for vision.
Bronn was so lightly armored he looked almost naked beside the knight. He wore only a shirt of black oiled ringmail over boiled leather, a round steel halfhelm with a noseguard, and a mail coif. High leather boots with steel shinguards gave some protection to his legs, and discs of black iron were sewn into the fingers of his gloves. Yet Catelyn noted that the sellsword stood half a hand taller than his foe, with a longer reach ... and Bronn was fifteen years younger, if she was any judge.
A Game Of Thrones
Fighting with helmet can restrict the vision range for a fighter and there can be other issues as mentioned below.
The slot in his helm limited Tyrion’s vision to what was before him, but when he turned his head he saw three galleys beached on the tourney grounds, and a fourth, larger than the others, standing well out into the river, firing barrels of burning pitch from a catapult.
A naked man fell from the sky and landed on the deck, body bursting like a melon dropped from a tower. His blood spattered through the slit of Tyrion’s helm. Stones began to plummet down, crashing through the decks and turning men to pulp, until the whole bridge gave a shudder and twisted violently underfoot, knocking him sideways.
Suddenly the river was pouring into his helm. He ripped it off and crawled along the listing deck until the water was only neck deep.
A Clash of Kings
Generally in Westeros it is considered to be stupid to fight without a helm (helmet)
“Don’t be impertinent. Cersei has a royal wedding to plan, I am waging a war, and you have been out of danger for at least a fortnight.” Lord Tywin studied his son’s disfigured face, his pale green eyes unflinching. “Though the wound is ghastly enough, I’ll grant you. What madness possessed you?”
“The foe was at the gates with a battering ram. If Jaime had led the sortie, you’d call it valor.”
“Jaime would never be so foolish as to remove his helm in battle. I trust you killed the man who cut you?”
A Storm of Swords
Updating the answer to include Tyrion's trial by battle.
Tyrion had his own doubts, now that they stood on the brink. When he looked at Prince Oberyn, he found himself wishing he had Bronn defending him . . . or even better, Jaime. The Red Viper was lightly armored; greaves, vambraces, gorget, spaulder, steel codpiece. Elsewise Oberyn was clad in supple leather and flowing silks. Over his byrnie he wore his scales of gleaming copper, but mail and scale together would not give him a quarter the protection of Gregor’s heavy plate. With its visor removed, the prince’s helm was effectively no better than a half- helm, lacking even a nasal. His round steel shield was brightly polished, and showed the sun-and-spear in red gold, yellow gold, white gold, and copper.
Dance around him until he’s so tired he can hardly lift his arm, then put him on his back. The Red Viper seemed to have the same notion as Bronn. But the sellsword had been blunt about the risks of such tactics. I hope to seven hells that you know what you are doing, snake.
Like a serpent’s tongue it flickered in and out, feinting low and landing high, jabbing at groin, shield, eyes. The Mountain makes for a big target, at the least, Tyrion thought. Prince Oberyn could scarcely miss, though none of his blows was penetrating Ser Gregor’s heavy plate. The Dornishman kept circling, jabbing, then darting back again, forcing the bigger man to turn and turn again. Clegane is losing sight of him. The Mountain’s helm had a narrow eyeslit, severely limiting his vision. Oberyn was making good use of that, and the length of his spear, and his quickness.
A Storm of Swords
Outside of westeros there are fighting styles which places higher emphasis on speed over safely, these styles of fighting require less armor. In general, fighters go lightly armored when they are looking for speed.
