The answer is that he does.
In his fight against the Nazgul on Weathertop he flings ligntning bolts about and puts on a show of power that can be seen from miles off:
As Frodo lay, tired but unable to close his eyes, it seemed to him that far away there came a light in the eastern sky: it flashed and faded many times. It was not the dawn, for that was still some hours off.
'What is the light?' he said to Strider, who had risen, and was standing, gazing ahead into the night.
'I do not know,' Strider answered. 'It is too distant to make out. It is like lightning that leaps up from the hill-tops.'
His fight against the wolves before entering Moria:
In the wavering firelight Gandalf seemed suddenly to grow: he rose up, a great menacing shape like the monument of some ancient king of stone set upon a hill. Stooping like a cloud, he lifted a burning branch and strode to meet the wolves. They gave back before him. High in the air he tossed the blazing brand. It flared with a sudden white radiance like lightning; and his voice rolled like thunder.
'Naur an edraith ammen! Naur dan i ngaurhoth!' he cried.
There was a roar and a crackle, and the tree above him burst into a leaf and bloom of blinding flame. The fire leapt from tree-top to tree-top. The whole hill was crowned with dazzling light.
He has the same intimidating presence as the Balrog which he displays at Helm's Deep:
There suddenly upon a ridge appeared a rider, clad in white, shining in the rising sun ... The White Rider was upon them, and the terror of his coming filled the enemy with madness. The wild men fell on their faces before him.
In Return of the King he again throws lightning bolts at Nazgul:
One wheeled towards him; but it seemed to Pippin that he raised his hand, and from it a shaft of white light stabbed upwards.
And many other examples.
In fact, Gandalf "whips it out" so often that a more pertinent question might be: "how can he get away with doing so much if his powers are supposed to be restrained?" (there's an obvious answer to that, but please make it a separate question). It's fairly obvious that the common understanding of Gandalf's power (and magic in general) as being subtle and low-key is at best misleading.