Plot goes roughly like this: boy lives in "down to earth" society and is, i think, a village shepherd. Then he "cries wolf" and loses trust of his village, he either runs away or is forced into exile. He finds shelter in a cave that houses backup planetary defense post from times long forgotten, and he speaks to it when alien invasion starts, activating defenses and saving the planet.
This is "Sign of the Wolf" by Fred Saberhagen, one of his Berserker stories. Per ISFDB, it was originally published in the May 1965 edition of If magazine, and has been republished many times in collections.
You can find what I think is the complete text of the story online, but here's a quote:
"Here I am." Duncan fell on his knees before the metal thing that bellowed. In front of the god-shape lay woven twigs and eggshells, so old as to be hardly more than dust. Once priests had sacrificed here, and then they had forgotten this god.
"Here I am," said Duncan again, in a louder voice.
The god heeded him, for the deafening shouting stopped.
In a glassy place beside the god strange things appeared: there was a huge ball, with little red specks crawling its surface, and a little red ball the size of a fist moving toward it.
"Response acknowledged, from defense control alternate 9,864," said the god. "Planetary defenses now under control of post 9,864."