Gladiator, by Philip Wylie.
The story begins at the turn of the 20th century. Professor Abednego Danner lives in a small, rural Colorado town, and has a somewhat unhappy marriage to a conservative religious woman. Obsessed with unlocking genetic potential, Danner experiments with a tadpole (which breaks through the bowl he's keeping it in), and a pregnant cat, whose kitten displays incredible strength and speed, managing to maul larger animals. Fearing the cat may be uncontrollable, Danner poisons it. When his wife becomes pregnant with their first child, Danner duplicates his experiment on his unknowing wife.
Their child Hugo almost immediately displays incredible strength, and Danner’s wife realizes what her husband has done. Though she hates him, she does not leave him, and they instead raise their son to be respectful of his incredible gift and sternly instruct him never to fight, or otherwise reveal his gifts, lest he be the target of a witch-hunt. Hugo grows up being bullied at school, unwilling to fight back. However, he finds release when he discovers the freedom the wilderness around his hometown provides, unleashing his great strength on trees as a manner of playing.
I haven't actually read it, but it appears to match.
Here is the fort scene courtesy of Project Gutenberg:
He jumped over its craggy edge and walked to its centre. There he selected a high place and with his hands he cleared away the growth that covered it. Next he laid the foundations of a fort, over which he was to watch the fastnesses for imaginary enemies. The foundations were made of boulders. Some he carried and some he rolled from the floor of the man-made canyon. By the end of the afternoon he had laid out a square wall of rock some three feet in height. On the next day he added to it until the four walls reached as high as he could stretch. He left space for one door and he made a single window. He roofed the walls with the trunks of trees and he erected a turret over the door.
"Moonshiners," Smith whispered.
"Rubbish. Moonshiners don't build like that. It's a second Stonehenge. An Indian relic."
"Can you tell us anything about these ruins?"
They pointed to his fort. Hugo was hurt. "Those aren't ruins. I built that fort. It's to fight Indians in."
The pair ignored his answer and started toward the fort. Hugo did not protest. They surveyed its weighty walls and its relatively new roof.
"Looks recent," Smith said.
"This child has evidently renovated it. But it must have stood here for thousands of years."
"It didn't. I made it—mostly last week."
They noticed him again. Whitaker simpered. "Don't lie, young man."