The story is There Will be Time by Poul Anderson. Confirmed in a comment by the OP.
bingo! just got the book . Thanks!
The name of the major character is Jack Havig. I do remember two things not mentioned in the link:
When Jack was younger he first found out about his abilities when in a fight and several future-self's came back to help him.
In an attempt to explain the potential plot hole that when he travels in time how come he stays on Earth (the Earth moves so if he travelled in the future by six months, if he stayed in the same place, Earth would be on the other side of the Sun) by suggesting that gravity was linked to his time travel and this kept him in tethered to the place where he started his time jump.
I remember it starting with a boy who can slip through time.
Jack Havig was born in the American midwest in 1933 with a genetic mutation that allows him to travel through time.
Later the story plays out in the future where he's leading a group of rebels.
Jack is discovered by other time-travelers who are agents of a time-traveling organization called the "Eyrie," that is based in the far future and is led by a racist man born in 19th century United States. Initially Jack joins the group, but eventually rebels against them when he discovers and experiences first hand the extent of the Eyrie's rampant brutality and inhumanity as they attempt to achieve their goal of stopping the Maurai ascendancy. To defeat the Eyrie, Jack returns to the 20th century and devises a plan of his own to recruit time-travelers and create a "tribe" that will return to the future to destroy the Eyrie.