This sounds very much like The Hairy Parents, a short story by A. Bertram Chandler first published in 1975.
The story is set at a resort on the Queensland coast, and the protagonist meets an unusually hairy, bow-legged man and discuss the reasons for the Neanderthals dying out. The man says it is due to them suffering from vitamin D deficiency, since they wrapped themselves in furs and never exposed their skin to the sun. He claims to be able to send his consciousness back in time:
"When I smoke, it takes my mind back to the Ice Age in Europe ... My
mind? It takes me back, I see what my ancestors saw, I feel what they
felt, I know what it's like to stand up to a bear armed with only a
stone axe, and come out on top. I've watched the women scraping the
hide with their stone knives, cutting it to pattern, sewing it with
their bone needles with sinew for thread .. "
His aim is to start a religious cult involving ceremonial exposure of the skin to sunlight. The final line of the story is indeed a play on the words "Hairy Parent"
And suppose that somewhen in the remote Past the world was being
switched on to a different Time Track ...
Was Neanderthal Man, rather than his Cromagnon cousin, the Heir