That sounds a lot like ["Bank and Shoal of Time"](https://www.isfdb.org/cgi-bin/title.cgi?60086), a novelette by [R. A. Lafferty](https://en.wikipedia.org/wiki/R._A._Lafferty), first published in the 1981 paperback anthology [*A Spadeful of Spacetime*](https://www.isfdb.org/cgi-bin/pl.cgi?1165) ([Fred Saberhagen](https://en.wikipedia.org/wiki/Fred_Saberhagen), ed.), which can be borrowed (for free but registration required) from the [Internet Archive](https://archive.org/details/spadefulofspacet0000unse). That must be where you read it "more than 20 years ago" as it seems to have been reprinted only once, in the 2017 Lafferty collection [*The Man with the Speckled Eyes*](https://www.isfdb.org/cgi-bin/pl.cgi?600287).

Michael Main's summary from [The Internet Time Travel Database](https://ittdb.com/R.%20A.%20Lafferty):
>Peter Luna brings five “time attempter” experts to his estate, hoping that he’ll be able to pass on his secret for getting over the time shoal that prevents them from exploring the past.
>

There will be more to this answer after I've reread the story.