The story involves a number of high-powered espers / telekenetics / etc., all having decided to leave Earth to start their own world.
One way or another, they acquire a spaceship, and take off, only to encounter some much more evolved beings in space who point out that the same thing happened in their culture many centuries ago, and take them to a planet to view the results. The planet the go to has bunches of beings that have effectively devolved to little more than animals, to the horror of our Heroes.
While they are talking about the immorality of leaving regular humans behind if that would be what happens to them, they aliens interrupt to explain that what they are seeing is NOT the 'left-behinds', but rather those who left. The psychic powers made things too easy for them, and they never had the drive to discover / invent / etc., that the less gifted did, so their culture quickly reached it's rather low pinnacle and fell. Our heroes decide to go back to Earth.
I wanted to say it was one of Anne McCaffrey's 'Talent' stories, but haven't been able to find it; it was just a short story, as I recall, as well.
Anyone recognize it?