Potentially Ghetto by Poul Anderson, originally published in 1954 but anthologised a few times, including in the 1980s.
The hereditary crew are Kithmen, and live in their own community (Kithtown) on Earth; at any one time there may be a few ships and a few thousand of them on Earth. Attitudes towards them change over time, from valued, to tolerated, to persecuted - they have all left the planet in the past and at the time of the story, anticipate that they might need to do so again soon.
Travel takes months to years subjectively, and years to centuries on Earth. The protagonist (Kenri Shaun) has just returned from Sirius on the Fleetwing, which also had a passenger (Dorthy). They have started a relationship during the voyage, and he is planning to leave his people to live with hers - accepting that at best he will be an old man when his friends and family next land.
During the course of the story he discovers how much attitudes have changed to treat the Kith as second class citizens; he can cope with the idea of joining a society he considers dull, but not having to join the society and effectively support the attitudes:
"For you, I could have stood that. But you are asking me to be a tyrant, or at least a friend of tyrants. You're asking me to countenance evil. I can't do it. I wouldn't if I could."
At the end he walks away, choosing to leave Dorthy behind and continue to travel with his own people.