My memories are quite vague, but the big central premise is that while humans comprise the majority of the population of the galaxy, they are clustered in ghettos and are not the dominant species. I believe humans or slaves were used as menial workers. There was possibly some kind of revolt. At some point in human history, the Earth was moved and it was known as "the day that the stars changed" or something like that.
I read this my first year of college which would’ve been 85 through 86, I think. No later than 87, but I’m thinking 86.