Have been trying to remember this for a while now; there were four books I read them in the late 80's or early 90's, each set in a different, but related, world. Sort of a sword and sorcery theme. I seem to remember that there were different main characters in each book, but there were two races of humans, one that could use magic (more powerful ruling class) and one that couldn't (compensated by being more physically fit/agile, more prone to violence). The first book was narrated from the pov of someone from non-magic race; they had maybe lost a war and were fighting for survival? Against oppression? Also seem to think that both races, as well as each separate world were related somehow - different sides of same original race/world or something of that sort. Does any of this ring any bells? Haven't found anything on google yet...
In the Death Gate Cycle by Margaret Weis and Tracy Hickman, the Sartans and the Patryns, two powerful races descending from humans, are at war. The Sartans sent the Patryns in a Labyrinth world from which Xar the Patryn escaped.
The Earth was destroyed. Four worlds were created out of the ruin. Worlds for ourselves and the mensch: Air, Fire, Stone, Water. Four Gates connect each world to the other: Arianus to Pryan to Abarrach to Chelestra. A house of correction was built for our enemies: the Labyrinth. The Labyrinth is connected to the other worlds through the Fifth Gate: the Nexus. The Sixth Gate is the center, permitting entry: the Vortex. And all was accomplished through the Seventh Gate. The end was the beginning. — Weis & Hickman, Into The Labyrinth pg. 9
There are 7 books in the series so this may not be what you're looking for but the description is pretty close otherwise.