I'm have been trying to find this story/book for several years now without any success.
I must have read this around 1990, give or take a few years. It may have been a (shortish) novel or a novella (published in a bundle or as stand-alone book). I'm certain I've read it twice, once in English and once in Dutch (most likely a translation of the English original).
The main character is a man (I'll call him John, but I don't remember what he was really called) who is a anthropologist/historian/sociologist. He is sent by the Earth government to study/spy on the ruling class of the Galactic Empire that has quite recently annexed Earth.
The Empire is populated by many sub-species of human (with adaptations to various planets) and the ruling nobles are a particular sub-species of Super-Humans. They are of superior intelligence, strength, dexterity, will-power and regard all other sub-species of humanity as inferior.
During the story John learns that these super-humans were originally a group of leading scientists within the Empire. Due to inter-marriage and bio-engineering they gradually evolved into the super-humans and eventually took control of the Empire. They keep very tight control of scientific knowledge and interstellar trade.
There is also a "warrior"-caste of genetically enhanced humans that are bred and indoctrinated to serve the nobles with unquestioning loyalty. These "one man armies" give the rulers the military power to stay in control. The "warriors" primary weapon/armor is a set of silver bands/bracelets around arms, legs and torso that act as exoskeleton enhancing muscle-power, speed and reflexes. The bands are also very powerful energy-weapons. They can project defensive shields and offensive beams. Only the "warriors" and the nobles can use these bands.
The nobles are easily bored and to get entertainment they organise "America got talent" style shows on various planets and invite the winners to travel to the central throne-world. John participates in such a talent-hunt as a matador and stages a bull-fighting demonstration. This sort of barbaric show really appeals to the nobles saturated appetites and John had carefully planned that in order to win his invitation.
While on the throne-world John studies the nobles and learns what he can about their special abilities. However, he finds that, even though they are quite superior people, he can after some practice keep up with the best of them. It seems he is naturally talented in many ways.
He starts to see a pattern: The nobility is quite decadent and there is a lot of political intrigue. In fact John starts to realize it looks a lot like Imperial Rome before the collapse of the Roman Empire.
Additional problem is that many of the nobles are not mentally stable. Their superior intellect is so great it borders on insanity and in some cases crosses that border. However, the nobles are very much in denial about this and turn a blind eye whenever one of them shows strange behavior. Instead they just call it a personal quirk of that person.
The Emperor (supposedly the most perfect human) at times lapses into a sort of dream-state and sees visions. (This is carefully hidden from the general populace and even most nobles don't know.) His cousin (and next in line for the throne) is a power-hungry political schemer who is a borderline psychopath.
The Empire is rotting at the core and the nobility seems unable/unwilling to see that.
Eventually John gets wind of a plot from the cousin to depose the Emperor. This involves intentionally triggering one of the Emperors "lapses of reason" during a public gathering so everyone can see the Emperor is incapable of ruling. The cousin hopes to be able to use that to become the new emperor.
John foils that plot. The Emperor is shocked back into full lucidness and then tells John that he is aware of the problems that plague the Empire and that eventually the Empire must collapse. He sends John back to Earth, together with his love-interest (a noble lady) and with a personal body-guard (one of the warrior-class guys) to keep him out of harms way.
I don't recall if this was explicitly mentioned in the story or just strongly implied, but I distinctly remember that one of the reasons to send John and friends back to Earth is to have some people of power, intellect and reason located far away from the throne-world in a position that they can start picking up the pieces when the Empire eventually collapses.
I hope this gives someone a clue what this story was.