The aliens in this book are an humanoid alien mercenary race, fighting for some other aliens, against humans?.
The Ker are the fighters, the sen are the wise ones, and the kaa the family/non-fighters. Their clan gets almost wiped out because both their employers and humanity fear them, and the last survivors head towards their race' home planet, together with a human captive, who learns about their ways.
The other aliens are some blob-like aliens with little genetic variety.
Ring any bells?
This is the "Faded Sun Trilogy" by C. J. Cherryh
The castes you're referring to are the Kel, the Sen and the Kath
Per wikipedia;
The mri are divided into a rigid structure of three castes: kel, sen, and kath. These are functional rather than hereditary castes, but each has a clearly defined role in their society, and the mri regard any change in those traditional functions as completely unacceptable.
Kel are warriors, and dress in black. Because they interact with outsiders, they veil their faces to preserve the privacy of the People. Furthermore, they are forbidden to read or write, lest they inadvertently betray the ancient secrets of the People. They are masters of both the traditional edged weapons of the People and the modern weapons they use as mercenaries. They play shon’ai, the traditional passing game of the People, with knives.
Sen are scholars, and dress in gold, except for the she’pan or matriarch of the tribe, who dresses in white. They keep the sacred writings of their people, and are celibate. Because they do not interact with outsiders, they do not veil their faces. They play shon’ai with wands.
Kath are the children and those women who do not join either the kel or the sen, and dress in blue. No man past puberty may remain in kath, and those unsuited for kel or sen die. The kath are the lowest caste of the mri, responsible for the upkeep of the tribe's dwellings. Kath play shon’ai with a pair of rounded stones.
