This is a community wiki answer. Add at will:
Edo legal system
- TNG: 'Justice' - Wesley Crusher is found guilty of an infraction in a randomly selected "punishment zone" and is sentenced to death.
J'naii legal system
- TNG: 'The Outcast' - Soren, a closeted female in a strictly genderless society, is brought before a tribunal as a formality before having her gender forcibly neutralized.
Banean legal system
- VOY: 'Ex Post Facto' - Tom Paris is convicted of murder on an alien world, and his punishment is to witness the murder from the victim's perspective every 14 hours.
Mari legal system
- VOY: 'Random Thoughts' - B'Elana Torres is punished for having violent thoughts. The punishment is to remove the thoughts from her memories.
Metron legal system
- TOS: 'Arena' - The case Kirk vs. Gorn is resolved in a trial by combat.
Voth legal system
- Voyager: 'Distant Origin' - The Voth put one of their people on trial for heresy.
Ventaxian legal system
- TNG: 'Devil's Due' - Data arbitrates a hearing to decide whether a 1000-year-old contract gives "Ardra" the right to enslave the planet.
Borderline case: Argrathi legal system
DS9: 'Hard Time' - O'Brien experiences a 20-year sentence in his mind and has difficulty re-integrating. This is borderline because everything onscreen occurs in the cell, right after the sentence is applied and back on DS9. However, when he is being discharged, there is the following line, which borders on legal proceedings:
RINN: The Argrathi Authority has been conducting a review of your case. Do you wish to add anything to the record?
Borderline case: Treaty with the Sheliak Corporate, arbitrated by the Grizzelas
- TNG: ‘The Ensigns of Command’ – Picard outsmarts/outmaneuvers the Sheliaks
by invoking a clause in the Treaty of Armens
(between the Federation and the Sheliak Corporate)
saying that disputes should be arbitrated by a third party.
Picard chooses the Grizzelas,
who are in a species-wide hibernation, and will be for another six months
(leading the Sheliaks to agree to delay occupying Tau Cygna V for a few weeks).
This is borderline because it concerns a treaty that the Federation is party to,
but it does involve two “one-off” alien species.