The Minbari surrendered at the battle of the line, but what were their plans if they hadn't surrendered and won?
Up to this point the war itself resulted in relatively few casualties – about 250,000 – so most people are still alive.
Would they simply leave? Establish a colony? Enslavement? Genocide? Something else?
Delenn gives us a fairly unambiguous clue in s2e12 "Acts of Sacrifice" (just over 7 mins in), when she tells G'Kar that the Minbari "nearly exterminated an entire species" (meaning humanity). Since the Minbari killed only a small proportion of the human race, it seems to me that the only feasible interpretation of "nearly" is that they brought a weapon capable of destroying human civilisation to the Battle of the Line and, having breached the Line, were close to the point of using it. It could have been a mass driver (as used against the Narn homeworld by the Centauri a few years later), or something more sophisticated. Either way, it was lucky for both sides that the Minbari encountered Sinclair.
The indication given to the viewer (even if not directly stated) is that the Minbari were going to utterly destroy the Human race, or at least render it into a stone-age community with no space travel.
There are a few things which suggest this, some of which I have put here:
Earth President (from a speech inspiring troops at the Battle of the Line): "We have continued to broadcast our surrender and a plea for mercy, and they have not responded. We therefore can only conclude that we stand at the twilight of the Human race.[...] We do not believe survival is a possibility. We believe that anyone who joins this battle, will never come home again.[...] Though Earth may fall, the Human race must have a chance to continue elsewhere."
Londo: "The humans, I think, knew they were doomed. But where another race would surrender to despair, the humans fought back with even greater strength. They made the Minbari fight for every inch of space."
Sinclair: "We never had a chance. You say we could have won, but you weren't there, you didn't see them! When I looked at those ships, I...I didn't just see my death -- I saw the death of the whole damn human race!"
Delenn (essentially giving the order to start the war in the first place after the death of Dukhat): "Follow them back to their bases and kill them, all of them. All of them! No mercy!"
In particular, the final quote there, from Delenn, is most indicative of what the Minbari intended: the council was divided on action (talk, and find out what went wrong; or assume malice, and return the same), and Delenn had the final say, so her command to "kill them all" can be interpreted as a command to the Warrior caste to end the war by death of the Human race.
Also, while not canon, there is a fan-fiction story of Babylon 5 (Alternate Universe) in which the Minbari did not surrender. The story, titled "A Dark Distorted Mirror" (5 parts, somewhere over a million words in all) can be found here
It's not detailed anywhere.
You can apply some educated guessing based on the one enemy we do see that the Minbari had previously faced - the Streib, who in the episode All Alone in the Night, end up capturing Captain Sheridan and subjecting him to multiple experiments. They had previously captured and experimented with Minbari, and per Delenn, were "taught the depths of their mistake". They are not seen actively monitoring, or controlling, the actions of the Streib afterwards.
The Minbari refer to the Earth-Minbari War as "their holy war", as the killing of their leader carries great weight and caused great rage. It is not unthinkable, from a human perspective, to think that they may have planned to wipe out humans completely. It is very plausible that some factions on Minbar pushed for this (the ones who took in Jha'dur from Deathwalker specifically). It is unlikely the entire race thought so however, or would be apathetic to a genocide.
Delenn was already regretting giving her support to the war in the immediate aftermath of Dukhat's death. It is likely that the Grey Council would have been able to halt the war once the ... "honor", or bloodlust, of the Warriors had been sated. The next step, as Delenn herself noted, was invading their planet.
Clinically, this would probably mean the destruction of any military or space-faring capabilities humans had, along with the government that had been responsible for them.
Practically, this would probably have involved obliterating humans back to some kind of stone age, caused the deaths of billions, and while not practically genocide, would have served to wipe humanity off of anything resembling a star map.
How far the Minbari would have let it proceed is left as an opinion exercise for any given reader.
The movie In the Beginning openly states their end goal was eventual genocide, and that all sides understood that (transcript, all emphasis mine)
General Fontaine, in an early war briefing
FONTAINE: They are moving methodically through the outer colonies, wiping out our defense structures and leaving the colonies vulnerable. Civilian structures are being left alone for now. Now we know the Minbari use a caste system including a warrior caste. And we think they're fighting in a way consistent with that structure, taking out our warriors first, and coming back for the rest later. They intend to eliminate our defensive capability all the way to Earth. Then, with no one to stop them, they'll head back out again and finish the job of wiping out every last man woman and child of the Human race.
Later, Delenn realizes the warrior caste has been reinvigorated by the war against a power that poses no real threat to them. Morann, the leader of the warrior caste, openly confirms their goal is genocide
DELENN: What pleasure can be found in beating an enemy that never had the slightest chance of defeating us?
MORANN: Is that sympathy I hear in your voice, Delenn? I'm surprised. You were the chosen of Dukhat, after all.
DELENN: Dukhat would have never approved of this slaughter!
MORANN: This is simple retribution.
DELENN: This has gone beyond retribution, Morann. This is madness. It is genocide.
MORANN: Yes, it is. And you're quite right, Delenn. The Humans cannot oppose us.
DELENN: So, one can only wonder why the Warrior Caste has embraced this war so enthusiastically. Do you want to hear what I suspect?
MORANN: I'm breathless with anticipation.
DELENN: The Warrior Caste loves to win and hates to lose. It is easier to fight a weaker opponent and be guaranteed victory than to oppose a far more dangerous enemy.
General Lefcourt, speaking to Sheridan just before sending him to meet with Lenonn, Delenn's envoy to start peace talks
LEFCOURT: We've heard that elements of their government want a meeting to discuss a way of ending the war without the annihilation of Earth.
The Earth President, in her final broadcast before the Battle of the Line
PRESIDENT: Our military intelligence believes that the Minbari intend to bypass Mars and hit Earth directly and the attack may come at any time. We have continued to broadcast our surrender and a plea for mercy. And they have not responded. We therefore can only conclude that we stand at the twilight of the Human race.
Delenn, in her final meeting with Kosh before The Battle of the Line.
DELENN: In a little while the final slaughter will begin. I know the others do not want it. Even Morann is tired of war, tired of blood. But the war has taken on a life of its own. And it will continue to its bloody end no matter our feelings.
For a show with multiple pragmatic characters, not a single one disputes the end-game for the Minbari is genocide.
