This is kind of a reach, but could the series be The Book of the New Sun by Gene Wolfe? It was not exactly unfinished by 2000 (it was published 1980–1983), but the follow-up The Book of the Short Sun was (which consists of three novels published 1999–2001).
The story centered around political events in a kingdom and capital, and the rise of a magical/clerical sect.
Not a perfect fit, but perhaps an amalgamation of several plot threads? The series has a backdrop of political events in the kingdom called the Commonwealth and the city Nessus, seat of the Citadel and de facto capital.
There's not a rise of a sect per se, but there are several sects in the story. Notably, the main character Severian belongs to the Guild of Torturers (which is neither magical nor clerical), the female clerical sect of the Pelerines who worship a Messianic figure called the Conciliator, and a pervasive religious cult surrounding the coming of the "New Sun" common to the entire society. The Commonwealth is ruled by the Autarch, who is surrounded by a personality cult.
I vividly remember the first book dealing with a group of brothers taking a long voyage down a river, and stopping at a 'mystical' or forgotten island
This could also fit several subplots in the series, although none of them feature brothers.
- In the first book, The Shadow of the Torturer, the main character Severian and his fellow torturer apprentices (who are like brothers to him; they have grown up together as fellow orphans) swim in Nessus's big river Gyoll, and Severian later travels on the river.
- In the second book, The Claw of the Conciliator, a fairy tale story-within-the-story called "The Tale of the Student and his Son" chronicles the titular son travelling on a ship to a mystical and uncharted island with many inland waterways to slay a monster (in a kind of retelling of the myth of Theseus and the Minotaur).
- In the third book, The Sword of the Lictor, Severian travels on the rivers of a city called Thrax.
- Also in the third book, Severian travels on a huge lake together with a group of natives who live on movable ship-islands, to reach a mystical castle.
There was also, later on, a masked ball at said capital where someone was drowned
In the third book, The Sword of the Lictor, Severian works as a lictor, or bodyguard/executor, in the city of Thrax (so not the capital of Nessus, but still a major city). He attends a masked ball, and receives the order to execute a female guest by drowning. (Spoiler: He doesn't do it, but jumps into the water himself.)
Note also, since you referred to "the first book", that the four-book series has been released in two volumes as well, called Shadow and Claw (containing The Shadow of the Torturer and The Claw of the Conciliator) and Sword and Citadel (containing The Sword of the Lictor and The Citadel of the Autarch).