In Mass Effect 2 you learn that the Council still believes the Geth build Sovereign. So why did the Council (or anyone else) never tried to destroy a race supposedly capable of building a ship that bested the Ascension?
At the very beginning of Mass Effect 2, Shepard is actively searching for geth holdouts. According to Anderson, by the time the game properly begins, the geth are no longer appearing regularly, and conflict between the geth and Council-aligned forces has become quite rare. The Council presumably interpreted this as a de facto victory.
So the question is, why didn't they send a whole fleet after the geth right away? Because sending a fleet into the Perseus Veil would almost surely result in war with the Terminus systems. This is specifically discussed during the events of Mass Effect; Udina demands exactly this response and gets shot down (in favor of making Shepard a Spectre, so he arguably got the better end of the deal anyway).
After Mass Effect, the Citadel Defense Force is greatly weakened, possibly including the loss of the Destiny Ascension. Starting two simultaneous wars at that point is simply unsustainable.
So why did the Council (or anyone else) never tried to destroy a race supposedly capable of building a ship that bested the Ascension?
Because they believed the Geth built a ship that bested the Ascension, though that is just rumor. They also have little to no idea of how strong the Geth navy is, where their bases are, or if it really was the Geth. Most of all, the Geth in ME 1 Did not attack anyone except the Quarians. The Council and the intergalactic society at large still believed that the Geth were the Quarians problem and burden. Any remaining Geth in the Terminus systems after Sovereign was defeated by the FULL Systems Armada.
Mainly the Geth kept to behind a spacial divide, the Perseus Veil. Anyone that ventured in were overwhelmingly eradicated by the Geth, to keep themselves hidden. From the Codex:
The Veil's total opacity prevents Council intelligence from surveying geth activity. Theoretically, the geth could be preparing a devastating attack against which the Council could be defenseless, or the geth could have died out, so that the defense budget against them could be gaining the Alliance nothing but economic ruination.
Aside from that, the Council in every Mass Effect is full of morons masquerading as politicians. Not only are they slow to act, They are wrong at every turn, as demonstrated by your Shepard. In ME2, they (either the Council you let live, or the ones that took over after you let them die) believe the threat is gone, even though Shepard proves them wrong again in the story. After Sovereign's destruction, most of it was destroyed, and provided little evidence of Reaper existence.
In short, the Council is cheap, afraid, slow and short sighted.