There was more involved than elimination of the Jedi order. Darth Sidious had risen to power as his normal ego, Palpatine, using his Force abilities to manipulate events both within and without the Republic. He was the driving "Force" (pun intended) behind the Separatist movement which led to the militarization of the Jedi order AND the raising of a Clone army. There had been a Republic Navy, but it seemed to be more like a Galactic Coast Guard, chasing smugglers, pirates, and quelling the occasional outbreak of warfare between star systems. The Jedi themselves seemed to be more a Gendarmerie and a diplomatic service than an officer corps. Darth Sidious needed for the Clone Wars to be prosecuted in order to drawn in all interested parties into the conflict, knowing that eventually, after the loss of many (including Jedi), that the Republic would prevail, but would be weakened from without and within by internal strife. And in the eyes of the Senate and the public, would undoubtedly saw Palpatine as a heroic figure, would not the attempt of the Jedi to arrest the Chancellor be seen as a coup d'etat?
Luke: Rejoice, good people. Palpatine the terrible shall rule no more!
Empire Citizen #1: But Palpatine was the greatest ruler we ever had.
Luke: What?
Empire Citizen #1: He built roads, hospitals, libraries ...
Empire Citizen #2: To us, he was Palpatine the Consensus Builder.
(Yes, I know, I've ripped off the Simpsons...)
As another pointed out, since part of the takeover was getting Anakin won over to him, Palpy also had to wait until Anakin would become dissafected with the Jedi over having to choose his wife and family or his status as a Jedi Knight. Their refusal to confer upon him the status of "master" after all he'd done in the Clone Wars did much to embitter Anakin towards the order. Palpatine was a patient and wily old bastard, willing to wait, until, like the cagey chess player he was, he'd moved his pieces where they needed to be, then strike.