The Jedi learned that Palpatine was a Sith Lord when Palpatine told Anakin. Anakin decided not to attack Palpatine at the time but instead to report him to the Jedi Council, at which point he informed Mace Windu about Palpatine. When Windu learned the truth about Palpatine he had to realize that time was of the essence: a Sith Lord was the Supreme Chancellor of the Republic and knew that the Jedi were aware of him (because he told Anakin). Windu knew that Palpatine, as the Supreme Chancellor, had considerable power that he could potentially use against the Jedi before the Jedi could act. Thus, Windu had to have concluded that he needed to attack Palpatine immediately.
At the time, Yoda was leading the Republic troops in the Battle of Kashyyyk and was unavailable to help. Nonetheless, Windu had Agen Kolar, Saesee Tiin, and Kit Fisto with him. It was reasonable for him to think that four Jedi Masters -- all on the Jedi Council -- would be sufficient to arrest one Sith Lord.
Since (a) time was of the essence, (b) Yoda was unavailable, and (c) it was 4 Jedi Masters vs. 1 Sith Lord, it was reasonable for Windu to attack Palpatine immediately and think that he could win the showdown.
(Review the Episode III synopsis if you don't remember the sequence of events.)