Mace Windu seemed to truly believe that he and the other 3 Jedi would defeat the Sith Lord. Did not he consider the possibility he and allies would be overwhelmed? Shouldn't he at least set up a back-up plan? (Like why not set up escape plan or take clones with him - as at this time order 66 had not yet commenced so clones were still allies).
No, he was not.
As per the film's official novelization by Matthew Stover, he already had the best fighters he could:
Now Obi-Wan did face him. “Palpatine faced Mace and Agen and Kit and Saesee—four of the greatest swordsmen our Order has ever produced. By himself. Even both of us together wouldn’t have a chance.”
Moreover, he himself was basically correct - he probably COULD have defeated Darth Sidious, given his peculiar Force talents (Shatterpoint ability and Vaapad style). He HAD defeated Sidious in direct combat, as a matter of fact - until he was betrayed by Anakin (OK, in all fairness, there's a possible case to be made that Palpatine was faking it 100% and fully threw the fight - but that's not the sense I get from the Novelization's full text).
He was not afraid. The darkness had no power over him. But -
Neither did he have power over it.
Vaapad made him an open channel, half of a superconducting loop completed by the shadow; they became a standing wave of battle that expanded into every cubic centimeter of the Chancellor’s office. There was no scrap of carpet nor shred of chair that might not at any second disintegrate in flares of red or purple; lampstands became brief shields, sliced into segments that whirled through the air; couches became terrain to be climbed for advantage or overleapt in retreat. But there was still only the cycle of power, the endless loop, no wound taken on either side, not even the possibility of fatigue.
He could feel the end of this battle approaching, and so could the blur of Sith he faced; in the Force, the shadow had become a pulsar of fear. Easily, almost effortlessly, he turned the shadow’s fear into a weapon: he angled the battle to bring them both out onto the window ledge.
.... Which might have gone on forever, if Vaapad were Mace’s only gift.
Out where the shadow’s fear made it hesitate. Out where the shadow’s fear turned some of its Force-powered speed into a Force-powered grip on the slippery permacrete.
Out where Mace could flick his blade in one precise arc and slash the shadow’s lightsaber in half.
One piece flipped back in through the cut-open window. The other tumbled from opening fingers, bounced on the ledge, and fell through the rain toward the distant alleys below.
Now the shadow was only Palpatine: old and shrunken, thinning hair bleached white by time and care, face lined with exhaustion.
“For all your power, you are no Jedi. All you are, my lord,” Mace said evenly, staring past his blade, “is under arrest.”