At the beginning of Duel in the Senate both Yoda and Palpatine managed to knock down each other for few seconds. First Palpatine uses Force Lightning on Master Yoda, Yoda goes down, but Palpatine does not continue with his attack, arrogantly allowing his opponent to stand up (ironically, right after he accuses Yoda of being arrogant: "Your arrogance blinds you Master Yoda").
Then Yoda Force pushes Palpatine, this time Palpatine goes down, but again Yoda doesn't continue with attack (either another Force push, or with lightsaber) . After this Palpatine rises up, they exchange few words and duel begins in earnest. During the duel neither of them gets another chance, i.e. both remain on their feet right up to that fateful blast of Force energy which separates them and ends the duel.
I know that out-of-universe both characters wear plot armor :) , but is there some in-universe explanation why they didn't press on with attack while they had chance ? I imagine Yoda would not want to attack some ordinary opponent while he is down and helpless, but even he had no second thought about killing someone when it was necessary. For example, Yoda decapitated Clone Troopers on Kashyyyk, and Palpatine was far more dangerous than those clones. As for Palpatine, he liked to play cat and mouse games with his victims, but he had to know that Yoda was to powerful for that. After all, he killed Mace Windu right when he got the chance, and Yoda was more stronger in the Force then Windu .