For those wishing to avoid spoilers, don't play the videos linked here.
There's a few reasons for Kai just not immediately leaving. The end of this video from the very beginning of the movie sets the initial context.
We see two things here.
Oogway challenges Kai by saying that he "sent another" to stop Kai. Kai immediately vows to find him and take his chi as well.
The move to transport from the spirit world back to the real world appears to take several seconds. Unless Kai has defeated or otherwise delayed his opponent for several seconds, he'd probably be vulnerable to an attack which, I'm reasonably assuming, would interrupt the transport.
Next, we have this scene
Here, Kai is reaffirming his promise that he will take Po's chi.
Later on, we have the scene you're asking about.
Kai specifically says, "You brought me back!?" From his tone, I don't really get the feeling that he feels trapped, but that he's simply really annoyed. Po makes a lame attempt to placate Kai, who responds angrily that they'll finish their fight there in the spirit world. This probably puts Kai on tilt a little bit.
But,
the real reason is that Kai wants more power and Po's chi will give him that, and it's not clear if Kai would have had an opportunity to return until after he defeated Po and taken his chi.