ScreenRant.com compiled a list of all the differences. I won't go through the whole list, but some of the more important changes are:
The woman from Nairomi, who in the Theatrical version we only see giving testimony at a Senate hearing, is given a more significant subplot. We run into her a few times, and ultimately learn that
Lex Luthor paid her to lie in her testimony, to make Superman look bad
Clark's introduction to The Bat is longer, spanning multiple scenes where he investigates the death of the Human Trafficker branded by Batman. It makes somewhat more sense this way, rather than him seeing one news report and being violently perturbed by it
Lois's investigations are longer, and they answer several questions that help the film make a lot more sense
There's a bit of a sequel hook with Lex after the final battle
There are a bunch more changes, with varying degrees of length and necessity, but you can go through ScreenRant's article if you want the full run-down. Most of the additions are either scenes that were cut to dodge the R rating (violence, profanity, substance abuse, nudity), small character moments, or minor clarifying scenes.
The merits of the film itself aside, personally I found the Ultimate Edition to be a better presentation; the plot was tighter, and the character arcs were better-developed. If you're going to buy it anyway, the Ultimate Edition is probably worth the money.