Official policy is that you don't need to have seen any other part of the MCU to enjoy Age of Ultron, except possibly for The Avengers. In an interview with AlloCiné, Whedon commented on this:
"I have to make my movie assuming that people will only have seen the first one, or possibly not even seen the first one. I can't assume that everybody went to see Thor [The Dark World], Captain America [The Winter Soldier], and Iron Man  in-between. I have to go from one movie to the next and be true to what's happened, but not be slavish to it... The model I'm always trying to build from, my guiding star, is The Godfather Part II where a ton has happened in-between and it's a very different movie [from The Godfather], but you don't need any information: it's there in the film."
In theory, watch The Avengers and you'll be good to go. You may not get the complete experience without having seen the other dozen MCU movies, but you shouldn't miss any of the important elements of the story.
Now that I've finally seen the movie, I confirm that Whedon was telling the truth. The main plot is perfectly comprehensible without having seen any other MCU movies. Having seen The Avengers is sufficient to introduce you to most returning characters (the Avengers themselves, Maria Hill, Fury, Erik Selvig) and to Loki's scepter.
There are a few passing references to events from other MCU movies, and references to the Infinity Stones, that won't make sense unless you've seen other movies:
- James Rhodes (War Machine) and Sam Wilson (Falcon) each have brief appearances. They were introduced respectively in Iron Man (and in Iron Man 3 as War Machine) and Captain America: The Winter Soldier
The major plot twist from Captain America: The Winter Soldier and season 1 of Agents of S.H.I.E.L.D. is referenced.
This event being the fall of S.H.I.E.L.D. The fact that character have a hard time seeing the difference between S.H.I.E.L.D. and Hydra is referenced at least once
Thor remarks that four of the Infinity Stones have emerged in the last four years, and they are shown in a montage. The Tesseract and Loki's Sceptre both featured prominently in The Avengers, although the Tesseract was introduced in Captain America, but the other two are from Thor: The Dark World and Guardians of the Galaxy. Keen covered the Infinity Stones in his answer here (No spoiler tags in use there, so be careful if you care about that sort of thing)