The short version is that this has not yet been addressed. Moffat had an explanation (based on an earlier concept of Davies'), but it doesn't seem to have made it to the show before his departure.
Word of God
Steven Moffat claimed that he had an explanation for Capaldi's two previous appearances in the "Whoniverse", which was substantially based on an explanation Davies developed to explain Capaldi's two appearances:
"We are aware that Peter Capaldi’s played a part in Doctor Who before and we’re not going to ignore the fact," Moffat told Nerd3. "I’ll let you in on this. I remember Russell [Davies] told me he had a big old plan as to why there were two Peter Capaldis in the Who universe, one in Pompeii and one in Torchwood. When I cast Peter, [Russell] got in touch to say how pleased he was, I said 'Okay, what was your theory and does it still work?' and he said 'Yes it does, here it is'. So I don’t know if we’ll get to it... we’ll play that one out over time. It’s actually quite neat".
However, he was coy on what that explanation was. It doesn't seem to have made it into the show before his and Capaldi's respective departures (I'll cover the closest we came in the next section), and it's not clear if Chris Chibnall, the new showrunner, will pay it any mind in future seasons.
Canon
Dialogue in "The Girl Who Died" implies that the Doctor exerts some (possibly subconscious) influence over his own face; during the episode he comes to a realization, and says:
Clara: What? What's wrong?
Doctor: My face.
Doctor: (flashback) Who frowned me this face? Why this one? Why'd I choose this face?
Clara: Doctor, what's wrong with your face?
Doctor: (present-day) I think I know why I chose it.
Doctor: (flashback) It's like I'm trying to tell myself something.
Doctor: I think I know what I'm trying to say.
Doctor Who Series 9 Episode 5: "The Girl Who Died"
This is the probably the closest we came to actually learning Moffat's explanation on the show, but you'll note that it doesn't explain Capaldi's appearance as John Frobisher on Torchwood.
There's evidence that the High Council of Time Lords can exert influence over the "face" a regenerating Time Lord gets; when the Second Doctor is captured and sentenced to a forced regeneration, they give him the chance to pick his new face. He's presented with a number of options (all of which he, quite naturally, hates):
Time Lord: Your appearance has changed before, it will change again. That is part of the sentence.
Doctor: You can't just change what I look like without consulting me!
Time Lord: You will have an opportunity to choose your appearance.
Doctor: Oh, well, that's not so bad. But I warn you, I'm very particular.
Time Lord: Here is your first choice.
[A number of sketches are shown]
Doctor: Oh he's too old! Well he's too fat, isn't he. No, he's too thin. That one's too young. Oh now, that won't do at all. It's ridiculous.
Doctor Who Season 6 Episode 7: "The War Games Part 10"
However, it's never made clear where the Time Lords got those faces from. It's also not clear if this is something they can (or do) influence on a normal basis; the Doctor is a somewhat special case.
On at least four occasions, the Doctor has suggested that the regeneration process is random1, 2:
As the Seventh Doctor:
Seventh Doctor: Even so, that's probably how I am now. You don't understand regeneration, Mel. It's a lottery, and I've drawn the short plank.
Doctor Who Season 24 Episode 1: "Time and the Rani"
As the Ninth:
Ninth Doctor: I might never make sense again. I might have two heads, or no head. Imagine me with no head. And don't say that's an improvement. But it's a bit dodgy, this process. You never know what you're going to end up with.
Doctor Who Series 1 Episode 13: "The Parting of the Ways"
Later on, the Tenth Doctor gets one over on the Eleventh:
[The Tenth and Eleventh Doctors flourish their Sonic Screwdrivers at each other]
Tenth Doctor: Compensating?
Eleventh Doctor: For what?
Tenth Doctor: Regeneration. It's a lottery.
Doctor Who 50th Anniversary Special: "The Day of the Doctor"
Thanks to Maurycy Zarzycki for reminding me that, while regenerating, the War Doctor hopes for certain parts of his appearance to be less noticeable, suggesting that he can't directly influence them:
War Doctor: [Begins regenerating] Oh yes, of course. I suppose it makes sense. Wearing a bit thin. I hope the ears are a bit less conspicuous this time.
Doctor Who 50th Anniversary Special: "The Day of the Doctor"
Ohila, High Priestess of the Sisterhood of Karn, also says that regeneration is a random process in the minisode "The Night of the Doctor":
Ohila: Mock us if you will, but our elixir can trigger your regeneration, bring you back. Time Lord science is elevated here on Karn. The change doesn't have to be random. Fat or thin, young or old, man or woman?
Doctor Who Minisode "The Night of the Doctor"
However, the Doctor is not the only Time Lord to regenerate on-screen; on at least three occasions, we see that other Time Lords have some ability to influence their appearance:
This is played for laughs with Romana, as she tries on four bodies before settling on one3. The clip is on YouTube, and the transcript of the episode is here:
When the Master regenerates, his comments indicate that he's influencing his appearance somewhat:
Master: Killed by an insect. A girl. How inappropriate. Still, if the Doctor can be young and strong, then so can I. The Master reborn.
Doctor Who Series 3 Episode 11: "Utopia"
Curiously, Mels demonstrates the process twice: one random, one less so:
Mels: Last time I did this, I ended up a toddler in the middle of New York.
[...]
Mels: Shut up, Dad. I'm focusing on a dress size.
Doctor Who Series 6 Episode 8: "Let's Kill Hitler"
However, even in these cases it's unclear how much influence they exert over the specifics of their appearance.
Interestingly, Capaldi isn't the first time a Time Lord character was played by an actor who had been on the show previously4:
Colin Baker, who played the Sixth Doctor, had appeared as a Gallifreyan guard in the Fifth Doctor story Arc of Infinity, the first story of season 20. Baker would debut as the Doctor in the next season, beginning in the season 21 finale The Twin Dilemma. The resemblance was never mentioned
Lalla Ward debuted as Fourth Doctor companion Romana's second regeneration in the season 17 premiere Destiny of the Daleks. Ward had appeared in the previous story (The Armageddon Factor) as Princess Astra, the political leader of a Cold War expy state. Unlike the Colin Baker example, the resemblance was mentioned on the show; the Doctor tells Romana off for going around using another person's body, but ultimately gives in
Josh in comments reminds me of The Curator, the character played by Tom Baker in "The Day of the Doctor." Baker's dialogue heavily implies that he is a future regeneration of the Doctor:
Doctor: I could be a curator. I'd be great at curating. I'd be the Great Curator. I could retire and do that. I could retire and be the curator of this place.
Curator: You know, I really think you might.
Doctor: I never forget a face.
Curator: I know you don't. And in years to come, you might find yourself revisiting a few. But just the old favourites, eh?
Doctor Who 50th Anniversary Special: "The Day of the Doctor"
If we take this at face value (an admittedly dubious assumption in Doctor Who), it would seem to imply that the Doctor will be able to exert some control over his appearance in the future, even if he can't now.
Speculation
Since you asked for theories, I remember reading this one published on Tor.com in 2012 (Updated in 2013). This interesting theory suggests that the appearance (and mannerisms) of one regeneration is a subconscious reaction against the attitudes and lessons-learned (and, often, perceived failures) of the previous one.
Whether this theory is in any way compatible with Moffat's remains to be seen.
1 I'm deliberately not counting occasions immediately following a regeneration where the Doctor complains about his new appearance (Except in the case of the Seventh Doctor, but only because he explicitly mentions the randomness); nearly all of the Doctors have done this (even River Song does). A notable exception is the Sixth Doctor, who is actually pleased with his new appearance. Ironic, considering.
2 The First Doctor comes close in the 2017 Christmas special, "Twice Upon a Time", remarking:
First Doctor: I assumed I'd get younger.
Doctor Who "Twice Upon a Time"
But I'm relegating it to a footnote because it's not unambiguously referring to the process as random
3 Because this sequence is so radically unlike how regeneration has worked before or since, it tends to be ignored by most fans. At least one of the Doctor Who novels (I can't remember which) retconned it by saying that the TARDIS had imprisoned Romana in a jealous rage, and was projecting a false image. YMMV.
4 To say nothing of the number of times old extras have ascended to become companions; Nicholas Courtney and Karen Gillian are two examples that spring to mind, with Jacqueline Hill doing it the other way round.