In the Doctor Who Children in Need minisode Time Crash, the Fifth and Tenth Doctors briefly meet, and the Tenth saves them from a destructive paradox by cancelling out a supernova with a black hole. He was only able to do this because he had retained his memories of the incident ever since he'd been the Fifth Doctor:
FIVE: Far too brilliant. I've never met anyone else who could fly the TARDIS like that.
TEN: Sorry, mate, you still haven't.
FIVE: You didn't have time to work all that out. Even I couldn't do it!
TEN: I didn't work it out. I didn't have to.
FIVE: You remembered.
TEN: Because you will remember.
FIVE: You remembered being me, watching you doing that... You only knew what to do because I saw you do it.
But we've seen in other multi-Doctor episodes that only the latest incarnation of the Doctor is ever able to remember their shared experiences. From the 50th anniversary special, The Day of the Doctor:
WAR: I won't remember this, will I?
ELEVEN: The time streams are out of sync. You can't retain it, no.
So how come the Fifth Doctor could remember anything at all? And given that he could, how much could he remember? When he finally regenerated into his tenth incarnation, did he think "ah, it's this face - I knew this one was coming"? (Because it sure didn't look like it.) When the Master showed up in Utopia, did he think "ah yes, I remember myself telling me he was going to show up again"?
I realise the answer may just be "timey wimey", but I'm hoping for something a bit better than that.
And this isn't a dupe of this question, which is asking about the bootstrap paradox rather than why the Fifth Doctor could remember anything at all.