How does time work in "Avengers: Endgame"?

Question

We know thanks to the Ancient One's explanation in Avengers: Endgame that when an Infinity Stone is removed from its own timeline and goes to the past or future, that an alternate reality is created. However, Bruce says if they return the Stones to the same time, that specific reality won't be affected and it'll be as if nothing happened.

My question is: the survival of the universe and all realities seem to be hinging on if the Stones get returned, so how does time work in the other realities/universes?

As soon as, let's say the Time Stone, is taken, that reality which it was taken from splits off and becomes a new one instead of the current one. But Bruce (and the Avengers) spend a lot of time using the Stones. So what is happening in that alternate reality with the Stones taken?

The entire fate of time itself seems to rest on if they return the Stones, not when. But, to me, this doesn't seem plausible because if it does rely on if they return the Stones (not when), then that other reality can only be doing so many things.

• That alternate reality can be moving time as usual, but if this is true, then a lot of things will have happened in all three realities where the Stones were taken and them taking the stones doesn't eliminate any of the risks. If time keeps moving as usual then the alternate realities are still doomed without the Stones.
• That alternate reality could have been frozen in time. However this doesn't seem plausible as that invalidates all the risks.
• That alternate reality could be moving a lot slower or faster in time, however nothing seems to suggest this

So really that leaves only the first option but again, that doesn't make sense. If it takes them 10,000 years to return the Stones, supposedly that would mean the alternate reality is fine. But if time doesn't matter, why does the "if" matter? Why not just wait a million years to return the Stones? Why not just wait an eternity to return the Stones?

Answer 1

I think you've fundamentally misunderstood the argument between the Ancient One and Bruce Banner.

When the Ancient one shows the effect of removing a stone from the timeline, there are 2 separate branches of time created - Main Timeline continues as normal once the Avengers return to it, and the Alternate Timeline starts off from when the stone was removed.

Since each branch has its own variant of people, by moving a stone from Alternate to Main, the Avengers would have saved half the universe in the Main timeline (by undoing the snap) at the cost of the entire universe in Alternate timeline. This is a negative game - millions will suffer.

What Banner proposes is that when the Avengers are finished with undoing the snap, they will instantly return to where the stones were taken out, and return them back in. In essence, this will close off Alternate time so that it doesn't actually branch off. While it may not have each individual stone for, say, seconds, the Main timeline continues on without Stones as it would have anyway. Direct quote from Banner:

No, but we can erase it. Because once we are done with the stones, we can return each one to it's own time line at the moment it was taken. So, chronologically, in that reality, they never left.

This doesn't affect how Time flows in those realities, what it is doing is saying that 4:37 PM-AT, the Reality "lost" the Time stone, and at 4:38 PM-AT, it "regained" the Time stone. There's no special flow involved, as long as nothing momentous happens between those two timestamps* in Alternate Timelines (AT). The time it takes in Main Timeline could be as long as it takes.

This leaves the Ancient One with this concern:

But you are leaving out the most important part. In order to return the stones, you have to survive.

Consider if the Avengers lose to Thanos? Now the Alternate timeline is doomed, because it lost the Time stone, and the Main timeline is even more doomed; Thanos wipes it out completely instead of removing half of all life.

Consider if the Stone was returned with a gap of 1000 years-AT? Well, AT has spent 1000 years without an Infinity Stone, so it might suffer those negative consequences mentioned. That's clearly not acceptable.

Consider if the Avengers kept the stones for 10,000 years, then went back to 4:38 PM-AT to return it? In theory, this makes no difference to the AT, but this is a theory. The Avengers will age in 10,000 years, they will face other threats, they may lose, they may be corrupted. There is no specific gain to the AT in keeping the stones longer in MT, and it increases the risk that the stone will never make it back to where it was supposed to be, and millions will suffer.

Now, the Ancient one has no reason to believe Banner isn't being genuine (or any other Avenger, including Dr. Strange) in wanting to return the stones, so she has no reason to believe they will delay and cause undue risk, so her main concern has to be on "What if you lose, and can't make it back?"

Hence, again:

In order to return the stones, you have to survive.

As it turns out, once she hears that Doctor Strange willingly gave it away, that is what convinces her that someone may have seen a future where this does happen as planned, as opposed to the Avengers making desperate if well-intentioned measures to try and save something.

* Of course, we do know that Loki gets his hands on the Tesseract, and the consequences of that have yet to be explored.

Answer 2

Passage of Time is not a Problem when one is a Time Traveler

As asked, the question operates under the impression that how much time passes in the alternate timelines while each Stone is absent is relevant. One cannot forget that the Stones are being returned by a time traveler. No significant time needs to pass in any given timeline's Stone-less state.

For example, if the time traveler takes a Stone and departs at 2015/05/11 17:01:22.33, a time traveler with the Stone could return at 2015/05/11 17:01:22.34 (one hundredth of a second after having "left") and give it back.

In effect, nothing happens in the other timelines because the stones are not gone long enough to have an impact.

In the cases where the Stone has to be returned in some form other than the rock, it still doesn't matter. The time traveler has well... all the time in the world... to figure out how to put the Stone back in the cube, and so on. Once the time traveler has that figured out, they can show up to make the swap back whenever they like.

Failure is (Not) a Possibility

The problem is not the passage of time, despite the question having three bullet points detailing different possibilities. The problem is success or failure of the operation as a whole. If the Avengers fail, none of the stones will be returned.

In the case of the Time Stone, the Ancient One's objection is:

In order to return the stones, you have to survive.

Fortunately, she has a great respect for Doctor Strange and what he will become. When she learns that he gave up the Stone because he saw this whole chain of events as the "one valid solution", she gives up the Stone to allow his plan to move forward. Without all the stones, the plan will fail.

BRUCE BANNER: Then, why the hell did Strange give it away?
THE ANCIENT ONE (2012): [Surprised] What did you say?
BRUCE BANNER: Strange, he gave it away. He gave it to Thanos.
THE ANCIENT ONE (2012): Willingly?
BRUCE BANNER: Yes.
THE ANCIENT ONE (2012): ...Why?
BRUCE BANNER: I have no idea. Maybe he made a mistake.
THE ANCIENT ONE (2012): Or I did. [She returns to Banner control of his Hulk body and opens the Eye of Agamotto, revealing the Time Stone] Strange is meant to be the best of us.
BRUCE BANNER: So he must have done it for a reason.
THE ANCIENT ONE (2012): I fear you might be right.
BRUCE BANNER: Thank you.
THE ANCIENT ONE (2012): I'm counting on you, Bruce. We all are.