I believe you're going to have to wait until something else in Phase 4 or season 2 of Loki itself for a definitive answer. At the moment though we have two conflicting pieces of information from Marvel themselves so it seems it isn't finalised yet.
Farahani, the production designer, has stated that Loki was actually sent to a different timeline. This would mean then that the TVA exists in a higher level outside of time but only outside of time to their own timeline, sort of like Sakaar or the quantum realm. In this case it isn't that the TVA is affected, it's that we're actually in a different TVA.
Farahani reveals that the look and design of the Kang statue was a game day decision designed by the in-house VisDev team. He assures Marvel.com that the entire set dressing of the TVA architecture was identical to the TVA we started the series with, in order “to delay the audience and Loki’s understanding that they were in a different place, that they were in a different timeline.”
Marvel, Loki: Deconstructing He Who Remains’ Life’s Work at the Citadel at the End of Time
Hiddleston gives a slightly different explanation in that the TVA we're in is the same as we've seen previously but it has been changed. He doesn't explain how this happens though. I will note though that outside of time doesn't mean it can't change at all. Kang, or another He Who Remains variant, could have changed things at the TVA without re-writing it all. Note that they have all the technology to do this including TemPads, Reset Charges, variants etc. We also don't really know how long Loki was sat down in the Time Theatre.
It’s a good plan that quickly goes sideways. No sooner does he reach Mobius and Hunter B-15, “He realizes that in the time he's been sitting on that step, something has changed.”
“Something has changed reality, including the reality of the TVA,” Hiddleston says. “The three statues of the Time Keepers are no more. In their place is a statue of Kang. And that his friend Mobius doesn't recognize him and doesn't know who he is. His destabilization in that moment is profound.”
Marvel, ‘Loki’: How the Cliffhanger Ending Sets the Stage for What’s Next
Kate Herron has since gone on record to say something similar to both Farahani and Hiddleston in that it's a different TVA but the same. The reality of it has changed because of what has happened.
Since the TVA resides outside of time, what can you say about the mechanics of the final scene?
So the way I see it in my head is that the TVA exists outside of space and time, but reality and everything as we understood it has completely changed in the last few minutes. With the multiverse branching, how do we know the TVA still exists in that way? We don’t know, and I suppose that’s a big question that will be answered as the show goes on. But in my head, the intention is that Sylvie thinks she’s sending him back to the TVA, but because of the way time and branches are crossing each other outside the window, Loki has unfortunately been sent back somewhere very different. So reality has shifted just by the nature of what He Who Remains said, and the idea is that he’s in this alternate TVA now.
The Hollywood Reporter, ‘Loki’ Director Kate Herron on Casting Jonathan Majors with Peyton Reed and Sylvie’s “Horrible Goodbye”