I think there are two factors that allow Master Chief to succeed against impossible odds:
- He is highly trained, heavily augmented, and naturally skilled.
- He is lucky.
I did my research, watched as you became the soldier we needed you to be. Like the others, you were strong and swift and brave. A natural leader. But you had something they didn't. Something no one saw... but me. Can you guess? Luck. -Cortana
The first factor is what makes the SPARTANs so effective in battle. John in particular survived the entire Human-Covenant war, and for the majority of that time he had less effective armor than what he wore during Combat Evolved. When players play the Halo games, it becomes clear that even Elites are no match for the Flood, even though their armor is just as good as the Master Chief's. The latter is just more skilled than the former, which keeps him alive longer.
The second factor is what made Cortana choose the Chief over the other SPARTANs. For whatever reason, he just seemed luckier than everyone else. In the video games, I believe this is how you can explain dying twenty times in a single level before actually beating it; the odds say Master Chief should be dead many times over in each game, but somehow he keeps on surviving, almost as if the universe keeps on choosing the potential future where the Chief survives instead of the dozens where he dies. I don't think there's a canon explanation for why Master Chief is so lucky, just that Cortana, an inhumanly intelligent AI, knows that he is.
I have another theory that might explain this, though I have less evidence to back it up. In Halo 3, the Flood has evolved into the Pure Form. This shows that in a span of weeks (the time between Halo 2 and 3) the Flood can adapt into stronger and more versatile forms. It's likely that when the Forerunners faced the Flood, the Flood had more time to evolve and become more deadly. When the Flood was released in Combat Evolved, it had to start all over, and thus was easier to fight.