From the Architect's speech, transcribed here:
You are here because Zion is about to be destroyed. Its every living inhabitant terminated, its entire existence eradicated. ... The function of the One is now to return to the source, allowing a temporary dissemination of the code you carry, reinserting the prime program. After which you will be required to select from the matrix 23 individuals, 16 female, 7 male, to rebuild Zion. Failure to comply with this process will result in a cataclysmic system crash killing everyone connected to the matrix, which coupled with the extermination of Zion will ultimately result in the extinction of the entire human race.
So, if a previous One had been tempted not to return to the source, probably it would have been because they wanted to try to prevent the extermination of Zion somehow, and maybe find a way to avert the "system crash" (in this iteration of the cycle, we saw no immediate system crash when Neo chose the left door, and the disruption to the Matrix caused by Smith hadn't happened on previous cycles, so maybe the crash was somehow supposed to be contingent on the machines being successful in destroying Zion). The choice would have basically been "go along with our plan to wipe out Zion and reboot it, or try and fight us and risk the likely extinction of humanity."