Well, we don't know exactly how long it took Sauron to re-assume physical form - we know how long it took him to make his presence known. But first of all, the One Ring has nothing to do with Sauron being able to come back quickly or not. From Tolkien's letters:
But to achieve this he had been obliged to let a great part of his own inherent power (a frequent and very significant motive in myth and fairy-story) pass into the One Ring. While he wore it, his power on earth was actually enhanced. But even if he did not wear it, that power existed and was in ‘rapport’ with himself: he was not ‘diminished’.
Sauron was not in any way diminished by not wearing the Ring. His powers were simply not enhanced. We are never told, nor is it implied, that one of the powers the Ring enhances is shapeshifting, or the ability to form a body. Sauron forged the One Ring in order to accomplish the following:
This was the essential weakness he had introduced into his situation in his effort (largely unsuccessful) to enslave the Elves, and in his desire to establish a control over the minds and wills of his servants. There was another weakness: if the One Ring was actually unmade, annihilated, then its power would be dissolved, Sauron's own being would be diminished to vanishing point, and he would be reduced to a shadow, a mere memory of malicious will.
The One Ring was to control the wielders of the Three Rings and dominate the minds and wills of his servants. It enhanced his native capability to terrify and dominate minds, which he had posssessed for millennia, documented in The Silmarillion. But without possession of the Ring, he was not diminished, he was just plain old Sauron again.
The more likely explanation is simply that an Ainur's restorative powers are not infinite. When they are injured or killed, those injuries exist on more than a physical level, even though the assumption of material form is supposed to be like raiment. We have a lot of evidence for this. For example, even after he again took material form and announced himself, he was still missing a finger. From "The Black Gate Closed" LotR chapter:
‘Yes, He has only four on the Black Hand, but they are enough,’ said Gollum shuddering. ‘And He hated Isildur's city.’
After he drowned in Númenor, it took him some time to regain a physical form, and he was not able to appear fair anymore.
Sauron was, of course, ‘confounded’ by the disaster, and diminished (having expended enormous energy in the corruption of Númenor). He needed time for his own bodily rehabilitation, and for gaining control over his former subjects. He was attacked by Gil-galad and Elendil before his new domination was fully established.
But Sauron was not of mortal flesh, and though he was robbed now of that shape in which he had wrought so great an evil, so that he could never again appear fair to the eyes of Men, yet his spirit arose out of the deep and passed as a shadow and a black wind over the sea, and came back to Middle-earth and to Mordor that was his home. There he took up again his great Ring in Barad-dûr, and dwelt there, dark and silent, until he wrought himself a new guise, an image of malice and hatred made visible; and the Eye of Sauron the Terrible few could endure.
In The Silmarillion, Sauron's injuries clearly remain regardless of the form he takes:
But no wizardry nor spell, neither fang nor venom, nor devil's art nor beast-strength, could overthrow Huan of Valinor; and he took his foe by the throat and pinned him down. Then Sauron shifted shape, from wolf to serpent, and from monster to his own accustomed form; but he could not elude the grip of Huan without forsaking his body utterly. Ere his foul spirit left its dark house, Lúthien came to him, and said that he should be stripped of his raiment of flesh, and his ghost be sent quaking back to Morgoth; and she said: ‘There everlastingly thy naked self shall endure the torment of his scorn, pierced by his eyes, unless thou yield to me the mastery of thy tower.’
Then Sauron yielded himself, and Lúthien took the mastery of the isle and all that was there; and Huan released him. And immediately he took the form of a vampire, great as a dark cloud across the moon, and he fled, dripping blood from his throat upon the trees, and came to Taur-nu-Fuin, and dwelt there, filling it with horror.
It seems clear that, over thousands of years, as Sauron spent his strength in evil and the domination of others, his ability to change form diminished. This is exactly the same thing that happened to his master Morgoth, except all of Arda was Morgoth's Ring.
Morgoth was eventually restricted to one form, was unable to heal his wounds, and is spending thousands and thousands of years regathering his strength so he can take form (the Valar beheaded him and expelled him from the world) to start the Last Battle. But it was a gradual process, just as it seems to have been with Sauron, that got worse over time.
Alas, I recall an even clearer statement by Tolkien that spending your time dominating others and dominating others weakens Ainur and affects their physical form, but I can't recall where it is. It doesn't seem to be in Letters.
Nevertheless, it seems clear it took Sauron so long to recover because he had been beaten so badly - Elendil and Gil-Galad "overthrew" him - and because he had been beaten so often, and still hadn't even recovered from Númenor. We can certainly imagine Sauron expended enormous energy in inspiring the troops to fight off the Last Alliance and the Siege.
This is almost certainly what I was thinking of, from The Flight of the Noldor.
For now, more than in the days of Utumno ere his pride was humbled, his hatred devoured him, and in the domination of his servants and the inspiring of them with lust of evil he spent his spirit.
This was Morgoth, but Sauron did the same. Sauron's personal strength declined with time because of this.