The End of the World as We Know It by Dale Bailey. I read it in This Way to the End Times edited by Robert Silverberg.
As you say, the protagonist, Wyndham, is a UPS driver:
On the day the world ended, Wyndham didn’t even realize it was the end of the world—not right away, anyway. For him, at that point in his life, pretty much every day seemed like the end of the world. This was not a consequence of a chemical imbalance, either. It was a consequence of working for UPS, where, on the day the world ended Wyndham had been employed for sixteen years, first as a loader, then in sorting, and finally in the coveted position of driver, the brown uniform and everything.
On the day of the apocalypse he finds his wife has died in her sleep:
Concerned, Wyndham reached down and touched his wife’s face. It was like touching a woman made of wax, lifeless and cool, and it was at that moment—that moment precisely—that Wyndham realized the world had come to an end.
Everything after that was just details.
And at the end of the story he and the last woman in the world are indeed sitting together drinking:
By this point the woman (You want her to have a name? She deserves one, don’t you think?) has started to weep softly. Wyndham gets to his feet and goes into the dark kitchen for another glass. Then he comes back out to the porch and makes a gin and tonic. He sits beside her and presses the cool glass upon her. It’s all he knows to do.
We never do discover what killed everyone. The story is deliberately vague on this point:
You, like Wyndham, may be curious about the catastrophe that has befallen everyone in the world around him. You may even be wondering why Wyndham has survived.
End-of-the-world tales typically make a big deal about such things, but Wyndham’s curiosity will never be satisfied. Unfortunately, neither will yours.
It’s the end of the world after all.
The dinosaurs never discovered what caused their extinction, either