"The Cave of Night", a short story by James E. Gunn; first published in Galaxy Science Fiction, February 1955, available at the Internet Archive; as is the X Minus One radio dramatization (which was the subject of another question). It has been reprinted in a number of anthologies; does any of these covers ring a bell?
Rev McMillen, first man in space, stuck in Earth orbit:
Before Rev could override the automatics, the motors had flamed for almost half a minute. The fuel he had depended upon to slow the ship so that it would drop, reenter the atmosphere and be reclaimed by Earth was almost gone. His efforts to counteract the excess speed resulted only in an approximation of the original orbit.
The fact was this: Rev was up there. He would stay there until someone came and got him.
And there was no way to get there.
His receiver is gone, but he transmits to Earth:
"Do you hear me down there? Sometimes I wonder. You can't see me now. I'm in the Earth's shadow. You'll have to wait hours for the dawn. I'll have mine in a few minutes.
"You're all busy down there. I know that. If I know you, you're all worrying about me, working to get me down, forgetting everything else. You don't know what a feeling that is. I hope to Heaven you never have to, wonderful though it is.
"Too bad the receiver was broken, but if it had to be one or the other, I'm glad it was the transmitter that came through. There's only one of me. There are billions of you to talk to.
"I wish there were some way I could be sure you were hearing me. Just that one thing might keep me from going crazy."
The rescue ship gets there a little too late:
The tension mounted unbearably. We waited—a nation, a world—for relief.
At eighteen minutes less than two hours—too soon, we told ourselves, lest we hope too much—we heard the voice of Captain Frank Pickrell, who was later to become the first commander of the Doughnut.
"I have just entered the ship," he said slowly. "The air-lock was open." He paused. The implications stunned our emotions; we listened mutely. "Lieutenant McMillen is dead. He died heroically, waiting until all hope was gone, until every oxygen gauge stood at zero. And then—well, the airlock was open when we arrived.
"In accordance with his own wishes, his body will be left here in eternal orbit. This ship will be his tomb for all men to see when they look up toward the stars. As long as there are men on Earth, it will circle above them, an everlasting reminder of what men have done and what men can do.
"That was Lieutenant McMillen's hope. This he did not only as an American, but as a man, dying for all humanity, and all humanity can glory for it.
"From this moment, let this be his shrine, sacred to all the generations of spacemen, inviolate. And let it be a symbol that Man's dreams can be realized, but sometimes the price is steep.
And the twist ending:
The whole thing came back to me recently, an overpowering flood of memories. I was skirting Times Square, where every face is a stranger's, and suddenly I stopped, incredulous.
"Rev!" I shouted.
The man kept on walking. He passed me without a glance. I turned around and stared after him. I started to run. I grabbed him by the arm. "Rev!" I said huskily, swinging him around. "Is it really you?"
The man smiled politely. "You must have mistaken me for someone else." He unclamped my fingers easily and moved away. I realized then that there were two men with him, one on each side. I felt their eyes on my face, memorizing it.
[. . . .]
I think about Rev and the life he must have now, the things he loved and can never enjoy again, and I think perhaps he made the greater sacrifice
I think sometimes he must wish he were really in the cave of night, seated in that icy control chair 1,075 miles above, staring out at the stars.