This is a complex question: we have two stories to identify, and the collection they both appeared in.
I. THE FIRST STORY is definitely "Moment Without Time" by Joel Townsley Rogers, first published in Thrilling Wonder Stories, April 1952, available at the Internet Archive.
"One story was about a Russian scientist who was researching time travel, getting close to understanding it. He was arrested and about to be executed by a firing squad. Just before being shot, he has an insight about his work."
Piridov was facing the firing squad in the execution cellars when the solution hit him.
It was the answer he needed, the answer to the mathematical relation of time to the three spatial dimensions. It came to him in the form of an equation, as simple as the German's historic energy-mass formula—"e = mc2", where "c" represents the speed of light.
"He says 'Wait.' And time almost stands still. The bullets are still in the air, out of the rifles, have not yet reached him."
Just so, except that he doesn't way "Wait." He does say "What's happened?"
Only for the sudden coming to him of the time solution, too late to add it to the sum of human knowledge, did he feel regret in this final instant. An equation of blinding whiteness on his closed eyelids. But to be lost with him. . . .
Sergeant Death had shouted "Fire!", it seemed to Piridov, minutes ago. The waiting was painful. Slowly and reluctantly, he opened his eyes on the cellar scene.
The white-painted walls lit by thousand-watt bulbs, the four rifle muzzles pointed at him, the squinted eyes behind the sights, the taut retracted trigger-fingers, squat thick-chested Sergeant Death standing to one side with his hand down at his thigh in the conclusion of the sweeping axe-blade gesture with which he had accompanied the command to fire, the bullets coming at his breast. Four of them, straight at his heart, not more than twelve to fifteen inches away; while from the corner, out of the range of fire, the one-eyed scavenger directed the stream of water around his feet to wash the blood away.
No, the bullets weren't coming. They were motionless in space. The eyes behind the sights were motionless. The water hosing over his feet was motionless, both the stream and the up-splattering drops of it. The red second hand of the electric clock on the wall above the steel door was motionless, the minute and hour hands straight up and down at six.
Piridov brought up a numb hand and felt his throat.
"What's happened?" He moved his throat and lips.
No sound. Absolutely no sound could he hear. The silence and motionlessness remained.
"He leaves the site,"
The way out was long and multiple-guarded, but not labyrinthine. Four separate flights of stairs, a dozen doors, a thousand silent hurrying steps, and he was in the front receiving lobby of the huge old granite tsarist prison—historic Moscow landmark, from which all the modern underground extensions stemmed—and crossing the worn flagstones. Out through the grim barred entrance doors, leaving them open behind him like all the others, past the last machine-gun boxes beneath the stone archway.
In a moment more he was mingled with the motionless throngs of Lubianka Square, beneath the red sunset sky.
"visits his wife and others, has difficulty moving things,"
Another small discrepancy, he has no difficulty moving things. He visits the big bosses in the Kremlin, he visits his son at work, he visits his wife:
Aged Piridov bent over her, brushing his lips across her cheek.
"I have loved you very dearly, Anna," he whispered.
He laid the dewy rose beside her on the pillow. Just for the moment his heart was breaking.
"finally decides it is not worth continuing his life like this. So he goes back to finish being shot."
Piridov went past them as quietly as a shadow. At the farther wall he turned, facing the squad, placing his feet carefully on the dry spots, in the empty spaces of the spattering water, which marked where he had been standing. The bullets in their swift rifled flight were poised twelve inches from his breast.
Piridov closed his eyes.
"I had a dream," he told himself. "A dream of a bright equation within a timeless instant. Of things I love and things I hate. Of a sword sweeping. A dream of water. But I am very tired. So let the dream be over."
And so it was. . . .
"He uses one of the lead bullets to write his newly thought-of formula on the wall."
No, that happened at the beginning of the timeless moment, before his walk around town, because he has nothing else to write with:
The white cell walls themselves! If he only had time to reach the one back of the firing-squad, the whitest and cleanest, and only had something to mark or scratch with. A knife or a nail, or anything. Quickly! Before the swift instant moved on.
The four bullets were poised in space in front of him, like a flight of miniature wingless jets irregularly spaced. They were slightly flattened at their heads by the massed air in front of them, slightly oblate axially in their rifled spin. They were lead, he saw, for hitting power and spread, not steel.
With a breathless terror in him, watching the motionless eyes for indication that his gesture was being detected, he reached his thumb and finger up, and picked the nearest bullet from in front of his breast. He took a tentative step to the side away from the other three that remained suspended in their flight.
"Then he stands in front of the bullets and says "OK". Time resumes and he is shot/killed. The head officer at the execution sees the writing"
He frowned. On the white wall beside the door, just below the ceiling, there was some dim marking. He walked towards it, looking up. It looked as though someone had started to do an algebraic problem. He studied the figures, reading them aloud, for he had had algebra in high school, and was rather proud of it.
"T equals," he read profoundly, "a. Divided by pi. Times one-sixth e cubed. Which one of you half-witted characters wrote that there?"
"and tells one of the soldiers to clean it."
"What does that stuff mean, sarge?" said the one-eyed janitor. "That t, a, e stuff?"
"Why, nothing," said Sergeant Smert. "Just gibberish. Swing your hose on it, idiot, and wash it off."
II. THE SECOND STORY is almost surely "And It Comes Out Here" by Lester del Rey, first published in Galaxy Science Fiction, February 1951, available at the Internet Archive. Here is a review from timetravelreviews.com:
Author Lester Del Rey tackles an age old circular paradox about the future influencing the past which in turns creates a future which comes back to influence the past.
A stranger shows up on the doorstep of engineer Jerome Bell, one day with a fantastic tale. The stranger however isn't as strange as Jerome might expect since it is himself thirty years older. His older self relates a dizzying first person account of how Jerome will travel a hundred years into the future steal a model of the first household atomic generator from a museum, come back and invent the generator.
This mobius strip of a story is an interesting tale of a seemingly never-ending cycle of the past and present influencing each other. As with similar stories, such as Anson's "By His Own Bootstraps", the story becomes an exercise for both the reader and author in juggling the different perspectives of the characters. Its interesting to note that the 1950's promise of cheap atomic energy is reflected directly in Del Rey's future where small power plants are used to power households.
The biggest discrepancy with your description is that the hero rides the time machine into the future to steal an atomic generator, not a time machine. Here are the points that match your description:
1. The character is visited in his home by his older self, who has come from the future with a time machine.
2. The older guy gives the younger one instructions for stealing a device (the atomic generator) from a museum in the future.
3. The museum displays a series of models in chronological order; the later ones are smaller, the earlier ones bigger and clumsier. The time thief takes the earliest model (because it's the only one that's not bolted down).
4. He finds out that he is the inventor of the atomic generator. (However, he does not find this out in the museum; after he gets back to his own time (1951) he examines the papers that came with the device and finds an old patent application in his own handwriting.)
III. THE ANTHOLOGY containing both of these stories does not seem to exist.
The first story is fairly obscure. As far as the ISFDB or the Contento index knows, it was only picked up by one anthology, The Best from Startling Stories, which was reprinted in the UK as Startling Stories and as Moment Without Time. This is not an anthology of time-travel stories, it is Samuel Mines's compilation of stories from the two magazines he edited, Startling Stories and Thrilling Wonder Stories; it does not contain Lester del Rey's story, or any other story remotely resembling your description of the second story.
The second story, on the other hand, is rather famous; it has been reprinted many times, including translations into several foreign languages. In particular, it is included in two time-travel anthologies: Groff Conklin's 1953 Science Fiction Adventures in Dimension (an anthology of time-travel and parallel-worlds stories) reviewed here, and Robert Silverberg's 1967 Voyagers in Time. I think one of those two may be the time-travel anthology you have in mind; but you must have read "Moment Without Time" in the Mines anthology, unless you read it in the original 1952 magazine.
