The person being interviewed in that Radiolab segment is Jimena Canales, "an expert in 19th and 20th century history of the physical sciences". Canales published an article in the New Yorker, Albert Einstein’s Sci-Fi Stories on the same subject, containing this:
By the time that Felix Eberty, a German jurist and amateur astronomer, anonymously published “The Stars and World History,” in 1846, it was well known that light had a finite speed. [...] Eberty was particularly fascinated by what this delay meant for a faraway observer of our planet. Perched on a distant star, he wrote, such a person might “see the earth at this moment as it existed at the time of Abraham.” Furthermore, by hopscotching across the cosmos, “he will be able to represent to himself, as rapidly as he pleases, that moment in the world’s history which he wishes to observe at leisure.”
Later authors continued the thought experiment. [...] The popular-science writer Aaron Bernstein joined in. The great cosmic postal service, he wrote, knew neither past nor present: “Alexander the Great is still conquering the world.”
Among the impressionable young Germans who read Eberty and Bernstein was one named Albert Einstein. (He recalled devouring Bernstein’s work, in particular, “with breathless attention,” and it may have inspired one of the conjectures that led to his special theory of relativity.)
The radiolab transcript seems to credit Bernstein as being the author of the story being discussed:
PAT: According to Jimena, a story he read sort of led him to it. She
says Einstein loved science fiction as a kid.
JIMENA CANALES: And he said he was particularly taken by one author.
The name is Aaron Bernstein, who wrote quite a few volumes. And
Einstein says that he read them with quote "breathless attention."
PAT: And Jimena says the story that got Einstein thinking about
chasing light beams was about ...
JIMENA CANALES: A faster-than-light traveler, and what happens if we
travel faster than the speed of light....
The Wikipedia page on Aaron Bernstein has this passage:
Already in the edition of 1855, Bernstein published ideas on space, time and the speed of light which had appeared in the anonymous treatise The Stars and the Earth (German: Die Gestirne und die Weltgeschichte) written by 'an unknown clear-sighted thinker.' It was not until 1874 when a new German edition appeared that the name of the author - Felix Eberty - was made public. When this edition was re-published in 1923, Albert Einstein wrote a preface.
A story in volume 16 of Bernstein's Naturwissenschaftliche Volksbücher about riding along with the electricity travelling through a telegraph wire is often credited with inspiring the 16-year-old Albert Einstein to think about travelling along with a beam of light and seeing it stationary. Such thought experiments eventually led to his famous theory of special relativity.
A footnote indicates the specific story is "Eine Phantasie-Reise im Weltall" [A fantasy trip in space], available here. The story is in german.
The wikipedia page links to another publication from Galina Weinstein of Cornell, Einstein Chases a Light Beam (PDF). The paper was later published as part of Weinstein's book Einstein's Pathway to the Special Theory of Relativity. Quoting from the paper:
Einstein Chases a Light Beam
Between 1895 and 1896 in Aarau, Einstein was sixteen and the story
complicates. If we thought the starting point could be Milan, we
better return three or four years back to Munich. Einstein is twelve
or thirteen years old. He meets once a week Max Talmud. The latter
exposes him to Aaron Bernstein's Naturwissenschaftliche Volksbücher:
Wohlfeile Gesammt-Ausgabe. Einstein is thrilled and reads Bernstein's
books enthusiastically.
In volume sixteen Bernstein describes the wonders of the skies, and
then dedicates a chapter to each planet; finally he invites his
readers to join him for a fantasy journey into space. Under the title,
"Eine Phantasie-Keise im Weltall", "1. Die Abreife", Bernstein
described his imaginary journey.
Suppose you want to perform a voyage to space. You need a
passing-card, and some provisions, food, a suitcase. Although our
voyage is going to be very fast, we are going deep into space. In our
suitcase we will take our thoughts. "We travel by water? On the back
of the horse? By train? None of that! We travel with the help of an
electrical telegraphical apparatus!"
A few years later, Einstein at school in Aarau imagined a journey on a
light beam as well (not exactly on a telegraphic signal); the thought
experiment of him chasing a light beam. Friedrich Herneck thought that
Bernstein might have inspired Einstein when he propounded his Aarau
thought experiment. Herneck first described Einstein's thought
experiment from Aarau of him chasing a light beam. Then Herneck
referred to his earlier suggestion according to which Einstein might
have thought of the speed of light already in Munich when he was
twelve years old. He could find this in Bernstein's books, since
Bernstein raised the question right in the introduction and continued
to discuss it afterwards.
A longer description of Bernstein's "Fantasy Journey into Space" appears in the book Einstein: The Formative Years, 1879-1909 by Don Howard. The link goes to a Google Books page for the book where the Bernstein story is described beginning on page 27.
Both Eberty's and Bernstein's writings were in german. No english translation seems to be readily available.
To summarize:
- Eberty is directly credited with writing about someone traveling about the universe, looking back on earth, and seeing the light from the past.
- Bernstein is credited as being influenced by Eberty.
- Bernstein is credited with writing about traveling through space at the speed of light (or technically as some kind of transmission over a wire--note that radio hadn't been invented yet).
- Einstein is credited with being influenced by both Eberty and Bernstein. Notably, Bernstein's writings are credited with influencing his thinking about hitching a ride on a light beam.
Eberty's story "The Stars and World History" seems like a close match for what is described in the Radiolab segment. But historical scholars seem to credit Bernstein as the major influence on Einstein. But Bernstein was influenced by Eberty, and it's possible that Eberty's ideas appeared in Bernstein's writings in some form.