Stephen Vincent Benét's story By The Waters Of Babylon was published in 1937, and features a member of a primitive superstitious tribe living in a world where
The north and the west and the south are good hunting ground, but it is forbidden to go east. It is forbidden to go to any of the Dead Places except to search for metal and then he who touches the metal must be a priest or the son of a priest. Afterwards, both the man and the metal must be purified. These are the rules and the laws; they are well made. It is forbidden to cross the great river and look upon the place that was the Place of the Gods—this is most strictly forbidden. We do not even say its name though we know its name. It is there that spirits live, and demons—it is there that there are the ashes of the Great Burning. These things are forbidden—they have been forbidden since the beginning of time.
The narrator, the son of a priest, ends up entering the forbidden Place of the Gods, and encounters things which an alert reader will recognize as being in New York City, long after some apocalypse has destroyed modern civilization and any vague knowledge of the past has been relegated to myth and legend.
I'm not sure if this counts as a Time Machine like "they already knew it was Earth" type story; IIRC they've lost so much knowledge that it's not clear the people in the story even understand the concept of planet.