It might be the 1937 story By the Waters of Babylon by Stephen Vincent Benét. From this answer on another question:
On a similar note, "By the Waters of Babylon" was published before it became a standard trope of science fiction to have a young protagonist roaming around in a primitive world which used to be the home of a high-tech civilization.
Summary from Wikipedia:
Set in a future following the destruction of industrial civilization, the story is narrated by a young man who is the son of a priest. The priests of John's people (the Hill People) are inquisitive people associated with the divine. They are the only ones who can handle metal collected from the homes (called the "Dead Places") of long-dead people whom they believe to be gods. The plot follows John’s self-assigned mission to get to the Place of the Gods. His father allows him to go on a spiritual journey, not realizing John is going to this forbidden place.
John journeys through the forest for eight days and crosses the river Ou-dis-sun. Once John gets to the Place of the Gods, he feels the energy and magic there. He sees a statue of a "god"—in point of fact, a human—that says "ASHING" on its base. He also sees a building marked "UBTREAS". After being chased by dogs and climbing the stairs of a large building, John sees a dead god. Upon viewing the visage, he has an epiphany that the gods were humans whose power overwhelmed their good judgment. After John returns to his tribe, he tells his father of "the place New York." His father warns him against recounting his experiences to others in the tribe, for sometimes too much truth is a bad thing, that it must be told little by little. The story ends with John stating his conviction that, once he becomes the head priest, "We must build again."
Having read the story, I can confirm that it's not known to the reader at the beginning that it takes place in the future. There are progressively larger hints through the story (the "river Ou-dis-sun" used to be known as the Hudson, the "ASHING" mentioned is a damaged bust of George Washington, while the "UBTREAS" is the Subtreasury building at Federal Hall), but it's not until the final paragraph that it's made explicit:
Nevertheless, we make a beginning. It is not for the metal alone we go to the Dead Places now—there are the books and the writings. They are hard to learn. And the magic tools are broken—but we can look at them and wonder. At least, we make a beginning. And, when I am chief priest we shall go beyond the great river. We shall go to the Place of the Gods—the place newyork—not one man but a company. We shall look for the images of the gods and find the god ASHING and the others—the gods Lincoln and Biltmore and Moses. But they were men who built the city, not gods or demons. They were men. I remember the dead man's face. They were men who were here before us. We must build again.
The full story is available at the Broome-Tioga BOCES website.