One thing I noticed after finishing The Book of the New Sun is that early in the narrative Wolfe has included several places where the reader familiar with current human culture and technology can tell what Severian is describing even if Severian does not know what something is.
The Matachin Tower is a rocketship. Actually Severian does seem to be aware of this on some level as he mentions that a certain room was "the propulsion chamber of the original structure".
Severian sees Rudesind cleaning a picture that is clearly of an astronaut from our times on the un-terraformed moon. (This is as far as I know the only concrete reference in the story to anything that is conclusively a relic of the current era.)
Some people interpret the sun dials in the Atrium of Time as derelict satellite dishes. See, for example, Sam Weber's illustration in the Folio Society edition of the BotNS.
These are all that I know that are cannonical and notably all of these are from the beginning of the story, all from the first 100 or 200 pages of Shadow of the Torturer.
Are there any more that I missed later in the story? There was, for example, a section of Sword of the Lictor in which Severian is descending the mountains north of Thrax and describes a cliff face that cuts through strata of various eras and in particular describes some kind of gigantic geometric pattern that to me feels as though I am supposed to be able to figure out what he is talking about but I draw a blank.