The Forest of Time by Michael Flynn. I read it in The Year's Best Science Fiction: Fifth Annual Collection:
The story is told from the perspective of Oberleutnant Rudolf Knecht, Chief Scout of the Army of the Kittatinny. Knecht captures the traveller believing him to be a spy, and the story is told as extracts from the man's journal. The first entry sets the scene:
I am embarking on a great adventure. Does that sound grandiose? Very well, let it. Grandiose ideas deserve grandiose expression. Tomorrow, I make my first long range Jump. Sharon claims that it is too soon for such a field test, but she is too cautious. I’ve engineered the equipment. I know what it can do. Triple redundancy on critical circuits. Molecular foam memory. I am a certified reliability engineer, after all. The short Jumps were all successful. So what could go wrong?
But as you say the jump doesn't work and the traveller's journal records:
A slight miscalculation. I should be back in the lab with Rosa, but I’m in somebody’s apartment, instead. It’s still Philly out the window—though a shabbier, more run-down Philly than I remember.
I must leave immediately! That black thing on the lamppost kept nagging at the back of my mind. So I got out my binoculars and studied it. It was a nun in a black habit, hanging in a noose. Hanging a long time, too, by the looks of it.
Since this is the point you want to continue from I won't say where the story goes next.