I think this was Time Shards (link to story online) by Gregory Benford. (The note says it was originally published in UNIVERSE 9, edited by Terry Carr.)
"Researchers figure out how to play the grooves on a wheel-turned pot"
Brooks stooped forward. When he peered closer he could see the smooth finish was an illusion. A thin thread ran around the pot, so fine the eye could scarcely make it out. The lines wound in a tight helix. In the center of each delicate line was a fine hint of blue. The jug had been incised with a precise point.
[...]
Hart pressed a switch and the turntable began to spin. He watched it for a moment, squinting with concentration. Then he reached down to the side of the turntable housing and swung up the stylus manifold. It came up smoothly and Hart locked it in just above the spinning red surface of the pot.
"My recollection is that the potter lived in medieval England."
“Who made it?”
“Near as I can determine, somebody in a co-operative of villages, barely Christian. Still used lots of pagan decorations. Got them scrambled up with the cross motif a lot.”
“You’ve gotten . . . words?”
“Oh, sure. In early English, even.”