It's a bit of a longshot since not everything fits (and honestly, I've only been able to get scant information online) but possibly "The Doctor" by Theodore L. Thomas? It was published in Orbit 2, an anthology of SF in 1967 (edited by Damon Knight and one of the more notable of the timeframe).
The description I can find for it reads:
A doctor named Gant volunteers to be the first time traveler and ends up stranded in a time of cave people.
"There had been a time long ago when he had thought that these people would be grateful to him for his work, that he would become known by some such name as The Healer."
Ten years ago Gant was a doctor who volunteered to be the first for time travel. Now he is stuck five hundred thousand years in the past living among cave dwellers.
Obviously, he's a volunteer rather than just tossed there as you recall, but that seems like the kind of detail that could easily be confused between stories in decades of time. I also don't have any direct evidence of him breaking his glasses. However, he is stranded among stone age people, language difficulties seem natural, and based on the title (and paragraph fragment) it's not a huge leap to assume he might have attempted to use his modern-day medical knowledge to heal one of the natives, and perhaps did not get rewarded as he might have hoped.