"The Other World", a novella by Murray Leinster. You may have read it in one of these compilations. You can "borrow" the story from the Internet Archive (free but registration required) in either this Groff Conklin anthology or this Robert Adams anthology. (Thanks @DavidW for the links.) The story was originally published in Startling Stories, November 1949, which does not seem to have been scanned into the Internet Archive yet.
Tom Maltby, a consulting engineer in New York, manages to duplicate a strange artifact that his friend Dick Blair dug up in an ancient Egyptian tomb:
"I've got a one-inch peephole you can look into the other world with, and a space an inch across that you can push things through into the other world. I know how to make them, now. If I can stay awake, I can make a doorway somebody can go through."
The rulers of the other world have been robbing and enslaving Earth people for thousands of years:
The picture the red-beard painted was only partly like the picture Dick had imagined. There were human masters to be sure. They lived in the palace on the other side of the river. The red-bearded man had been a slave for years, but had never seen a member of the race or family he had been enslaved to serve. He had only rarely seen more than one overseer. Years ago he'd been an electrician in New York, and on his way home one night along Fourth Avenue, he suddenly felt himself falling, and all the world swirled about him and he was in a cage of wooden bars, in a forest like this of monster trees and unfamiliar vegetation.
[. . . .]
Over a very large part of the Earth , things which men had made and treasured, people whom others cared for and depended on, ceased to exist as far as the normal world was concerned. But nobody considered that anything requiring a new explanation had occurred; such things had been happening for five thousand years. Nobody thought to look for any common factors linking them. Nobody at all thought of the possibility of another world, beside this one in hyperspace and identical with it save in flora and fauna and population. Among all the two and a quarter billion humans on Earth, only Sam Todd and Maltby even guessed at such a thing.