Here are some earlier examples which may or may not qualify depending on how you interpret the question and how you interpret the pseudoscience in those old stories. These are stories about other worlds or "dimensions" which are "parallel" or "coexistent" with our own, separated from our world by "vibration". I may be wrong but it seems to me that, as you wander about in one parallel world, you are necessarily passing through solid objects in the other world. Unfortunately, and this may be a disqualifying point, the worlds are usually said to have different vibratory rates, quite a different thing (as I understand it) from being out of phase. The quotations below are from Bleiler's reviews in Science-Fiction: The Early Years or Science-Fiction: The Gernsback Years.
1921: The Blind Spot, a novel by Austin Hall and Homer Eon Flint, was originally published as a serial in Argosy All-Story Weekly starting 14 May 1921; it has a Wikipedia page and is available at Project Gutenberg. There is a sequel, The Spot of Life by Austin Hall.
The exact nature of the Blind Spot is revealed only diffidently, but it seems to be a gateway between parallel worlds that coexist at different vibratory rates, hence are mutually imperceptible. A fuller explanation, with larger implications, is given in "The Spot of Life."
1927: "Below the Infra Red", a short story by George Paul Bauer, appeared in Amazing Stories, December 1927, available at the Internet Archive.
Place: A parallel world separated from ours by vibratory rate. * The narrator becomes acquainted with Professor Carl Winter, who reveals his discoveries. Winter theorizes that there are worlds of experience cut off from us because we cannot see infrared and lower light, or hear ultra sound. Winter has now constructed a machine that will enable himself and the narrator to witness this new world. * Seated in the apparatus, the two men see a new world, perfect-looking humans in a wonderful landscape. And then the men discover that they are in the other-world, which the professor describes as defined by a different rate of vibration.
1928: "The Blue Dimension", a short story by Francis Flagg (pseudonym of George Henry Weiss), appeared in Amazing Stories, June 1928.
Place: California and an other-world. * The young man who assisted Crewe in his experiments is justifying himself, proclaiming that he did not really murder Crewe. * Crewe, a specialist in optics, explains to the narrator that we perceive only a limited range of vibrations, and that there are vibratory groups which amount to other-worlds. Crewe then produces a pair of spectacles for seeing into the other-world. What the men see is a blue world, with strange vegetation and odd animal life. There are gigantic, semi-humanoid beings with extensible tentacle "arms," a more civilized and smaller brownish form of similar humanoid life; and a strange city, into the buildings of which the doctor and the narrator cannot look, for unexplained reasons. Crewe has also invented a roller press-like arrangement, which, as he demonstrates with mice, can transfer things to the other-world. Against the wishes of the narrator, Crewe now hops into the machine, emerging into the other-world as a gigantic being. For a while the narrator loses sight of him, but when Crewe returns and signals that he wishes to be retrieved, there is a difficulty. He neglected to transfer a vibrator to the other-world, so that he is now stranded. He signals for food, which the narrator sends, but when the narrator drops the dimensional spectacles, they no longer function, and there is no telling what has happened to Crewe.
1931: "Through the Vibrations", a novelette by P. Schuyler Miller, was originally published in Amazing Stories, May 1931.
Dr. Alexander Gregory, scientific genius, theorizes that the extremely high wavelength of matter can be altered, permitting entry to other frames of existence. He has constructed an apparatus that functions on the basis of his theory, and he and his friend Jack, the narrator, decide to explore a different mode of existence. Donning space suits with radiation insulation, air supply, strap-on helicopters, and attached tool kits, they translate themselves into an other-world. This parallel, interpenetrating world is much like Earth, but with slightly different gravity and atmosphere, different colorations, and a large, cool, green sun. * As the men explore, they come upon a gigantic building mass with which is associated an incredibly powerful ventilation system leading to an inner world. The two men descend with their tiny helicopters and find the ruins of an enormous, beautiful city, within which is much human statuary. Gregory, on more exploration, declares that the dead civilization was that of Atlantis, which must have been translated from our world by a resonance apparatus similar to his own. As corroboration, he tells that back on Earth he had found a continuation of Plato's Timaeus, which described a situation that could be interpreted in this fashion. As for the population of the underground city, it lies in dust, consumed by a horrible fungus.