If we are defining "extra dimensional space" as a description of an alternate dimension or parallel world then:
As early as 1940 for Marvel comics
Kahalia is the earliest mention of an alternate dimension for Marvel:
In the 1930's, explorer Bruce Dickson attempted to climb Mount Kalpurthia, in the Himalayas. There, he encountered the entrance to the dimension of the lost civilization of Kalahia, which possessed extremely advanced technology. The Kalahian people altered his body to give him the same physical abilities they had: eternal youth, and tremendous physical resilience and flexibility, including the ability to become "thin."
Bruce also learned much of their advanced technology, and fell in love with Olalla, daughter of the Kalahians' leader. He and his new love traveled to the outside world and became crimefighters with Dickson take the name "Thin Man."
From Mystic Comics, August 1940.
Much later in 1962, although this may not exactly be the answer you are looking for - but it technically fits ... Marvel introduces Asgard as existing outside our dimension for Thor.
However if we consult Wikipedia's list of alternate dimensions in Marvel Comics, we see that the "Fifth Dimension" was in "Strange Tales" later that year - which might be a little closer to what we think of in terms of a sci fi alternate dimension.
From Marvel's wikia:
The first recorded time that Earth interacted with the Fifth Dimension began with a scientist from that dimension named Bolo who was inspired to build an army of robots based on the human race to use as slaves in his home dimension. His first and only creation to this end was the intelligent robot known as Marvex.
Note that Marvel's concept of multiverse is pretty complex. There are parallel universes, and there are also dimensions which can span those universes, and there universes outside of those parallel universes which can sometimes crash into either of those. All of these combines are called the Marvel Megaverse(tm).
Outside of Marvel comics, the earliest known reference to a "parallel world" is, according to Wikipedia, The Blazing World in 1666:
As its full title suggests, Blazing World is a fanciful depiction of a satirical, utopian kingdom in another world (with different stars in the sky) that can be reached via the North Pole. It is "the only known work of utopian fiction by a woman in the 17th century, as well as one of the earliest examples of what we now call 'science fiction' — although it is also a romance, an adventure story, and even autobiography."
A young woman enters this other world, becomes the empress of a society composed of various species of talking animals, and organizes an invasion back into her world complete with submarines towed by the "fish men" and the dropping of "fire stones" by the "bird men" to confound the enemies of her homeland, the Kingdom of Esfi.
Considering the similarity between Kalahia and Esfi, it's possible Marvel was influenced by The Blazing World. And, as @user14111 notes in the comments, over in the DCverse - Mister Mxyzptlk beats Thor by more than a decade:
Mister Mxyzptlk (/mɪksˈjɛzpɪtlɪk/ US dict: mĭks·yĕz′·pĭt·lĭk or /mɪksˈjɛzpɪtəlɪk/ US dict: mĭks·yĕz′·pĭt·l·ĭk), sometimes called Mxy, is an impish supervillain who appears in DC Comics' Superman comic books.
He was created by Jerry Siegel and Joe Shuster, and first appeared in Superman #30 (September 1944). He is usually presented as a trickster, in the classical mythological sense, in that he enjoys tormenting Superman. In most of his appearances in DC Comics, he can be stopped only by tricking him into saying or spelling his own name backwards (Kltpzyxm - "kel-tip-zix-um"), which will return him to his home in the 5th dimension and keep him there for a minimum of 90 days.
Also hailing from that pesky fifth dimension.
And yes, you can probably blame hyperspace for all this:
From the 1930s through to the 1950s, many stories in the science fiction magazines, Amazing Stories and Astounding Science Fiction introduced readers to hyperspace as a fourth spatial dimension. John Campbell's "Islands of Space," which first appeared in Amazing Stories in 1931, features an early reference to hyperspace.
The fifth dimension. Bringing you crazy midgets, killer robots and faster modes of travel since around 1930. Give or take.
