The concept of multiverse, sharing different universes and characters travelling in other alternate dimensions, when was the first time this happened in comic books ?
Possibly in Wonder Woman #59 from 1953:
This issue introduces the concept of a parallel universe alongside "ours". That is, Wonder Woman's, the universe now known as Earth-Two, Pre-Crisis on Infinite Earths. This parallel universe is Earth-59, according to The Essential Wonder Woman Encyclopedia. where alternate versions of everyone live, including Wonder Woman. Says she:
Earth must have a twin world, existing simultaneously alongside it! Everyone on it, is a double of everyone on Earth!
The Flash #123 from 1961 takes this concept further, to something more recognizable as today's DC multiverse, when Flash meets a different Flash from another universe where he himself is a comic book character who has inspired others. Justice League of America #21 from 1963, "Crisis on Earth One!", formalizes the DC multiverse.