Rogue Moon, a short novel by AlgIs Budrys. The "strange other-dimensional realm" was an alien artifact (a labyrinth with death traps) on the moon; the explorer on earth was telepathically linked to his matter-transmitter replica on the moon who would enter the labyrinth and die, over and over again.
A shorter version (novella length) was published in The Magazine of Fantasy and Science Fiction, December 1960 (available at the Internet Archive) and reprinted in The Science Fiction Hall of Fame, Volume Two B. If that's what you read, you might recognize one of these covers.
Wikipedia plot summary:
Dr. Edward Hawks runs a top-secret project for the U.S. Navy, using the facilities of Continental Electronics to investigate a large, deadly alien artifact found on the Moon. Volunteers enter and explore it, but are inevitably killed for violating the unknown alien rules in force within the structure. Hawks "must continue to send duplicates into the artifact, however, because each one moves a little closer to finding a way through the alien labyrinth" and, thus, closer to understanding what it is.
Vincent "Connie" Connington, Continental's head of personnel, tells Hawks that he has found the perfect candidate for the next mission. Connington is amoral and manipulative, openly testing Hawks and anyone else he meets for weaknesses. He takes Hawks to see Al Barker, an adventurer and thrill-seeker. Hawks also meets Claire Pack, a sociopath of a different kind. Where Connington covets power, and Barker seems to love death, Claire enjoys using sex, or the prospect of sex, to manipulate men. Connington wants her, but she stays with Barker because he has no weaknesses in her eyes. Hawks has to appeal to Barker's dark side to persuade him to join the project. Claire tries to get under Hawks' skin while simultaneously playing Connington off against Barker.
Hawks has created a matter transmitter, one which scans a person or object to make a copy at the receivers on the Moon. The earthbound copy is placed in a state of sensory deprivation which allows him to share the experiences of the doppelgänger. However, none of the participants have been able to stay sane after experiencing death second hand.
Barker is the first to retain his sanity, but even he is deeply affected the first time, exclaiming, "...it didn't care! I was nothing to it!" He returns again and again to the challenge, advancing a little further each time. Meanwhile, his relationship to Claire deteriorates, even as Connington continues his disastrous attempts to win her, at one point receiving a severe beating from Barker. Eventually, Connington announces he is quitting, and Claire leaves with him.
Meanwhile, Hawks starts a relationship with a young artist, Elizabeth Cummings, and expresses his torment over the project to her. Finally, Barker announces that he is almost finished finding a way through the artifact. Hawks takes Elizabeth to a romantic location and declares his love for her, then returns to the project. He transmits himself as well as Barker to the Moon, where his duplicate joins Barker's on the final run.
Together, the two weave their way through a series of bizarre landscapes containing death traps. Emerging from the other side, Hawks tells Barker that they cannot return to Earth. The equipment on the Moon is too crude to transmit a man back safely, and even if it were possible, there are already people living their lives. All the men working on the Moon are duplicates, mostly Navy men, all volunteers. Hawks elects to remain outside the base until his air runs out. Barker returns to try to be transmitted back anyway.
Back on Earth, Hawks removes his isolation suit and finds a note in his hand, which he knew would be there. It reads simply, "Remember me to her."
and a description of the main character:
Al Barker, a "daredevil of a man who has spent his life defying death;" Connington calls him "a man famous for split-second decisions. Always the same ones." He likes to taunt Hawks by saying, for example, "morituri te salutamus," but comes to respect Hawks (perhaps because Hawks replies, "I've also read a book"). Barker works successfully for the project: "each time Barker moves farther through the artifact" by inches, he maps "a route through the enigmatic structure."