To the best of my recollection, they went to a house of some sort for the evening to enhance the role-play aspects of some D&D clone/adaptation, and there was even a stuffed octopus or some such in the basement to serve as a monster. For some reason, an alternate reality began to seep into theirs. Apparently, in creating their game & characters, they had somehow tapped into or been influenced by beings in the other world, to create fictional equivalents of them for their game. There was some horrible crime of some sort that one or more of them had committed in their reality that had something to do with why this was happening, but I don't recall the details. The personas of their alternates, however, began to take over the players.
The stuffed critter in the basement even animates. It can speak, and explains some of what is happening to them.
I believe the cover showed tentacles from the octopus creeping around a door frame.
I read this in the early 80s, and would guess it was no more than a few years old, based upon what I recall.