In science fiction the idea is fairly common, and there's a whole genre called alternate history which plays around with the multiverse idea--some stories take place entirely in a history different from our own without any suggestion it coexists with other timelines, but many alternate history stories show multiple timelines. You can find plenty of well-known examples in the wikipedia article, here are some of the earliest examples illustrating multiple timelines being experienced by characters in the story:
Another example of alternate history from this period (and arguably the first to explicitly posit cross-time travel from one universe to another as anything more than a visionary experience) is H.G. Wells' Men Like Gods (1923), in which several Englishmen are transferred via an accidental encounter with a cross-time machine into an alternate universe featuring a seemingly pacifistic and utopian Britain. When the Englishmen, led by a satiric figure based on Winston Churchill, try to seize power, the utopians simply point a ray gun at them and send them on to someone else's universe. Wells describes a multiverse of alternative worlds, complete with the paratime travel machines that would later become popular with U.S. pulp writers. However, since his hero experiences only a single alternate world, this story is not very different from conventional alternate history.
In the 1930s, alternate history moved into a new arena. The December 1933 issue of Astounding published Nat Schachner's "Ancestral Voices," which was quickly followed by Murray Leinster's "Sidewise in Time." While earlier alternate histories examined reasonably straightforward divergences, Leinster attempted something completely different. In his "world gone mad," pieces of Earth traded places with their analogs from different timelines. The story follows Professor Minott and his students from a fictitious Robinson College as they wander through analogues of worlds that followed a different history.
"Sideways in Time" appeared in the June 1934 issue of Astounding magazine, which is available on archive.org here (story starts on p. 10), part of their pulp magazine archive.
It's also become common in time travel stories to imagine that when the time traveler changes history, they end up in a new timeline which exists parallel to the one they came from originally, thus avoiding paradoxes (for example, even if a time traveler kills their grandfather as a child in an attempt to cause a grandfather paradox, the universe where their grandfather lived to adulthood continues to exist, and that explains how the time traveler can continue to exist). On p. 299 of his book Time Machines, Paul Nahin suggests a 1935 story as the first one to use this device:
The first such tale in science fiction was probably the 1935 story "The Branches of Time" (Daniels), which also contained the observation that although alternate time tracks may allow for changing the past for the better (something that can't be done, for better or for worse, with just one time time track), in the end any such change may still be futile. As Daniels' time traveler puts it, sadly, "I did have an idea to ... go back to make past ages more liveable. Terrible things have happened in history, you know. But it isn't any use. Think, for instance, of the martyrs and the things they suffered. I could go back and save them those wrongs. And yet all the time ... they would still have known their unhappiness and their agony, because in this world-line those things have happened. At the end, it's all unchangeable; it merely unrolls before us."
"The Branches of Time" appeared in the August 1935 issue of Wonder Stories, which is available on archive.org here (the story starts on page 295).
As for the term "multiverse", the earliest use I found searching on google books was Spokesmen by Thomas King Whipple--the copyright page says the edition on google books is from 1963, but that it was originally published in 1922. And on pages 23-24 of the book you can read the quote "It was not, in fact, a universe at all, but a multiverse, for it was not under a single control, but was a congeries of warring forces." However, in this case he doesn't really seem to be talking about a set of parallel universes, but just a single reality that lacks a single organizing force or principle (whether God or some unified law). Likewise, I found a page here that says the philosopher William James was the first to use the term in an 1895 essay, Is Life Worth Living?, where he talked of a "moral multiverse" (the essay is on archive.org here, the term appears on p. 26), but it sounds like he was talking about the fact that our moral intuitions don't come from any unified moral system or authority.
In terms of the modern meaning of the term to refer to coexisting universes, this page says "multiverse" was first used to describe a set of parallel universes in a 1961 talk about the many-worlds interpretation of quantum mechanics, which physicist Hugh Everett III proposed in 1957. The page also says Michael Moorcock first used the term in his story "The Blood Red Game", which was published in the magazine Science Fiction Adventures in May 1963. Moorcock's 1963 story is also the first reference given for the term in Brave New Words: The Oxford Dictionary of Science Fiction on page 122.