Astral Projection is seen in so many movies and series like Doctor Strange, The Magicians, Chilling Adventures of Sabrina, and Stranger Things. Which is the first movie or TV series to show astral projection?
How broadly are you interpreting "astral projection?" Charles Dickens' A Christmas Carol (written in 1843, filmed as early as 1901 as "Scrooge, or, Marley's Ghost") has the spirits convey Ebenezer Scrooge through the walls of his bedroom to see visions of the past, present, and future.
In particular the Ghost of Christmas Present transports a disembodied Scrooge to the streets of London to witness how the "poor revellers" were celebrating Christmas:
...and they went on, invisible, as they had been before, into the suburbs of the town. It was a remarkable quality of the Ghost (which Scrooge had observed at the baker’s), that notwithstanding his gigantic size, he could accommodate himself to any place with ease; and that he stood beneath a low roof quite as gracefully and like a supernatural creature, as it was possible he could have done in any lofty hall.
The brief 1901 version apparently doesn't include the three spirits, instead relying on Bob Marley's ghost to provide the visions. But a 1908 version has the three spirits. Notably these early versions were early in the history of motion pictures; there's a long continuum stretching from stage performances to what we now think of as "movies."
Perhaps not what one traditionally considers sci-fi or fantasy, but A Christmas Carol does explore one man's encounter with extradimensional visitors. If nothing else it sets up the idea of a contemporary man learning from out-of-body experiences.
A good candidate may be One Glorious Day from 1922.
A disembodied spirit entity with a strange appearance and bulging eyes named "Ek" takes over the body of a meek psychical researcher, Professor Ezra Botts (Rogers), during an out-of-body experiment and proceeds to live it up while the researcher watches from limbo and tries to get back into his physical body and resume his life. Botts waits until the spirit collapses from exhaustion, then takes the opportunity to reinhabit his own body.
This synopsis expands on this being a conscious effort on Professor Botts's part (emphasis mine):
This quirky comedy with its mystical overtones was a departure for the down-to-earth Will Rogers. The story opens up with a title card that announces there are spirits floating around waiting to be born into the material world. One spirit, the mischievous Ek (young John Fox), misses being born by a few seconds and wanders around earth, looking for a body to enter. He happens upon Ezra Botts (Rogers), a retiring professor of spiritualism who has gotten tangled up with some crooked politicians who think they can make him do their will. Botts, who is in love with Molly McIntyre, his landlady's daughter (Lila Lee), has been nominated for mayor. When Botts uses his psychic powers to send his spirit to a meeting, Ek takes over his temporarily spiritless body. Ek in Botts body battles with the politicians, drinks liquor -- much to the horror of his fellow spiritualists -- and beats up Wadley (Alan Hale), who has insulted Molly. On top of everything else, Botts' new persona is a hit with the voters. Finally, Botts' body faints, since it's unaccustomed to all this activity and the real Botts is able to get his body back from Ek. But he returns a changed man and overcomes his shyness enough to propose to Molly. Much credit went to cameraman Karl Brown, who accomplished quite a bit with double exposures in an era where special effects cinematography was still in its rudimentary stages.
I had a vague memory of seeing a silent film clip involving a spirit leaving the body due to astral projection. Since this is a lost film, it's unlikely this is the one I'm thinking of, but it's the first one I've found.
There is an episode of "Ring-a-Ding Girl" from 1964.
This episode from the fifth season of the original Twilight Zone concerns an actress who is seemingly simultaneously onboard an airplane and present in her hometown.
The link from TV Tropes (warning!), gives a precis of the episode as:
Astral Projection: Bunny is seemingly able to astrally project herself to Howardsville while her physical body is on a plane. She does so in order to save as many townspeople as possible when the plane crashes during the Founders Day's picnic.
Not sure from the question if you're looking for an entire series built around a concept of Astral Projection, but as you've mentioned movies as well, I thought that this would fit...