Tsukimichi: Moonlit Fantasy perhaps?

Makoto Misumi (深澄真, Misumi Makoto)
Voiced by: Natsuki Hanae
Makoto Misumi is the main protagonist of this series. He is one of three Earth dwellers who was summoned to the Goddess' World. Originally meant to be the only person to be sent to the other world as per the contract between his parents and the Goddess, due to her own preferences in beauty standards, she abandons and throws Makoto to the edge of her world into the Wasteland, while secretly kidnapping two other people from Earth and sending them to her world as Heroes. Tsukuyomi, the Moon God of Earth who worked as the middleman for the completion of the contract, helped Makoto and provided him with his blessing and told him to be free and live as he wishes to. In the Goddess' World, he is also known by the alias "Raidou Kuzunoha" (in WN/LN) or "Makoto Kuzunoha" (in Manga/Anime), a powerful merchant and skilled teacher.
....
Shiki (識, Shiki)
Voiced by: Kenjiro Tsuda
Shiki (識しき), previously known as Larva, was a Lich who seeked endless knowledge, in particular the path to becoming a Grant, hyumans who have travelled to other worlds, whether successfully or not. After some convincing from Makoto due to his desire to learn more about magic, he makes a pact with Makoto and is presently in servitude of Makoto Misumi.
The talking black sword is likely Einkaref
A talking sword that once belonged to Muramatsu Iori. It is a long and thin cylindrical shaped sword that calls itself “The sword of a Dragon Slayer”.
Not quite matching is that he is thrown into a desert, where he saves an orc from an attack, and later settles down as a merchant due to how his overwhelming power tends to lead to people treating him like a walking atomic bomb. While acting as a merchant, he disguises his power by wearing several types of mana-suppressing items that would normally kill the wearer from excessive mana drain.
Came up with a search for manga lich familiar