Sounds like Tsuyokute New Saga.
At long last, the demon invasion is over. The hero Kyle Lenard has struck down the Demon Lord with the aid of his companions and brought peace to the land. Although the fighting is finished, the land is in ruin. Most of the heroes have fallen one by one, leaving a mortally wounded Kyle as the only survivor.
With his strength fading, Kyle accidentally activates a mysterious artifact within the slain Lord's lair and is sent back in time to a few years before the conflict. Suddenly gifted with a second chance, Kyle reunites with his friends Seran the swordsman and martial artist Riize. The group must once again rally together a fighting force and use Kyle's knowledge of the future to prepare the world for a war no one else knows is coming.
At the beginning of the story, the MC kills the demon lord, but at great cost. His magical sword is destroyed, the rest of his party is dead, and he himself is fatally wounded. Numerous human kingdoms had already fallen as well.
As the MC is dying within the demon lord's chamber though, he notices an altar with a glowing jewel floating above it. He musters the strength to get himself over to the jewel, and when he grasps it, there's a flash of light, and suddenly he finds himself in bed, except that he's younger and skinnier than he was a few moments earlier.
It turns out that the jewel was once the heart of a dragon, and that it's sent him back in time four years, with full memory of everything that happened. He resolves to use this opportunity to become stronger than before, and defeat the demon lord once again, but without so many lives being lost along the way.
Pretty early in the story, he also regains his magical sword, which contains the personality and memories of the late Magic King, Sildonia Zeeles.