The Country of the Fallen Dragon King and the Perishing Witches...?
From Baka-Updates:
In a world where witches are shunned and have been forced onto the brink of annihilation, a samurai with no memories is transported to the hideout of some of these witches. Having the name: "Dragon King" and a keen strategic mind, its up to him to help defend the witches from those that would seek them harm.
A samurai falls into a lake and lands in another world, right in front of some women who're bathing in the nude. There's some initial confusion, as they can't understand one another's languages, but the leader of these women -- Harrigan Harivey Haindora -- is eventually able to facilitate communication using a magical talisman.
She explains that these women belong to an isolated community of witches living in a forest, although the samurai doesn't initially know what a witch is. When asked who he is, the samurai realises he can't remember, except that he thinks that part of his name is "Naga," which apparently means 'Dragon King' in the witches' world.
Harrigan finds it noteworthy that the samurai isn't afraid of the witches, unlike humans in their own world, who fear and hunt them. Despite their magic and unity, the witches are still in danger due to their relative lack of numbers; there are no men in their community to breed with, they lose their magic when they give birth, and there's no guarantee that their children will be able to use magic.
Sensing that the samurai isn't an enemy, Harrigan allows him to stay with the witches as a part of their community. They subsequently learn that human soldiers are about to attack one of their forts, and prepare for battle. The samurai asks why they don't wear armour into battle, and they explain that metal destroys magical energy, and in general, the less they wear, the less their magic is obstructed.
In a later scene, the witches magically animate a giant 'puppet' made of wood, and send it out of their fort to attack the human soldiers.