Harry Potter and the Deathly Hallows, Chapter 21: The Tale of the Three Brothers
“‘The first brother traveled on for a week or more, and reaching a distant
village, sought out a fellow wizard with whom he had a quarrel. Naturally, with the Elder Wand as his weapon, he could not fail to win the duel that followed.
Leaving his enemy dead upon the floor, the oldest brother proceeded to an inn,
where he boasted loudly of the powerful wand he had snatched from Death
himself, and of how it made him invincible.
“‘That very night, another wizard crept upon the older brother as he lay wine-
sodden, upon his bed. The thief took the wand and, for good measure, slit the
oldest brother’s throat.
“‘And so Death took the first brother for his own.'"
That was Hermione reading The Tale of the Three Brothers from the popular wizard's fairy tale collection The Tales of Beedle the Bard. At this point in the book, Hermione and Ron are skeptical as to the existence of the Deathly Hallows. They believe it's just a children's story.
Harry Potter and the Deathly Hallows, Chapter 35: King's Cross
“So it’s true?” asked Harry. “All of it? The Peverell brothers—”
“—were the three brothers of the tale,” said Dumbledore, nodding. “Oh
yes, I think so. Whether they met Death on a lonely road ...I think it more
likely that the Peverell brothers were simply gifted, dangerous wizards who
succeeded in creating those powerful objects. The story of them being Death’s
own Hallows seems to me the sort of legend that might have sprung up around
such creations."
The tale says 'Death' gave them the Hallows. But that seems unlikely as Dumbledore says. Similarly, maybe the first brother was killed by an unkown assassin in an inn. But that might not be true as well. All we can say for certain is that the wand wasn't taken from him in a duel(if it was the story would have some exaggeration of how he lost it in a duel). We also know it cannot be passed like an heirloom since the wand's power dies if the owner dies a natural death. So we can assume he was murdered when not in a position to use the wand.
As to the identity of who took the wand, Rowling does not mention it in the books.