Craster's daughters believe their sons are transmogrified into White Walkers. (I won't refer to the women as "wives", because calling them wives is like calling slaves "business partners".)
We read in chapter Jon III in A Clash of Kings that his daughters know Craster gives their sons to something.
If it’s a girl, that’s not so bad, she’ll grown in a few years and he’d marry her. But Nella says its to be a boy, and she’s had six and knows these things. He gives the boys to the gods. Come the white cold, he does and of late it comes more often.
They know the boys are given to the White Walkers. Yet, they also seem to know (or at least believe) the White Walkers are their brothers and sons.
Gilly was crying. “Me and the babe. Please. I’ll be your wife, like I was Craster’s. Please, ser crow. He’s a boy, just like Nella said he’d be. If you don’t take him, they will.”
“They?” said Sam, and the raven cocked its black head and echoed, “They They They”
“The boy’s brothers,” said the old woman on the left. “Craster’s sons. The white cold’s rising out there, crow. I can feel it in my bones. These poor old bones don’t lie. They’ll be here soon, the sons.”
How would the women know the baby boys are transmogrified into White Walkers?
The White Walkers wouldn't tell them. Nor would they show up at family reunion with gifts for their little nieces saying "from Uncle Frank". Nor would one of Craster's daughters follow the White Walkers across a few hundred miles of ice to the Lands of Always Winter to find out.
Wouldn't the women just assume their boys are dead?
I prefer an answer from the book over one from the TV series. Please provide exact quotes from the books if you can. Thanks!
Edit to add: Is there anything from free folk culture to show wildlings knew that people taken by White Walkers were converted into White Walkers?