In the television series, we have yet to see what exactly the white walkers do with Craster's sons. Craster, for his part, tells Mormont that he "sacrifices" his children to the "real gods", implying that Craster (at least) assumes they are dead. However, we never see what became of the one son that Jon Snow saw being carried off.
In the novels, we don't actually "see" this event, but we do hear about it from Craster's wives. They claim to Jon and Sam that the sacrificed babies become Others (the book's term for the show's "white walkers"), and that this is how that species "procreates". However, we have not actually seen any Other that was positively identified as one of Craster's sons, so that is likely supposition on the part of the wives.
My suspicion is that the novel's claim is correct, and that we will eventually see this come up again in future seasons.
Edit: on television, in Series 4, in the episode "Oathkeeper" we actually see Craster's last son becoming a White Walker.