Just watched the 4-th episode of season 4. In the end we see the scene where
the baby is turned into one of the white walkers by a magic ritual.
That means that
The white walkers actually aren't immortal, they grow older over the time, and they perhaps even can die from old age.
But if that's how the process of reproduction is organized for the White Walkers, that brings the question:
Who was the first white walker ever and how did he come to such a state?
It is not yet mentioned who was the first white walker or how did he became like that. We only know that they first appeared during the Long Night (1)
A Game of Thrones, Chapter 1, Bran:
According to legend, the Others first appeared approximately 8,000 years before the War of Conquest, during a winter that lasted a generation and a period of darkness known as the Long Night. Eventually they were defeated, supposedly by the Night's Watch in the Battle for the Dawn, and the Wall may have been raised as a defense against them.
A Game of Thrones, Chapter 24, Bran:
According to legends, in the midst of this darkness a race of apparent demons, called the Others, emerged from the uttermost north, the polar regions of the Lands of Always Winter. They wielded razor-thin swords of ice and raised wights to fight the living. The children of the forest and their allies, the First Men, fought valiantly against them, but were driven southwards by their advance.
After the walkers lost the battle, Bran the Builder raised the Wall to keep the walkers away.
Also, keep in mind that what happened in the end of S04E04, has not happened to books, so we can't know for sure what that means.
(1): The Long Night is the name given to a period in history where a terrible darkness fell across Westeros and the east.
In Season 6 Episode 5:
Bran enters a vision in the tree of knowledge. In his vision he sees the forbidden forest tree and a man bound to it. As he gets closer he sees a group of forest nymphs (since revealed to be the Children of the Forest) emerging from the surrounding stones. One forest nymph gets close to the man and plunges a dagger shaped stone completely into the mans chest. He cries out in agony as his eyes turn blue.
Bran awakens from the vision back in the cave of reckoning. He sees...
...a nymph staring back at him. He says, "It was you. You made the White Walkers."
She replies
"We were at war. We were being slaughtered. Our sacred trees were being cut down. We needed to defend ourselves..."
Bran says
"From who?"
She says
"From you... from men."
Since the question is tagged with A song of Ice and Fire, I will get in some perspective from the books.
In a Feast for Crows Maester Aemon Says:
I should not have left the Wall. Lord Snow could not have known, but I should have seen it. Fire consumes, but cold preserves. The Wall . . . but it is too late to go running back. The Stranger waits outside my door and will not be denied.
He was referring to his physical state which started getting worse after he left the wall. If Cold can preserve an old man well into his hundreds from death what do you suppose happens to a being which is completely made of Ice and cold?
Quoting GRRM:
The Others are not dead. They are strange, beautiful… think, oh… the Sidhe made of ice, something like that… a different sort of life… inhuman, elegant, dangerous.
But while the cold did not stop aging process for Aemon, it most certainly protected him from death from effects of Old age.
It must be noted that show does not differentiate between Others and Wights. Wights are resurrected dead men while Others are the magical creatures who perform that resurrection.
As Melisandre said:
Necromancy animates these wights, yet they are still only dead flesh. Steel and fire will serve for them. The ones you call the Others are something more.
We do not know who was the first of the Others but we do know where did they come from. About 8,000 years ago, Long Night fell on Westeros (And most probably rest of the World because similar legends are found in traditions of Yi-ti and As'shai). The demons now known as the Others came from the farthest reaches of the North known as Land of always Winter. They were however defeated and were never heard of again until the events in ASOIAF.
But even in the Books, we know The Others do something with infants. We do not know if they turn them into one of their own or their thralls aka Wights but we do know that they come with the Others:
From ACOK, Craster's wife telling Samwell Tarly:
The boy’s brothers…Craster’s sons. The white cold's rising out there, crow. I can feel it in my bones. These poor old bones don’t lie. They’ll be here soon, the sons.
With Books part done, according to Season 6 of the show:
Children of the Forest created first Wight using their magic to defend against onslaught of the First Men. (But show forgets that in war against the Wights, First Men and COTF were allies). He was most likely a Captive from a battle between First Men and COTF.
