SPOILERS ALERT
This answer will be composed entirely of spoilers. You have been warned. Read on your own hazard.
Editors, please do not go through the trouble of neutralizing words which may be spoilerish. The warning above should be enough.
Because they were supposedly:
Charged by Prince Rhaegar to protect his paramour Lyanna Stark and
their child, Jon "Probably a Targaryen" Snow. Rhaegar was obsessed with an old prophecy of a prince that was promised and he may have believed that Jon was that prince, who shall be born with the Song of Ice and fire. Rhaegar thought of the child's security as imperative not just for the Targaryens but for whole humanity.
Change of Regime
This is what happened in the Tower:
“I looked for you on the Trident,” Ned said to them.
“We were not there,” Ser Gerold answered. “Woe to the Usurper if we
had been,” said Ser Oswell.
“When King's Landing fell, Ser Jaime slew your king with a golden
sword, and I wondered where you were.”
“Far away,” Ser Gerold said, “or Aerys would yet sit the Iron
Throne, and our false brother would burn in seven hells.”
“I came down on Storm's End to lift the siege,” Ned told them, and the
Lords Tyrell and Redwyne dipped their banners, and all their knights
bent the knee to pledge us fealty. I was certain you would be among
them.”
“Our knees do not bend easily,” said Ser Arthur Dayne.
“Ser Willem Darry is fled to Dragonstone, with your queen and Prince
Viserys. I thought you might have sailed with him.”
“Ser Willem is a good man and true,” said Ser Oswell.
“But not of the Kingsguard,” Ser Gerold pointed out. “The Kingsguard
does not flee.”
“Then or now,” said Ser Arthur. He donned his helm.
“We swore a vow,” explained old Ser Gerold.
Ned’s wraiths moved up beside him, with shadow swords in hand. They
were seven against three.
“And now it begins,” said Ser Arthur Dayne, the Sword of the Morning.
He unsheathed Dawn and held it with both hands. The blade was pale as
milkglass, alive with light.
“No,” Ned said with sadness in his voice. “Now it ends.”
The Kingsguards never changed their loyalty, at least not the three at the tower. For them the dragons remained the King and Robert a usurper and they were honor bound to fulfill the last duty Prince of Dragonstone charged them with.
You can note in this dialogue:
- Kingsguard refuse to refer to Robert as King and called him a usurper.
- They profess loyalty to Aerys even then.
- They refuse to bend knees to Robert despite almost all the realm doing it.
- They refuse to flee in exile with their queen and princelings
- It's a part of their honor-code! They do not flee,no matter what.
- They swore a vow to defend the royal blood and do whatever the royal blood charges them with. They may have not liked Aerys after all, as they explained to Ned that they swore a vow to be loyal however and meant to keep it.
Why did Selmy do it?
Selmy did it because of his own feelings. He knew what Aerys was and Robert struck him as a better man. He however later regretted these feelings afterwards in life as he saw Robert become what he became.
Robert valued character and chivalry. From Eddard XII in AGOT:
Robert could be merciful. Ser Barristan was scarcely the only man he
had pardoned. Grand Maester Pycelle, Varys the Spider, Lord Balon
Greyjoy; each had been counted an enemy to Robert once, and each had
been welcomed into friendship and allowed to retain honors and office
for a pledge of fealty. So long as a man was brave and honest, Robert
would treat him with all the honor and respect due a valiant enemy.
Robert refused to kill Selmy for fighting against him. From AGOT:
"Roose Bolton urged us to cut his throat, but your brother said, 'I
will not kill a man for loyalty, nor for fighting well,' and sent his
own maester to tend Ser Barristan's wounds." He gave the king a long
cool look. "Would that man were here today."
So it must have been a nice shock for Selmy who had seen his liege lord Aerys burn his enemies for non-existent treasons. That's how Robert won his heart. However he eventually came to regret it.
From ASOS, Daenerys V:
Ser Barristan had tears in his eyes. "I took Robert's pardon, aye. I
served him in Kingsguard and council. Served with the Kingslayer and
others near as bad, who soiled the white cloak I wore. Nothing will
excuse that. I might be serving in King's Landing still if the vile
boy upon the Iron Throne had not cast me aside, it shames me to admit.
But when he took the cloak that the White Bull had draped about my
shoulders, and sent men to kill me that selfsame day, it was as though
he'd ripped a caul off my eyes. That was when I knew I must find my
true king, and die in his service—"
Lyanna was not in any danger
That's true. But Jon would be. They were fighting for the Dragon inside Lyanna's womb mainly, not Lyanna herself.
Rhaegar may have known that first act of the rebels will be to wipe out all the potential claimants from the Royal House if they'd any hope to stabilize Robert's rule. Thus sensing a danger to the unborn child, he may have left the Kingsguard behind.
What orders did he give? Why did he do it? We do not know. We know however, it was Lyanna's words and Eddard's honor which protected Jon afterwards, not the white swords. Robert would have killed Jon if he had gotten so much as a whiff of who he really was.
What does GRRM say
GRRM has talked about this in a correspondence with a fan:
Q: The top three kingsguards, including the lord commander amd [sic] the best knight in ages, Ser Arthur Dayne are present there. Lyanna is in
the tower, she asked Ned to promise him something. This, so says the
general consensus us little Jon Snow, who is Lyanna's and Rhaegar's.
No sense denying this ;) However, what are the Kingsguards doing
fighting Eddard? Eddard would never hurt Lyanna, nor her child. The
little one would be safe with Eddard as well, him being a close
relative. So I ask you, was there someone else with Lyanna and Jon?
A: You'll need to wait for future books to find out more about the Tower
of Joy and what happened there, I fear.
I might mention, though, that Ned's account, which you refer to, was
in the context of a dream... and a fever dream at that. Our dreams are
not always literal.
Q: Also, did the Kingsguards know what was in the Tower?
A: Certainly.
