Some people have in fact figured out that it was The Twins who did the act, more precisely Jaime. Some people just know it.
Tyrion suspected from the beginning that his siblings may have something to do with Bran's fall. He was keeping an eye out for their reactions when they were breaking fast at Winterfell afterwards.
Myrcella gave a happy gasp, and Tommen smiled nervously, but it was
not the children Tyrion was watching. The glance that passed between
Jaime and Cersei lasted no more than a second, but he did not miss
it. Then his sister dropped her gaze to the table. “That is no
mercy. These northern gods are cruel to let the child linger in such
Later Cersei confirmed it to Tyrion herself in ACOK Chapter 54 when Tyrion tells him that there are reports of murder of Stark boys by Theon Greyjoy:
"I trust you're pleased," he said as she read. "You wanted the Stark
boy dead, I believe."
Cersei made a sour face. "It was Jaime who threw him from that window,
not me. For love, he said, as if that would please me....when did our
sweet brother ever stop to think?"
- Eddard Stark
Eddard Stark had suspected Lannister involvement with second assassination attempt (With knife) but Cersei somewhat confirmed the first one as well in AGOT:
“My son Bran...”
To her credit, Cersei did not look away. “He saw us. You love your
children, do you not?” Robert had asked him the very same question,
the morning of the melee.
Note how Cersei does not mention throwing the boy out of the window so it may be that Eddard may have been thinking that Cersei confessed about second attempt, not the fall. Or he may have figured out that if Bran saw the twins, his fall was not an accident.
Catelyn figured out1 the second attempt after she met Stannis and Renly.
Ned must have known, and Lord Arryn before him. Small wonder that the
queen had killed them both. Would I do any less for my own? Catelyn
clenched her hands, feeling the tightness in her scarred fingers where
the assassin’s steel had cut to the bone as she fought to save her
son. “Bran knows too,” she whispered, lowering her head. Gods be
good, he must have seen something, heard something, that was why they
tried to kill him in his bed.
While She harbored doubts about Bran's accident since the second attempt, she was not sure until she talked with Jaime in ACOK:
Jaime drank some more wine. “Ask your next.”
Catelyn wondered if he would dare answer her next question with
anything but a lie. “How did my son Bran come to fall?”
“I flung him from a window.”
The easy way he said it took her voice away for an instant. If I had a
knife, I would kill him now, she thought, until she remembered the
girls. Her throat constricted as she said, “You were a knight, sworn
to defend the weak and innocent.”
“He was weak enough, but perhaps not so innocent. He was spying on
- Brienne of Tarth:
Brienne was standing outside Jaime's cell when Catelyn made Jaime confess about throwing Bran out of the window. Brienne heard everything. From ASOS, Brienne tells Jaime that she heard everything:
“Talk with Ser Cleos then. I have no words for monsters.”
Jaime hooted. “Are there monsters hereabouts? Hiding beneath the
water, perhaps? In that thick of willows? And me without my sword!”
“A man who would violate his own sister, murder his king, and fling
an innocent child to his death deserves no other name.”
Bran himself knows what happened that day and who flung him. The memory came back to him when Clay Cerwyn came to visit Winterfell and his retinue mentioned incest and Lannisters.
“He’s a king now too,” Cley confided. “He says Queen Cersei bedded her
brother, so Joffrey is a bastard.”
“Joffrey the Illborn,” one of the Cerwyn knights growled. “Small
wonder he’s faithless, with the Kingslayer for a father.”
“Aye,” said another, “the gods hate incest. Look how they brought down
For a moment Bran felt as though he could not breathe. A giant hand
was crushing his chest. He felt as though he was falling, and clutched
desperately at Dancer’s reins.
His terror must have shown on his face. “Bran?” Cley Cerwyn said. “Are
you unwell? It’s only another king.” . . . . . [Quote skipped]
The pain was an axe splitting his head apart, but when the crow
wrenched out its beak all slimy with bits of bone and brain, Bran
could see again. What he saw made him gasp in fear. He was clinging
to a tower miles high, and his fingers were slipping, nails
scrabbling at the stone, his legs dragging him down, stupid useless
dead legs. “Help me!” he cried. A golden man appeared in the sky
above him and pulled him up. “The things I do for love,” he murmured
softly as he tossed him out kicking into empty air.
Later he thinks the following when the Reeds ask him what he is afraid of:
"The falling, Bran thought, and the golden man, the queen's brother,
he scares me too, but mostly the falling. He did not say it, though.
How could he? He had not been able to tell Ser Rodrik or Maester
Luwin, and he could not tell the Reeds either. If he didn't talk about
it, maybe he would forget. He had never wanted to remember. It might
not even be a true remembering."
1. Catelyn was wrong however. Cersei and Jaime were not behind the second attempt