Remember, Ned seemed to think that the blonde Princelings were Robert's. He never suspected any foul play until suspicious death of Jon Arryn.
Ned didn't really have a grasp of how genetics work of course. He simply saw the precedent and found out that whenever Black haired Baratheon and Blonde Lannisters bred, the off-spring had black hair. He did not realize it even after reading the records over and over again until Sansa pointed out to him that Joffrey was nothing like Robert. He just didn't see anything weird with Joffrey and his siblings being blonds.
“He is!” Sansa insisted. “I don’t want someone brave and gentle, I
want him. We’ll be ever so happy, just like in the songs, you’ll see.
I’ll give him a son with golden hair, and one day he’ll be the king of
all the realm, the greatest king that ever was, as brave as the wolf
and as proud as the lion.”
Arya made a face. “Not if Joffrey’s his father,” she said. “He’s a
liar and a craven and anyhow he’s a stag, not a lion.”
Sansa felt tears in her eyes. “He is not! He’s not the least bit like
that old drunken king,” she screamed at her sister, forgetting herself
in her grief. Father looked at her strangely.
“Gods, “ he swore softly, “out of the mouth of babes...”
AGOT - Chapter 44
That's when everything from Arryn's death, Bran's fall, Stannis' retreat started making sense. It had to be that. It appeared to be the natural and logical conclusion to him.
The seed is strong, Jon Arryn had cried on his deathbed, and so it
was. All those bastards, all with hair as black as night. Grand
Maester Malleon recorded the last mating between stag and lion, some
ninety years ago, when Tya Lannister wed Gowen Baratheon, third son of
the reigning lord. Their only issue, an unnamed boy described in
Malleon’s tome as a large and lusty lad born with a full head of black
hair, died in infancy. Thirty years before that a male Lannister had
taken a Baratheon maid to wife. She had given him three daughters and
a son, each black- haired. No matter how far back Ned searched in the
brittle yellowed pages, always he found the gold yielding before the
AGOT - Chapter 45
Even the bastards like Gendry and Barra whose mothers had light hair inherited the Baratheon Black Locks from Robert. As Ned says, everytime gold yielded before the coal.
Also in another context, the author said:
"I don't know if I want to get into genetics - this is fantasy, not
scifi" He replied.