I've never read of any loyalists jumping ship after the Heresy began. 

Horus recruited every Primarch to his side he could before the killing started on [Istavaan III][1] . 

That was the whole point of that trap, to kill everyone he knew would never join him. If Horus couldn't sway a Primarch to his cause, that Primarch's Astartes would certainly not join him.

  [1]: http://warhammer40k.wikia.com/wiki/Istvaan_III_Atrocity