Did Lore and B4 ever know each other? Lore knew Data as his younger brother but did he ever come to know his older brother?
We know memories can be extracted from a positronic brain (Data to B-4, Lal to Data). And we know Data (or at least Starfleet) had custody of Lore after his deactivation. Thus it stands to reason that if Lore was aware of B-4, someone in the Federation would've begun a search for him before Shinzon's storyline.
The risk is that positronic brains can also be wiped (Data not remembering Juliana Soong), so it's possible Soong might have "restarted" Lore's memories after activating Lore and before shutting down B-4.