In current Dr. Who canon (a phrase I never thought I'd use) we see that humans forget the Silence after looking away from them. The internet recently presented me with a question- "if Silent looks away from a Weeping Angel, does the Angel get closer or does the Silent forget the encounter?" That question aside- do you have to be sapient to be affected by the Silence? Do cats forget them after looking away? Do mosquitoes? Not sure if an answer exists in canon, but I am curious.
The nature of the psionic ability of the Silents appears to require a mind capable of interacting with their ability. Their power has proven to work on a mind as sophisticated as the Doctor so it can be assumed any lesser intelligence can be affected, making their ability a prodigious one, indeed.
While it has not been demonstrated on the show to work against an animal-like intelligence it has to be assumed their mental ability will work on animals because if it didn't, animals would have revealed their presence to humanity long before now. How pronounced an effect they have on animals is a matter of debate but I suspect at the very least, animals have no more ability to remember them than any other species shown so far.
I would think that any living thing which can retain memory would forget about seeing the Silents. They're described as "memory-proof," which seems to imply that they simply can't be remembered without the help of external devices like the eye drives (although one would assume that they are immune to their own abilities, so they wouldn't forget about seeing fellow Silents). I'm not sure, however, if their post-hypnotic commands could be used on less-sapient beings.
(As for a Silent and a Weeping Angel looking at each other, I think the Silent would have the advantage. While Weeping Angels are fast, they probably can't do a whole lot of damage if they can't even remember what they're trying to kill.)