The crack in time is the key element here.
1. It can be used to eavesdrop.
This is proven by Amelia hearing that "Prisoner Zero has escaped" through the crack.
DOCTOR: It's a crack. But I'll tell you something funny. If you knocked this wall down, the crack would stay put, because the crack isn't in the wall.
AMELIA: Where is it then?
DOCTOR: Everywhere. In everything. It's a split in the skin of the world. Two parts of space and time that should never have touched, pressed together right here in the wall of your bedroom. Sometimes, can you hear?
AMELIA: A voice. Yes.
(There is a vague growling from somewhere. The Doctor empties Amelia's nighttime glass of water and uses it to listen to the crack.)
ATRAXI [OC]: Prisoner Zero has escaped.
DOCTOR: Prisoner Zero?
AMELIA: Prisoner Zero has escaped. That's what I heard. What does it mean?
ATRAXI [OC]: Prisoner Zero has escaped.
DOCTOR: It means that on the other side of this wall, there's a prison and they've lost a prisoner. And you know what that means?
AMELIA: What?
DOCTOR: You need a better wall. The only way to close the breach is to open it all the way. The forces will invert and it'll snap itself shut. Or
AMELIA: What?
DOCTOR: You know when grown-ups tell you everything's going to be fine and you think they're probably lying to make you feel better?
AMELIA: Yes.
DOCTOR: Everything's going to be fine.
The Doctor also states a similar fact when placing Amy in the Pandorica:
(The Doctor places Amy in the Pandorica.)
RORY: So you've got a plan, then?
DOCTOR: Bit of a plan, yeah. Memories are more powerful than you think, and Amy Pond is not an ordinary girl. Grew up with a time crack in her wall. The universe pouring through her dreams every night. The Nestenes took a memory print of her and got a bit more than they bargained for, like you. Not just your face, but your heart and your soul.
(The Doctor mind-melds with Amy.)
DOCTOR: I'm leaving her a message for when she wakes up, so she knows what's happening.
2. The crack appears in many locations.
We've seen the same crack in many places. Most notably, Amy's wall and on one of the TARDIS' monitors.
But they also appeared in locations unrelated to the Doctor and Amy. The vampires (from "Vampires of Venice") ran from Saturnyne after seeing the cracks.
In fact, the crack is everywhere, in all of space and time. The cracks are the result from an exploding TARDIS. While it's exploding, it therefore affects all of space and time, creating cracks everywhere.
River explains this to Amy and Rory:
AMY: What?
RIVER: The Tardis is still burning. It's exploding at every point in history. If you threw the Pandorica into the explosion, right into the heart of the fire.
AMY: Then what?
RIVER: Then let there be light. The light from the Pandorica would explode everywhere at once, just like he said.
AMY: That would work? That would bring everything back?
RIVER: A restoration field powered by an exploding Tardis, happening at every moment in history. Oh, that's brilliant. It might even work.
3. Conclusion
The cracks are everywhere in time and space. We don't know how many there are. They could be millions or billions of them.
The cracks function as portals. They allow people to eavesdrop, but also to use it as a door (this is how Prisoner Zero escaped the Atraxi prison).
The Silence was created specifically to prevent the cracks in time from forming (which is why they consider the Doctor their enemy, as his TARDIS causes the cracks). It is very likely that the Silence therefore have spoken about their intentions while being near one of these cracks.
And if the Silence talk about their intentions, then anyone listening in (Prisoner Zero) could hear it.
Also, notice that Prisoner Zero claims to have had years to psychically link with Amy:
MOTHER: Then I shall take a new form.
DOCTOR: Oh, stop it. You know you can't. It takes months to form that kind of psychic link.
MOTHER: And I've had years.
(Amy collapses.)
DOCTOR: No! Amy? You've got to hold on. Amy? Don't sleep! You've got to stay awake, please.
RORY: Doctor.
(Prisoner Zero has transformed into a gangly man with a ripped shirt and floppy hair.)
DOCTOR: Well, that's rubbish. Who's that supposed to be?
RORY: It's you.
DOCTOR; Me? Is that what I look like?
RORY: You don't know?
DOCTOR: Busy day. Why me, though? You're linked with her [Amy]. Why are you copying me?
If Prisoner Zero has had years to link with Amy, that means he spent years listening through the cracks. This makes it very likely that he also picked up on other conversations that took place near a crack.