A better translation would be 'Silence MUST fall', per Dorium at the end of 'The Wedding of River Song'; the group 'The Silence' is not what the prophecy refers to - they took their name from the prophecy, not the other way around. (Possibly the name came from the 'Silent's' being a primary force in The Silence.. But which started which gets all Timey-wimey, wibbly-wobbly.)
DORIUM: On the fields of Trenzalore, at the fall of the Eleventh, when no living creature could speak falsely, or fail to answer, a question will be asked. A question that must never, ever be answered.
(later)
DORIUM: Silence MUST fall would be a better translation. The Silence are determined the question will never be answered. That the Doctor will NEVER reach Trenzalore.
Their intent is to make sure that he not answer the question; 'His Silence' is what is being called for, as the Doctor is surprised to discover.
DOCTOR: And silence would fall. All the times I've heard those words, I never realised...it was my silence. My death. The Doctor will fall. (looks around) Why are we here?
Now, all that being said.. We have no clue if it's actually going to be correct; attempting to analyze prophecy to circumvent prophesied events is an old, often attempted task, that usually ends up being counterproductive, often showing the prophecy to be self-fulfilling. It may actually be the group the Silence that falls.. due to their attempts to prevent his answer.
Wouldn't be the first time the show has yanked us back and forth regarding how to interpret upcoming events. (Case in point; the Doctor's 'death')
(Transcript clips from here)