This question already has an answer here:
When Qui-Gon encounters Darth Maul and the Jedi realise he was a Sith, they are amazed, having believed that they had been extinct for a long time.
On the other hand, at the end of Episode I, Yoda says "always two there are, the master and the apprentice", so they knew about the Rule of Two.
How can this be explained?
EDIT (given the answers)
Let's list the things that happened, which apparently the Jedi were aware of:
1) 1000 years ago the Sith were almost wiped out (became "extinct")
2) 1000 years ago, because they were almost extinct, the Sith started existing through a single pair (master and apprentice)
Given these two facts, why are they surprised? Yes, the Sith were extinct for 1000 years as a civilisation, but there are always two !