From the original Matrix:
SMITH: You hear that Mr. Anderson?... That is the sound of inevitability... It is the sound of your death... Goodbye, Mr. Anderson...
Then in the Revolutions:
ORACLE: Everything that has a beginning has an end. I see the end coming. I see the darkness spreading. I see death. And you are all the stands in his way."
ORACLE: Very soon, he is going to have the power to destroy this world, but I believe he won't stop there. He can't. He won't stop until there's nothing left at all.
NEO: What is he?
ORACLE: He is you, your opposite, your negative, the result of the equation trying to balance itself out.
NEO: And if I can't stop him?
ORACLE: One way or another, Neo, this war is going to end.
SMITH: Wait, I've seen this. This is it; this is the end! Yes, you were laying right there, just like that, and I...I stand here, right here, and I'm supposed to say something...I say, "Everything that has a beginning has an end, Neo."
SMITH: What? What did I just say? No, no, this isn't right. This can't be right. Get away from me!
NEO: What are you afraid of?
SMITH: It's a trick!
NEO: You were right Smith. You're always right. It was inevitable.
SMITH: It's not fair!
Smith has seen the future. Presumably, the Oracle saw it as well, and he assimilated the Oracle. She isn't really speaking through him, but Smith has seen what she saw.
As part of the same vision, Smith knows he -- one way or another -- will be destroyed, and that frightens him. He calls it "a trick", but Neo and his opposite Smith "balancing the equation" is inevitable .