This is a fine question, but there is no explicit answer from Tolkien. But if you look at what the he might have known and felt, Aragorn's mistrust of Saruman is unsurprising. We do have a lot of circumstantial information:
(1) Always remember that while we know that Saruman was corrupted by Sauron, none of the characters in the story had any idea of this. (If Gandalf was at all suspicious, would he have gone to Orthanc alone with knowledge of the One Ring's location?)
(2) We're not given many glimpses of Saruman's character before the days of LotR, but Saruman had settled in the fortress Orthanc and seems to have been a remote, forbidding figure, deep in esoteric studies; Gandalf was the "Grey Pilgrim" and, while scary at times, was easy to like. Aragorn was Gandalf's friend from before he went on his travels south.
(3) It's unclear where Elrond stood in the disagreement within the White Council between Galadriel/Gandalf and Saruman. Plainly he did not take the G/G side (or else Saruman would have been overruled) but there is no evidence that he took Saruman's side, either. He must have weighed the unknowns and concluded that they did not yet know enough to overrule the secretive and knowledgeable Saruman. But had he been on Saruman's side, this would have come up in the discussion of the White Council's deliberations. As he was Aragorn's foster father, if he talked with him at all about such high matters, he must have communicated his concerns to Aragorn: Saruman is an arrogant, secretive bastard, while G/G are perhaps too impetuous. (Which would appeal more to a young man?)
(4) Corruption -- both in the real world and in Tolkien -- is always a gradual process. Good people don't one day wake up and say, "Golly. I never realized that I have a lust for domination over the world and the ruination of lives. Who knew?" Corruption is the result of many (frequently small) bad choices. Saruman did not become corrupt the first time he encountered the flaming Eye in the palantir -- he was already arrogant and inclined to tell people what to do rather than to explain and honestly persuade. Sauron probably played him like a fish and let him think that he was gaining the upper hand from their encounters. Consider Saruman's monolog to Gandalf where he talks about Saruman and Gandalf "helping" Sauron and then after Sauron's victory guiding him and perhaps even taking over. By this time he was completely under Sauron's power, but (most of the time, anyway) fooled himself into thinking otherwise.
This sort of corruption can't happen without a corresponding change in Saruman's character. His colleagues may not have guessed what was behind it, but they had to have seen that Saruman had grown more obnoxious over the years.
(5) Everyone understood that one of Saruman's great strengths was his ability to persuade and daunt with his voice. I.e., he was a super-salesman. Who trusts such a person when not under his spell? Aragorn, especially, being a strong-willed man of action, has to have been repelled by what he knew of Saruman's powers.
Saruman - aloof, reluctant to confront Sauron, has a bit of skeevy feel to him, might be the world's greatest liar for all you can tell, at odds with Aragorn's mentors, maybe a bit hungry for domination
Gandalf - a friend, a wanderer without power, honest, respected by the mentors -- and a mentor himself
If you were Aragorn, who would you trust? Who would you want guiding affairs?