The Asgard probably wanted no part of the Tollan people due to the fact that they were just plain "do nothings." I mean, other than one solitary attempt to help the people of another planet in their solar system to advance--which ended in utter disaster for all involved, including the Tollans who lost their home planet--it is apparent that even with spaceships--which Narim said they had in the episode where Anubis was destroying their planet. Narim's last, garbled, fading message to earth, nonetheless, clearly said that "even their spaceships were being shot down before they could leave the atmosphere" (or clear orbit, or something similar)--the Tollan stayed home and kept to themselves except when they needed something from someone.
Consequently, since they would not pass on knowledge, nor help the other races battle the "bad guys" in the Milky Way, nor explore or stay inquisitive, nor even lend a hand, a la "The Peace Corps" to help lesser societies to have better, though low-tech improvements for a better life, the Asgard were probably correct in assuming that the Tollan were arrogant, unimaginative, non-inquisitive, unhelpful inhabitants of a galaxy that they should have, at the very least, been helping the various "peoples" therein to grow, prosper, and defend themselves from all unwarranted aggressors, foreign or domestic. Instead, they seemed to have an attitude of "We got ours, so the rest of you can f*ck off."
Despite the fact that, earlier in their history, they were willing to attempt to help a local race to advance, it was clear that they felt no need anymore to do anything for anyone. As such, they would obviously not be the sort of people whom the Asgard would want to give even more knowledge that would also be selfishly hoarded, for starters. The other reasons are in my argument and others' above. No, the Tollan would never have been called "The Fifth Race" under any circumstances--short of the entire galaxy being reduced down to only five races, total. In that case, the Tollan could have been called the fifth race simply because, semantically speaking alone, they were...if someone was counting and numbering the remaining races, that is. Otherwise, there is no way that they would have earned the appellation, without first undergoing many permanent upgrades to their core philosophies as a people.