In the movie Avengers Assemble, Loki says:

......what have I to fear?

Tony tells Loki:

The avengers. That's what we call ourselves. Earth's mightiest heroes.....

Then why does Thor form a part of the Avengers?

He is not from the Earth. He is from Asgard.