The post credits scene of Thor showed
Erik being controlled by Loki - apparently from nearby.
But at the beginning of The Avengers, Loki teleports back to Earth.
Is there any indication of where was coming from or why he apparently needed the power of the Tessarect/Cosmic Cube to make the trip?
Having recently re-watched the scene in question, I don't see any indication that Loki is physically present whilst influencing Selvig's decision to study the Cube--Loki is only seen by the audience, and does not interact with any of the physical objects in the room.
Given that we now know Loki's scepter contained
the Mind stone
It's not unreasonable to assume that he was able to reach across the universe to manipulate Selvig mentally (Mental powers being well within the bailiwick of his scepter), but required the power of the Tesseract to physically travel to earth.
In the beginning of the movie... After mysterious behaviour of Tesseract, it was guessed that Tesseract was being triggered from other side. Other side which wasn't visible to us could only be other dimension. So, Loki came from other dimension (probably, after a meeting with Chitauri).
In comics, its shown that he can cross dimensions on his own, but in Cinematic universe, he doesn't have such powers (yet). So, he needed power of Tesseract to switch dimension.
