I can't remember if in the movies anyone explicitly states that Bifrost is the only way in and out from Asgard, but in Norse mitology Bifrost is just the bridge between Asgard and Midgard (Earth), it does not connect all Nine Worlds. Jötunheimr, for example, is just a different region on the same "planet" as Asgard, and they are separated by a simple river.
As I cannot comment yet, editing for a few words about Heimdall: again, unless in the movie they clearly state the contrary, Heimdall can see everything...that happens on earth. This happens in both myths and comics, and as far I can remember, in the movie there is no one saying "Heimdall the omniscient who can see everything that happens in entire existence".