Middle-Earth is the name for the parts of the world in Tolkein's universe where men lived. It was based on the Old English/Norse term "Middangeard" as used, for example, in Beowulf. The term "middle" here comes from the Norse idea of nine connected worlds, of which men lived on the one in the center. This distinguishes Middle Earth from, for example, Valinor where the immortal Valar lived.
Tolkein's Middle Earth includes other places that are not on his maps of Middle Earth that are still part of that region, but there are also unmapped areas that are outside of Middle Earth. Tolkein described Middle Earth as basically being surrounded on all sides by ocean, so anything that was "across the seas" would be somewhere else.
In practice, Tolkein fans often just use "Middle Earth" to mean the setting of the novels, without being too concerned with where it stops and starts