"Middle-Earth" has several meanings
It has a very specific cosmological definition: all the lands and seas of the world (Arda), excluding Aman (where the Valar live). However, it is also used colloquially to refer to either the main continent or even just the Northwest region of the continent.
Cosmology
Initially, the term "Middle-Earth" referred to all the lands and seas of Arda.
The Middle-earth lies amidst the World, and is made of land and water; and its surface is the centre of the world from the confines of the upper Vaiya to the confines of the nether. Of old its fashion was thus. It was highest in the middle, and fell away on either side into vast valleys, but rose again in the East and West and again fell away to the chasm at its edges.
And the two valleys were filled with the primeval water, and the shores of these ancient seas were in the West the western highlands and the edge of the great land, and in the East the eastern highlands and the edge of the great land upon the other side. But at the North and South it did not fall away, and one could go by land from the uttermost South and the chasm of Ilmen to the uttermost North and the chasm of Ilmen.
The History of Middle-Earth Volume 4, The Shaping of Middle-Earth, The Ambarkanta, p. 238
ibid., p. 243
In this sketch, Middle-Earth refers to the land marked "Ambar", including the two seas marked ëar ("water, sea"). The middle part of the land, the "bump" surrounded by the seas, is called Endor ("midmost point"). In later texts, Endor would become synonymous with Middle-Earth, so it is unclear if this distinction still existed.
Below is a simplified map of Endor, without the West and East seas. It only represents the "bump" in the middle of the sketch. At this time, the Valar dwell on the Isle of Almaren, in the middle of the Great Lake.
Map of Arda before the destruction of the Pillars, The Atlas of Middle-Earth, p. 2
In the middle of what?
A full cosmological discussion is beyond the scope of this answer, but here's a simplified explanation. The following terms are all found on the sketch above.
When Eru created Arda, he first made a globe. It is a simple sphere of mostly impassable walls (Ilurambar, "Walls of the World"). Outside the globe is the Void (Kúma). The only instances of something passing through these walls are the Valar coming in and Melkor being thrusted out.
Inside this globe is Vaiya ("fold, envelope"). It is described as something of a fluid, forming a sea at the bottom of the globe and air at the top.
"Middle-Earth", then, is a chunk of land that floats on the bottom, sea-like part of Vaiya. Its surface is literally in the middle of the globe and it is surrounded on all sides by Vaiya.
Side-note: Canonicity
It is often difficult to figure out what material was written the latest. Christopher Tolkien, in his History of Middle-Earth series, never makes much of an effort to establish the "real story", that is, what Tolkien considered "real" on the day of his death.
It shows, I think, that even Tolkien himself was unsure about many things and that his mythology was in a constant state of change, sometimes with fundamental repercussions. Many important ideas were introduced late in his life, but without ever changing earlier texts.
The cosmological discussion above is based mostly on The History of Middle-Earth Volume 4: The Shaping of Middle-Earth. I know some of the terms were changed later on. For example, Ekkaia is used in The Silmarillion instead of Vaiya and its properties are somewhat different.
In short, what I wrote above is either correct, or was correct for a period of time. In any case, it gives a good idea of the origin of the name "Middle-Earth".
Creation of Valinor
After the Pillars are destroyed, the Valar leave the Isle of Almaren and move to the Westernmost part of Middle-Earth, past the mountains. They raise more mountains and push Middle-Earth to the East.
Then the Valar removed into the West and forsook the island; and upon the highland at the western side of the West Sea they piled great mountains, and behind them made the land of Valinor. [...]
For their further protection the Valar thrust away Middle-earth at the centre and crowded it eastward, so that it was bended, and the great sea of the West is very wide in the middle, the widest of all waters of the Earth. [...]
The History of Middle-Earth Volume 4, The Shaping of Middle-Earth, The Ambarkanta, pp. 238-239
The following is a map of all of Middle-Earth, plus Valinor. You can see the shores being "bent" towards the East, corresponding to the land that was pushed to accomodate Valinor. The hand-drawn sketch above is a cross section of this map (but without Valinor).
Map of Arda before the destruction of the Pillars, The Atlas of Middle-Earth, p. 2
Pushing so much land around also had the effect of creating the major mountain chains on Endor:
And the thrusting aside of the land caused also mountains to appear in four ranges, two in the Northland, and two in the Southland; and those in the North were the Blue Mountains in the West side, and the Red Mountains in the East side; and in the South were the Grey Mountains and the Yellow.
The History of Middle-Earth Volume 4, The Shaping of Middle-Earth, The Ambarkanta, pp. 238-239
My understanding at this point is that, strictly speaking, Middle-Earth goes from the land East of the mountains of Valinor to the Lands of the Sun in the East, including both Seas.
The Second Great Battle
This particular geography would only last until the Elves are discovered and the Valar fight the Second Great Battle against Melkor. After the battle, several regions are now defined within the continent, such as Beleriand in the Northwest, Palisor in the East and the Hither Lands in the South. These are merely subdivisions of the continent.
Map of Arda after the Second Great Battle, The Atlas of Middle-Earth, pp. 4-5
Note that the Dark Lands appearing after the Second Great Battle is the Eastern part of the continent, now sundered by a sea. It is still part of Middle-Earth.
The War of Wrath
The next major change in geography is at the end of the First Age, during the War of Wrath, when a large chunk of the Westernmost part of the continent (Beleriand) is completely destroyed and the island of Númenor appears.
Map of Arda after the War of Warth, at the beginning of the Second Age, The Atlas of Middle-Earth, pp. 38-39
The Akallabêth
Finally, near the end of the Second Age, Arda becomes round instead of flat and the island of Númenor is destroyed, but this event does not significantly change the geography of the continent.
Other usage
In most of Tolkien's writings not concerned with cosmology, "Middle-Earth" only refers to the central continent, not including the Seas. In colloquial usage, it can also refer to the Northwestern part of the continent where most of the stories about Elves and Men take place.