"Middle-Earth" has several meanings
It has a very specific cosmological definition, but is also used colloquially to refer to various regions.
Early history
Initially, the term "Middle-Earth" referred to all the lands and seas of Arda.
The Middle-earth lies amidst the World, and is made of land and water; and its surface is the centre of the world from the confines of the upper Vaiya to the confines of the nether. Of old its fashion was thus. It was highest in the middle, and fell away on either side into vast valleys, but rose again in the East and West and again fell away to the chasm at its edges.
And the two valleys were filled with the primeval water, and the shores of these ancient seas were in the West the western highlands and the edge of the great land, and in the East the eastern highlands and the edge of the great land upon the other side. But at the North and South it did not fall away, and one could go by land from the uttermost South and the chasm of Ilmen to the uttermost North and the chasm of Ilmen.
The History of Middle-Earth Volume 4, The Shaping of Middle-Earth, The Ambarkanta, p. 238
ibid., p. 243
In this sketch, Middle-Earth refers to the land marked "Ambar". The middle part of the land, surrounded by seas, is called Endor, "midmost point". In later texts, Endor would become synonymous with Middle-Earth, so it is unclear if this distinction still existed.
Below is a simplified map of Endor, without the West and East seas. It only represents the "bump" in the middle of the sketch. At this time, the Valar dwell on the Isle of Almaren, in the middle of the Great Lake.
Map of Arda before the destruction of the Pillars, The Atlas of Middle-Earth, p. 2
Creation of Valinor
After the Pillars are destroyed, the Valar leave the Isle of Almaren and move to the Westernmost part of Middle-Earth, past the mountains. They raise more mountains and push Middle-Earth to the East.
Then the Valar removed into the West and forsook the island; and upon the highland at the western side of the West Sea they piled great mountains, and behind them made the land of Valinor. [...]
For their further protection the Valar thrust away Middle-earth at the centre and crowded it eastward, so that it was bended, and the great sea of the West is very wide in the middle, the widest of all waters of the Earth. [...]
The History of Middle-Earth Volume 4, The Shaping of Middle-Earth, The Ambarkanta, pp. 238-239
Doing so also creates the mountain chains on Endor:
And the thrusting aside of the land caused also mountains to appear in four ranges, two in the Northland, and two in the Southland; and those in the North were the Blue Mountains in the West side, and the Red Mountains in the East side; and in the South were the Grey Mountains and the Yellow.
ibid.
The following is a map of all of Middle-Earth, plus Valinor. The sketch above is a cross section of this map (but without Valinor).
Map of Arda before the destruction of the Pillars, The Atlas of Middle-Earth, p. 2
My understanding at this point is that, strictly speaking, Middle-Earth goes from the land East of the mountains of Valinor to the Lands of the Sun in the East, including both Seas.
The Second Great Battle
This particular geography would only last until the Elves are discovered and the Valar fight the Second Great Battle against Melkor. After the battle, several regions are now defined within the continent, such as Beleriand in the Northwest, Palisor in the East and the Hither Lands in the South. These are merely subdivisions of the continent.
Map of Arda after the Second Great Battle, The Atlas of Middle-Earth, pp. 4-5
Note that the Dark Lands appearing after the Second Great Battle is the Eastern part of the continent, now sundered by a sea. It is still part of Middle-Earth.
The War of Wrath
The next major change in geography is at the end of the First Age, during the War of Wrath, when a large chunk of the Westernmost part of the continent (Beleriand) is completely destroyed and the island of Númenor appears.
Map of Arda after the War of Warth, at the beginning of the Second Age, The Atlas of Middle-Earth, pp. 38-39
The Akallabêth
Finally, near the end of the Second Age, Arda becomes round instead of flat and the island of Númenor is destroyed, but this event does not significantly change the geography of the continent.
Other usage
In most of Tolkien's writings not concerned with cosmology, "Middle-Earth" only refers to the central continent, not including the Seas. In colloquial usage, it can also refer to the Northwestern part of the continent where most of the stories about Elves and Men take place.