There are a number of reasons for this.

First of all, the kingdom of Arnor wasn't a single united kingdom, but split (it's important that this split happened in the time before the Witch-king):

After Elendil and Isildur there were eight High Kings of Arnor. After Earendur, owing to dissensions among his sons their realm was divided into three: Arthedain, Rhudaur, and Cardolan.

Then, the three petty-kingdoms began to squabble between each other (it's also notable that the line of Isildur was only maintained in one of the petty-kingdoms):

In Arthedain the line of Isildur was maintained and endured, but the line soon perished in Cardolan and Rhudaur. There was often strife between the kingdoms, which hastened the waning of the Dunedain.

Then Rhudaur, as the weakest of the three, was easily overcome and dominated:

In the days of Argeleb son of Malvegil, since no descendants of Isildur remained in the other kingdoms, the kings of Arthedain again claimed the lordship of all Arnor. The claim was resisted by Rhudaur. There the Dunedain were few, and power had been seized by an evil lord of the Hill-men, who was in secret league with Angmar.

From there we get a bit of divide-and-conquer:

A great host came out of Angmar in 1409, and crossing the river entered Cardolan and surrounded Weathertop. The Dunedain were defeated and Arveleg was slain. The Tower of Amon Sul was burned and razed; but the palantír was saved and carried back in retreat to Fornost, Rhudaur was occupied by evil Men subject to Angmar, and the Dunedain that remained there were slain or fled west Cardolan was ravaged.

The Great Plague didn't help matters much either:

In the days of Argeleb II the plague came into Eriador from the Southeast, and most of the people of Cardolan perished, especially in Minhiriath. The Hobbits and all other peoples suffered greatly, but the plague lessened as it passed northwards, and the northern parts of Arthedain were little affected. It was at this time that an end came of the Dunedain of Cardolan, and evil spirits out of Angmar and Rhudaur entered into the deserted mounds and dwelt there.

By now Arthedain is left on it's own:

In 1974 the power of Angmar arose again, and the Witch-king came down upon Arthedain before winter was ended. He captured Fornost, and drove most of the remaining Dunedain over the Lune; among them were the sons of the king.

So it's time to bring on the Easterlings:

But neither kingdom was able to send help to the other; for Angmar renewed its attack upon Arthedain at the same time as the Wainriders reappeared in great force.

And then Arthedain is destroyed:

Then Eärnil sent his son Earnur north with a fleet, as swiftly as he could, and with as great strength as he could spare. Too late. Before Earnur reached the havens of Lindon, the Witch-king had conquered Arthedain and Arvedui had perished.

All references: Lord of the Rings, Appendix A.

Finally, it's wrong to think that the population of Arnor was exclusively Dunedain: it wasn't. The Realms in Exile were actually mostly made up of indigenous people, who the Númenoreans just ruled over.

There are a number of reasons for this.

First of all, the kingdom of Arnor wasn't a single united kingdom, but split (it's important that this split happened in the time before the Witch-king):

After Elendil and Isildur there were eight High Kings of Arnor. After Eärendur, owing to dissensions among his sons their realm was divided into three: Arthedain, Rhudaur, and Cardolan.

Then, the three petty-kingdoms began to squabble between each other (it's also notable that the line of Isildur was only maintained in one of the petty-kingdoms):

In Arthedain the line of Isildur was maintained and endured, but the line soon perished in Cardolan and Rhudaur. There was often strife between the kingdoms, which hastened the waning of the Dunedain.

Then Rhudaur, as the weakest of the three, was easily overcome and dominated:

In the days of Argeleb son of Malvegil, since no descendants of Isildur remained in the other kingdoms, the kings of Arthedain again claimed the lordship of all Arnor. The claim was resisted by Rhudaur. There the Dunedain were few, and power had been seized by an evil lord of the Hill-men, who was in secret league with Angmar.

From there we get a bit of divide-and-conquer:

A great host came out of Angmar in 1409, and crossing the river entered Cardolan and surrounded Weathertop. The Dunedain were defeated and Arveleg was slain. The Tower of Amon Sul was burned and razed; but the palantír was saved and carried back in retreat to Fornost, Rhudaur was occupied by evil Men subject to Angmar, and the Dunedain that remained there were slain or fled west Cardolan was ravaged.

The Great Plague didn't help matters much either:

In the days of Argeleb II the plague came into Eriador from the Southeast, and most of the people of Cardolan perished, especially in Minhiriath. The Hobbits and all other peoples suffered greatly, but the plague lessened as it passed northwards, and the northern parts of Arthedain were little affected. It was at this time that an end came of the Dunedain of Cardolan, and evil spirits out of Angmar and Rhudaur entered into the deserted mounds and dwelt there.

By now Arthedain is left on its own:

In 1974 the power of Angmar arose again, and the Witch-king came down upon Arthedain before winter was ended. He captured Fornost, and drove most of the remaining Dunedain over the Lune; among them were the sons of the king.

So it's time to bring on the Easterlings:

But neither kingdom was able to send help to the other; for Angmar renewed its attack upon Arthedain at the same time as the Wainriders reappeared in great force.

And then Arthedain is destroyed:

Then Eärnil sent his son Eärnur north with a fleet, as swiftly as he could, and with as great strength as he could spare. Too late. Before Eärnur reached the havens of Lindon, the Witch-king had conquered Arthedain and Arvedui had perished.

All references: Lord of the Rings, Appendix A.

Finally, it's wrong to think that the population of Arnor was exclusively Dunedain: it wasn't. The Realms in Exile were actually mostly made up of indigenous people, who the Númenoreans just ruled over.