From Wikipedia (emphasis mine):
A castle is a type of fortified structure built in Europe and the Middle East during the Middle Ages by European nobility. Scholars ... usually consider [the scope of the term] to be the private fortified residence of a lord or noble. This is distinct ... from a fortress, which was not always a residence for nobility; and from a fortified settlement...
... In its simplest terms, the definition of a castle accepted amongst academics is "a private fortified residence".
Castle Black is not the private residence of any Westerosi noble. Why, then, is Castle Black not merely a Fortress, like the Nightfort, for example?
One could argue that maybe the Lord Commander of the Night's Watch is the equivalent of a noble, but - traditionally, he would reside in the Nightfort, and that's not Castle Night or the Nightcastle. Also, neither fortress is "his", i.e. it's not a private residence in any way. So that can't be it.