The info I remember about the Wall was that it was built, using magic, by the Children of the Forest after the last long winter and Others invasion. It's purpose is to retain the Other should they return.
No doubt the Night's Watch is very concerned about the Others rising. They don't seem to think the Wall alone is nearly adequate defense against the Others. Is it ever explained how exactly the Other can threaten them around the Wall? Can they climb the Wall? Chisel thru the Wall, take over the tunnels?
I know Mance had a horn that could supposedly shatter the Wall. But there's no mention that the Others are searching for that horn or anything like that. If the Night's Watch just brought the horn south of the Wall that would be it for that threat.