A) Programmers prevent or allow.
B) semantics and definitions, even philosophy
Any unattended(read, uninfluenced) self learning system, even with a few hardcoded, unalterable instructions could devolve into something unforseen and dangerous in some way or another.
The fact that we humans haven't overcome this (see our world of extremes in bliss and suffering as it exists today and history) makes me wary of AI that is not monitored in its learning and possible self programming.
At the very least it needs to have a failsafe installed for immediate shut down or human interference.
But even if learning and action is monitored or influenced very quickly there is a point where no human is able to oversee or understand every line of code or behavioural pattern matrix or tensor field.
Also, hacking,malfunctions and viruses could always circumvent anything or crash or stall the systemt, thus rendering even failsafes useless.
There will need to be crystal clear definitions of what is meant by what instruction in order to overcome ambiguities.
Again we haven't mastered this, so there is no hope for AI - an intellect so foreign once it reached its true potential that we can't even fathom its thought processes, let alone ethics.
However if the instructions are sufficiently black and white, reducing any shade of grey into a yes or no, there at least might be a way.
For instance the AI could determine that humans require freedom to thrive, even exist or be happy.
Thus taking it away would harm mankind and be an invalid choice.
Should nobody have thought of "telling" the AI this natural urge to freedom and there was no instance where it could have been observed by the AI (or dismissed as irrelevant), it could very well decide to take the action you propose.
As you see it comes down to human nature and the biological, mental and social intricacies that are sometimes so abstract and interwoven that centuries, even millennia of philosophy and science fall short in understanding them.
A collection of hardcoded conditions to test against might sufficiently mitigate the risk of death or harm to humans or society.
First step would be NOT to put weapons into AI driven machinery that may roam freely.(Yeah, because we'd never do that)
Another would be NOT to hand over every vital control system to AI without manual override possibilities.(but it is sooo convenient)
Yet another would be to keep the AI simple in their range of actions and fields of expertise, making it easier to predict most, even all possible actions and setting a proper framework (at least according to the then current ideology, societal norm, law, ethical codex etc. - oh no this already falls apart as well as soon as these change).
There are many more and the more cases we think of, the more we enter B) and C) again as these are actually all the very same problem...feels like a recursive loop...
So in dealing with highly evolved AI essentially you either create tools that are somewhat intelligent (possibly sentient) and effectively enslaved(under the control of humans) or you do what we (mostly) do with humans: let them learn from their surroundings and then roam free and hope for the best and that their impulse to self preservation keeps them from going on rampages out of fear of retaliation...well, we see how well that works for us...so good luck either way...