Can you tell my robot to kill itself? (Three Laws)

Question

In the worlds of Asimov's robots, what's to stop a bunch of vandal humans from going around telling robots to destroy themselves? According to the Second Law, they would have to obey. Here's a refresher of the laws, copied from Wikipedia:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Even if you suppose a robot is ordered by a human not to destroy itself (say, by a forward-thinking engineer in the factory)…

Robot, assume it will harm a human or humans if you arbitrarily destroy yourself for no reason other than being told to do so.

…wouldn't that pretty much constitute an implied law exists between 2 and 3?

Answer 1

The Wikipedia article does say that

Asimov himself made slight modifications to the first three in various books and short stories to further develop how robots would interact with humans and each other. In later fiction where robots had taken responsibility for government of whole planets and human civilizations

Having said that, your Denial-of-Service attack idea is interesting and would be useful when thinking about autonomous systems being designed today or in the fictional context, when writing plot structures relevant to current ideas.

This subject has been studied in robotics -- you'll find this space.com article an interesting read:

Science Fiction’s Robotics Laws Need Reality Check

by Jeremy Hsu, Special to SPACE.com - August 19, 2009.

Referring to David Woods, a systems engineer at Ohio State University.

Woods worked with Robin Murphy, a rescue robotics expert at Texas A&M University, to create three laws that recognize humans as the intelligent, responsible adults in the robot-human relationship. Their first law says that humans may not deploy robots without a work system that meets the highest legal and professional standards of safety and ethics. A second revised law requires robots to respond to humans as appropriate for their roles, and assumes that robots are designed to respond to certain orders from a limited number of humans.

The third revised law proposes that robots have enough autonomy to protect their own existence, as long as such protection does not conflict with the first two laws and allows for smooth transfer of control between human and robot. That means a Mars rover should automatically know not to drive off a cliff, unless human operators specifically tell it to do so.

Answer 2

In the story Runaround, Asimov clearly indicates that the Three laws are not purely a boolean logic system. In that story SPD is confused because he hits a balance between the two 'potentials': one from the casually stated order to go and get essential minerals (second law), one by the 'third law' desire to protect his existence, since the area the minerals are in is dangerous to him. The 'second law' potential is weak (because the order was given casually) and the 'third law' potential has been heightened because Speedy is an extremely expensive robot. This leads us to believe that the laws are not hard-and-fast boolean conditions, but a weighted balance of priorities.

The likely answer to your question is therefore that it depends on the circumstances. A casual instruction from someone the robot didn't know might be ignored. A forcefully given instruction from someone in authority might be obeyed.

Answer 3

Some of the stories set inside the universe handle this case:

1. In Runaround there is a conflict between the second and third law (check the answer by DJClajworth)

2. In the Bicentennial Man / The Positronic Man one of the elements of the storyline is that people order the protagonist to dismember himself, which he has to do.

3. In the second robot trilogy (Caliban, Inferno and Aurora), which was not written by Asmiov one of the main point in the storyline is, that any capable man can ask a robot to destroy himself, and that's what makes a scientist to create a new set of laws, that make the robots more of an ally instead of a slave.

Answer 4

A bit late, but I just read this very relevant part from The Robots of Dawn (chapter Daneel). Elijah Baley suggests ordering self-destruction as a way a "roboticide" might have been committed to Daneel. His reaction:

On Aurora - or on any of the Spacer worlds - robots are regarded more highly than on Earth, and are, in general, more complex, versatile, and valuable. The Third Law is distinctly stronger in comparison to the Second Law on Spacer worlds than it is on Earth. An order for self-destruction would be questioned and there would have to be a truly legitimate reason for it to be carried through - a clear and present danger.

Answer 5

You are harmed if a vandal takes a sledge hammer and destroys your robot, aren't you? You suffer financial loss; perhaps you even grieve for a lost friend.

If a robot is instructed to destroy itself without motive it would cause you the same harm, and hence feel compelled not to obey.

Answer 6

Yes, you can tell your robot, or rather any robot, to destroy itself. The Bicentennial Man was already briefly mentioned by SztupY, but I'd like to expand on his answer, because this story addresses exactly the scenario described by the OP: what's to stop a bunch of vandal humans from going around telling robots to destroy themselves? Apparently nothing:

(emphasis by me)

The tall one said, “Just lie there.” He said to the other, “We can take him apart. Ever take a robot apart?”

“Will he let us?”

“How can he stop us?”

There was no way Andrew could stop them, if they ordered him not to resist in a forceful enough manner. The Second Law of obedience took precedence over the Third Law of self-preservation. In any case, he could not defend himself without possibly hurting them and that would mean breaking the First Law. At that thought, every motile unit contracted slightly and he quivered as he lay there.

The tall one walked over and pushed him with his foot. “He’s heavy. I think we’ll need tools to do the job.”

The nose said, “We could order him to take himself apart. It would be fun to watch him try.”

Andrew Martin is only saved because a member of the Martin family comes along. As a result of this incidence, the Martins fight for robot rights, and eventually they are successful:

“[...] if the robots have Three Laws to protect men, is it too much to ask that men have a law or two to protect robots?”

[...] in the end a law passed which set up conditions under which robot-harming orders were forbidden.

So after these events, ordering a robot to destroy itself would be illegal. Whether or not this would stop vandals is another question.

Answer 7

By killing itself a robot may break the first law. If your robot's job (or at least part of it) involves protecting you, or otherwise looking after you in any way, an order to kill itself would render it unable to do so; that's a clear first law violation so the robot could choose to disobey the order.

Answer 8

The purpose of the laws is to protect humans from robo-harm, not the other way around.

Of course many other rules can exist to help the bots respond sensibly to human input, but that's not Asimov's problem.

Answer 9

I think it is pretty clear that a person can order a robot to destroy itself, and the robot would comply.

IIRC there is an example of this in the novel The Caves of Steel. There is a robot that had knowledge the murderer didn't want disclosed, and IIRC the murderer simply told the robot to destroy itself. The robot took some sort of radiation-emitting device and held it up to its head, scrambling the positronic brain.

We didn't see the actual command given. Probably "Robot, hold this thing up to your head and destroy yourself" would have been adequate. Certainly "Robot, a human being will come to harm unless you scramble your positronic brain" would have worked.

IMHO, Asimov's Three Laws of Robotics would not work in the real world. They are chiefly important because they marked the first time a writer even contemplated a framework of safety rules to keep robots from becoming a threat... before Asimov's robot stories, robots tended to be dangerous things in SF stories.

Asimov himself wrote a bunch of stories exploring the boundaries of the Three Laws. For example, in The Naked Sun, someone pointed out that you could put a robot brain in a spaceship with no crew on board (a purely automated spaceship) and tell the robot brain that all spaceships are machines with positronic brains, and any radio broadcasts from other ships are untrustworthy... then you would have a robotic space ship that would be able to destroy other space ships despite the First Law.

Other authors poked at the Three Laws in stories. The one I remember best is "A Code for Sam", where a Three Laws robot is miserable because the Three Laws are impractical in real life. (Example, a human is smoking a cigarette, and the robot feels it must keep the human from harming himself, so the robot must take the cigarette away from the human; but both human and robot are distressed.)

http://en.wikipedia.org/wiki/The_Three_Laws_of_Robotics_in_popular_culture

There is a really excellent web comic called Freefall that is actually a hard SF story and which has been exploring the issues surrounding robots and the laws under which robots operate. One of the main characters, perhaps the main character, is Florence Ambrose; she is an uplifted wolf, and legally her status is exactly like a robot (they refer to her as a "biological AI" or just an "AI" sometimes). In the Freefall comic, there are about 20 thousand humans living with about 450 million robots on a world that is being terraformed.

At one point, Florence discusses hierarchy of command: if the Mayor of the town ordered the robots not to destroy themselves without a good reason, then an ordinary person would be unable to countermand the order. Florence suggests making a temporary position that outranks all other positions, then having the person in that position give the robots new orders that in effect give them basic rights. The position could always be re-created at need, if any of the new orders turned out to have problems. It would cut down on abusive treatment toward the robots.

http://freefall.purrsia.com/

The human woman in these three comics is the Mayor. I think the planet just has the one city, so she is the highest-ranked government official on the planet. Also in the room are Florence (the uplifted wolf), and the Mayor's intern.

http://freefall.purrsia.com/ff2200/fc02137.htm

http://freefall.purrsia.com/ff2200/fc02138.htm

http://freefall.purrsia.com/ff2200/fc02139.htm

Answer 10

If you can tell something to the robot? Yes, of course, you can. Whatever.

And what would prevent a vandalism in society full of robots, which are able to understand human orders? Oh yes, and what will prevent a vandalism in current society without robots? It is the same.

More than question to the 3 laws, it is a question about human morale and ethics / and vandalism. 3 laws count with ideal human society, as Asimov hoped, as many other thinkers, that evolution will bring humans to the ideal state/society... but experience told us, evolution will not do this.

Solution is when you think about robots as personal property of someone. If you will destroy or harm personal property of another human, you will receive penalty or go to jail. Way of realization in case of destroyed robots can be different. In case of robots with high value you as the owner can give a priority order. If you destroy your own property, you probably loose guarantee or at least value it brings to you. Both things will prevent humans from doing so... Every concept without personal ownership will go to hell, as everything in current society does the same. Public property without owner are usually in a poor state.

No change to the 3 laws is necessary, as any change could lead to machine's decision about humans as a potential danger and their elimination, as it is described in Terminator saga.

Answer 11

well if its true that

it will harm a human or humans if you arbitrarily destroy yourself for no reason other than being told to do so

then by destroying itself it would be breaking rule #1, (regardless of whether it was told so or not) which overrides rule #2. however it depends how you interpret rule #3, does it mean if it does conflict with rule #2 or #1 he must not protect his existence - ignoring all else? or simply that it cannot override rule #1 or #2.

Answer 12

Yes, a human can operate most of machines in a way that leads to the destruction of the machine.

To prevent vandalism you describe, robots need authentication system that would allow them to accept orders from authorised persons only. Same way as you cannot just pick and drive a random car in the street or get access to the controls of the train, in reality you should not be able to control a random robot.

Trains and many factory machines have a "Stop" control that can be operated by a random person. Maybe a robot could possibly accept a "Stop" command from any person in case it is safe for other humans to do so.

Answer 13

On a basic level yes if you tell a robot to pull it's own head off it will as cited in the example from The Caves of Steel mentioned in another answer though in that case the robot involved was a relatively simple one.

Other robots with a more complex view of the three laws balance them somewhat differently, while the order of the laws remains fundamentally the opposing 'potentials' between the second and third laws would lead a robot to at the very least question the necessity of it's destruction and ignore the order entirely if the need to survive has been impressed upon it strongly enough by another human.

Answer 14

Not if I told my robot to not destroy itsself. It would have to follow that order just as much as the order from the person who told it to.

Now it has two conflicting orders, so it either drops to the 3rd law telling it not to harm itsself, or it has to prioritize the conflicting commands. I think that prioritizing those of it's owner when there is no other way to decide would be a decent default.

Answer 15

Tonight with a friend we discuss about the three laws even the fourth when the zeroth law was added.

We detected the problem. A robot must obey on order given by human, so if a human tell him to kill itself, the robot will do it because the third law only came next.

For the robot not killed itself a simple law must be added to the precedent laws, between to first and second law.

A robot must not end its own existence, except where such orders would conflict with the First Law or cause greater harm to humanity itself

It change all the following laws so it will became :

• A robot may not harm humanity, or by inaction, allow humanity to come to harm.
• A robot may not injure a human being or, through inaction, allow a human being to come to harm, except when required to do so in order to prevent greater harm to humanity itself.
• A robot must not end its own existence, except where such orders would conflict with the First Law or cause greater harm to humanity itself.
• A robot must obey any orders given to it by human beings, except where such orders would conflict with the First or Second Law or cause greater harm to humanity itself.
• A robot must protect its own existence as long as such protection does not conflict with the First, Second or Third Law or cause greater harm to humanity itself.