57

The second law states that:

A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

But it says nothing about property. What if a robot owned by someone is given an order by someone else?

Surely if the same person gives contradictory orders, the last one will be executed:

Move that box upstairs. Wait no, just put it on the porch.

But what if a robot is carrying groceries outside, and someone says:

Hey you, stop what you're doing and take those groceries to my apartment.

It shouldn't agree to do that.

So, do people just say "Only obey me" when they buy a robot? Does that mean a robot ignores every order by non-owners, even if it's small help like helping an old lady cross the road? The robot would have to tell between harmless orders from strangers and orders that cause non-physical harm (like stealing their stuff, smashing their car, etc.)

If a households owns a robot, they'll say something like "Only obey me and my family", then what if orders contradict between family members?

How does it work exactly? Is it just managed by some part of the AI less fundamental than the 3 rules?

I've never read Asimov, sorry if it's explained in the first chapter of the first book. I didn't see it discussed anywhere.

  • 74
    The potential conflicts between the three laws is actually what I, Robot spends most its time examining. I highly recommend giving it a read. – Harabeck Jul 31 at 15:41
  • 5
    You might check out the answers to Can you tell my robot to kill itself? Some of them explain that an earlier order has a higher potential than a later order. And an order that does not involve causing distress to the people it works for would also have higher potential than one that does. – DavidW Jul 31 at 15:51
  • 11
    en.wikipedia.org/wiki/Little_Lost_Robot and less so but en.wikipedia.org/wiki/Liar!_(short_story) - but yeah...pretty much, read the book :) - as Harabeck says, the entire point of the stories is figuring out how conflicts in the laws work themselves out – NKCampbell Jul 31 at 20:51
  • 18
    I suggest you resolve the issue raised in your last line. Its worth doing. – Criggie Aug 1 at 4:03
  • 23
    Out-of-universe but Asimov's Laws are not actually designed to regulate robot behavior, but to generate conflict for storytelling. They are inherently contradictory so they generate interesting discussion for Asimov's stories. – Jacques Aug 1 at 9:08
71

Consider, all Robots stories are short stories or novellas published during a very long time spawn, so they are not 100% coherent among themselves sometimes, though the overall framework is.

As highlighted in the comments, Positronic brains reaction to orders, situations and environment is generally described in-universe as the outcome of the "potentials" induced by them on the 3 Laws. There is no AI at work as we know it today for the Robot to decide to do something or not, but more like a compulsion towards a generic line of action, driven by priorities between the laws and the Robot understanding of his surroundings. To give you an idea, there is some discussion in one of the stories if I remember well on a potential scenario where a strong enough order, repeated continuously under different wording and reinforced with information on how negative would it be for the person issuing it if not followed, could have potentially led to a breach of First Law by simple buildup of potential on Second Law adherence. The idea is rejected as impossible in practice, but it is discussed.

Also take into account there is an in-universe evolution on how Laws are enforced by design; in the first stories, which are supposed to take place earlier in-universe, Laws are rigid, literal absolutes with no space for interpretation. As in-universe timeline progresses, they become more flexible and open to interpretation by the Robot, in part as an attempt by US Robotics to prevent scenarios such as a third party ordering a Robot to, for instance, “get that bike and throw it to the river”, when the bike owner is not present to negate the order.

This flexibility is often presented as a re-interpretation of First Law, where “damaging a human” is originally presented as implying exclusively physical damage, and it slowly includes the concept of mental damage or distress. Under this implementation of the First Law, breaking it is to build up enough negative potential vs. it as per Robot understanding of human mind and social conventions, which leads to amusing side-effects (read the books!).

So the bike will not be thrown to the river because the robot guesses the owner will not like it (negative potential vs. First Law).

So, what does all this rambling mean to your question:

  • An early Robot will tend to do as ordered literally if there is no evident violation of First Law, or a violation of a stronger previous order. A way to protect your Robot from abuse would be to issue a second order in strong terms not to obey counter-orders. This will generate two positive potentials a single counter-order will find difficult to overrule.
  • A later Robot will tend to be more mindful of negative psychological impact of following an order or taking a course of action, which leads to amusing side-effects (seriously, read the books!).
  • Any Robot of whichever period, faced with a situation where contradictory lines of action will have the same potential on all Laws, or inevitably break First Law (later interpreted as leading to a negative high enough potential vs. it, not necessarily implying physical damage), will enter a loop looking for an impossible solution and be blocked.
  • 11
    Yours is the best answer. One should be reminded that the three laws aren't the sole programming in a robot and that the answer to OP would depend on the period of time and the model of the robot. – Echox Aug 1 at 7:22
  • 1
    The development over time is a key part of the answer to this question. By the time of the novels there have been generations of development of the positronic brain to the extent that a Space robot (which are pretty much the only robots by then) would not have a Law 2 conflict ignoring an order from an Earth man, becuase they don't see them as fully human. – Matt Hollands Aug 1 at 7:55
  • 6
    @Echox And also, as is often forgotten, the three laws aren't actually the written laws that everybody remembers - they're very complex programs of their own intertwined throughout the whole brain. The wording is just simplification for the benefits of humans. This is explored many times where they need to make specialised robots that ignore some of the laws. Which really follows nicely, since to interpret the human-worded three laws reasonably correctly, you need to make the same assumptions a sane human would. – Luaan Aug 1 at 7:56
  • 4
    Its worth noting that the laws, by authorial design, we're not perfect because they were impetus for stories. The laws were the conflict, not the solution. – Draco18s Aug 1 at 16:07
  • It is feasible that a robot could be ordered at activation to not become the instrument of a crime. This would prevent it from following an order that would result in theft/vandalism. Of course, there would be other perhaps undesirable side-effects, which would be interesting material for further stories. – called2voyage Aug 1 at 16:10
34

In some stories, robots distinguish importance of an order based on amount of urgency/importance in the language and on who gave the order.

I am thinking of Little Lost Robot as mentioned by NKCampbell in a comment. The premise is that a supervisor gets very angry at a robot and tells it to "Get lost!" using extremely strong and colorful language; the robot responds by hiding among other identical-looking models. It is explained that simply ordering the robot to reveal itself won't work, as it would not countermand the direct order to get lost given in the strongest possible language by 'the person most authorized to command it'

“They are?” Calvin took fire. “They are? Do you realize one of them is lying? One of the sixty-three robots I have just interviewed has deliberately lied to me after the strictest injunction to tell the truth. The abnormality indicated is horribly deep-seated, and horribly frightening.”

Peter Bogert felt his teeth harden against each other. He said, “Not at all. Look! Nestor 10 was given orders to lose himself. Those orders were expressed in maximum urgency by the person most authorized to command him. You can’t counteract that order either by superior urgency or superior right of command. Naturally, the robot will attempt to defend the carrying out of his orders.

I seem to also recall a comment elsewhere to the effect that a robotics expert would better know how to phrase an order to be as strong as possible and likely overcome an order given by an ordinary person.

In general, interpreting how the laws apply to a specific situation is very much a part of the challenge in-universe. Simple models are at risk of burning out their brains if faced with a difficult dilemma involving conflicting laws, orders, or situations. So there's not supposed to be an easy answer to the kinds of questions you're asking, but there are some reasonable principles to follow.

  • 6
    There was a third element in Little Lost Robot, that the robot had a superiority complex which was further reinforcing its desire to successfully hide from the humans' attempts to find it. But that was a much less significant factor than either authority or urgency. – Arcanist Lupus Aug 1 at 3:36
25

As far as I can recall, he doesn't.

It's important to remember that Asimov was writing the robot stories for Astounding and his readers liked logical puzzles, surprise consequences and clever gimmicks — none of which would be present in a simple story of two different people telling a robot to do two different things.

The closest I recall, is it “Robot AL-76 Goes Astray” where a robot built for use on the Moon for some reason gets lost is the back woods. It invents a disintegration tool, causes some havoc and is told by someone to get rid of the disintegrator and forget about building more. When the robot is finally found, the owners want to know how it built the disintegrator, but it can’t help — it’s forgotten everything.

This wasn’t a case of contradictory orders, but it was an order by an unauthorized person.

  • Since the order of the rules is so significant, perhaps order of instructions is significant too. – marcellothearcane Jul 31 at 19:42
  • 1
    @marcellothearcane: Maybe, but notice in the case mentioned in this answer that the first order didn't take precedence over later orders. Instead it was to take an action ("forget" = delete/erase memory) that made later orders impossible to satisfy. Although presumably given the same time and circumstances, it could re-invent a disintegration tool from the same starting point, unless that "forget about building more" order is still in effect and wasn't literally "forget", but "never build more". – Peter Cordes Aug 1 at 3:26
23

There was a case in the short story “Runaround” where a robot had two conflicting objectives. In this case, it was because he was built with an extra-strong Third Law, so could not decide between obeying human orders and self-preservation.

He essentially tries to satisfy both conditions, running circles a safe distance from his dangerous destination, and otherwise acting erratic and drunk.

I would think a similar thing would happen in the case of two conflicting orders. The robot would do what it could to satisfy both, but could become stuck if it was impossible.

In the above story, the deadlock is resolved when the human operators put themselves in danger, thus invoking the First Law. I would think in most cases committing a crime would be considered a minor violation of the First Law, so smart enough robots would try to avoid it. But as Zeiss mentioned, it would really depend on the model of robot.

  • 3
    It wasn't an extra strong 3rd law, but a weak order. The robot didn't know that the order he had been given was vital to the humans' continued survival in the long run. It had been issued as a "oh, by the way, we're running low on phlebotinum, could you get some more?" rather than, "We require a restock, get some." – Draco18s Aug 1 at 16:10
  • 3
    @Draco18s It was both; it was an expensive robot, so the third law was strongner. – wizzwizz4 Aug 1 at 20:25
15

The answer to this question depends strongly on the level of sophistication of the robot. The robot stories (most of them were short stories, but a number were novel-length) covered an internal time span of centuries -- even though the core of the stories involved Susan Calvin from her graduate days until her old age.

Early on, robots weren't "smart" enough to know the difference between sensible orders and nonsensical ones. Later, they reached a level of sophistication where the could debate amongst themselves exactly what constitutes a human -- and conclude that they are more qualified for that moniker than any of these biological entities. Somewhere in between, Second Law developed into some level of qualification relative to the order giver (orders from small children and known incompetents could be ignored). It was out of this latter capability that robots eventually decided that concerns for harm of humans (and humanity) overrode all their orders -- humans must be protected from themselves to the point where they were all declared incompetent to give orders to a robot.

  • Don't forget the Elijah Baley stories (four novels, and one short story), which also featured robots and explored the Three Laws even further. – gidds Aug 1 at 16:39
  • @gidds True, they did -- but I haven't read them except for the first one, and they weren't part of the original overarching robot saga. All but The Caves of Steel were published long after the "last robot story". – Zeiss Ikon Aug 1 at 16:42
  • 2
    Only if you restrict ‘robot stories’ to the ones you know. The Elijah Baley novels were published in 1954, 1957, 1983, and 1985; other robot stories included ‘…That Thou Art Mindful Of Him’ in 1974, ‘The Bicentennial Man’ in 1976, ‘Robot Dreams’ in 1986, ‘Cal’ and ‘Kid Brother’ and ‘Robot Visions’ in 1990. – gidds Aug 1 at 17:03
8

In the extreme, contradictory orders can lead to the permanent shutdown of a robot. This was dealt with in Robots and Empire, where a robot, R. Ernett Second, was under strict orders not to tell anything about his master. When he is forcefully ordered to do so anyway, his brain freezes irreversibly.

“Will you answer my questions and accept my orders, Ernett?”
    “I will, madam, if they are not counteracted by a competing order.”
    “If I ask you the location of your base on this planet what portion of it you count as your master’s establishment—will you answer that?”
    “I may not do so, madam. Nor any other question with respect to my master. Any question at all.”
    “Do you understand that if you do not answer I will be bitterly disappointed and that my rightful expectation of robotic service will be permanently blunted?”
    “I understand, madam,” said the robot faintly. [...]
    Gladia said, in a voice that rang with authority, “Do not inflect damage on me, Ernett, by refusing to tell me the location of your base on this planet. I order you to tell me.”
    The robot seemed to stiffen. His mouth opened but made no sound. It opened again and he whispered huskily, “... mile...” It opened a third time silently—and then, while the mouth remained open, the gleam went out of the robot assassin’s eyes and they became flat and waxen. One arm, which had been a little raised, dropped downward.
    Daneel said, “The positronic brain has frozen.”

3

The robot would only have two real options.

It could malfunction. Runaround, for example, where the robot acted drunk. Or it could shut down entirely.

Or, it could prioritize one order over another. The key there is that the priority scheme is not dictated by the Laws themselves. That means they're flexible to the design of a specific robot and its intended purpose. Prioritize by time, rank, ownership, more complex schemes involving broad understandings of a situation...

Basically, this is a work-around of the second law, by playing with the definition of "order." "That command I got isn't really an order, because it was given by a criminal." But if the laws can be gamed that way, they can also be gamed by defining "human" in a super-narrow fashion, like the Solarians did.

And suddenly the laws may as well not exist.

There's a fundamental conflict between the very concepts of an absolute rule of behavior, and an intelligent actor.

0

In fact there's a whole Asimov story about this problem : The Robots of Dawn

And the answer is simple : mental block, then "death" of the robot.

Two situations happen:

  • first, a robot has an order to find Elijah Baley and guide him somewhere, but the Baley (a human) refuses. This action makes the robot "bug", and Baley complies to the initial order to not break the robot which might be expensive. This is of course an expositional situation for the main plot of the story

  • and the main plot : Jander is a humanoid robot which had a mental block, Elijah must investigate why but the general hypothesis is contradictory orders...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.