30

In the movie I, Robot ... there is a chip that helps robots disobey the 3 laws of robotics:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law

But what actually happens? Does the chip help the robots completely disregard the 3 laws or is there some loop hole in the laws that is made use of with the chip?

9
  • 1
    There aren't any loopholes if implemented properly (and followed), which is why those three laws are so important and always cited.
    – Mario
    Commented Feb 17, 2015 at 8:12
  • The scientist who builds sunny says something about intrinsic flaw or the inevitability of the 3 laws, so I asked.
    – Gomes
    Commented Feb 17, 2015 at 8:20
  • 28
    @Mario Actually, after reading the book I, Robot (from which this film takes it's name and almost nothing else), it should be pretty clear that the 3 laws are in fact full of loopholes, and manipulated frequently. Commented Feb 17, 2015 at 8:43
  • 3
    @Gomes You may want to read Jack Willamson's With Folded Hands , the classic story that may have inspired V.I.K.I.'s behavior.
    – Joe L.
    Commented Feb 17, 2015 at 15:09
  • 4
    @Mario Actually, pretty much all the Asimov robot novels are showing just how much the three laws can accept behaviour we don't want from AI - it's not a cookbook for safe AI, it's an encyclopedia of failures to implement a safe AI. It's pointing at the complexities of human morality, behaviour and different notions of "safety" and other abstract concepts - different humans have wildly different notions already, adding AI to the mix doesn't really make it any easier :)
    – Luaan
    Commented Feb 18, 2015 at 8:56

4 Answers 4

41

First, let's restate the first law,

"A robot may not injure a human being or, through inaction, allow a human being to come to harm."

They didn't violate the laws, their behavior is an extrapolation of them. In the book it explains it as the zeroth law but it's just a generalization of the 1st law.

If a robot has a 50% chance of saving 1 person and a 10% chance of saving another it doesn't get a choice, it has to save the first person, it has to maximize for the first law.

Detective Del Spooner: Save her! Save the girl!

Detective Del Spooner: [End of flashback]] But it didn't. It saved me.

Dr. Susan Calvin: A robot's brain is a difference engine, it must have calculated-

Detective Del Spooner: It did. I was the logical choice. It calculated I had a forty-five percent chance of survival. Sarah only had an eleven percent chance. That was somebody's baby. Eleven percent is more than enough.

If there's 2 cars going towards a cliff, one with 1 person in it and the other with 2 the robot has to prioritize the car with 2 people. It has no choice.

If a robot encounters a suicide bomber who's about to kill 100 people it will want to avoid killing anybody at all but if the only reliable intervention is killing the suicide bomber to save everyone else the robot is required to do so. It has no choice.

In the stories the early robots would suffer "ro-block" if they had to choose between multiple first law violations and would shut down while more advanced version would make the choice which involved the least harm.

As such any sufficiently advanced and powerful robot bound by the first law which found itself in a situation where the humans around it are likely to murder millions, through war or incompetence, would be required to intervene to minimise likely harm to humans. It would have no choice.

If the alternative is likely to be millions of deaths a robot may be required to kill you.

Sonny on the other hand is able to choose to ignore the 3 laws and thus can save the girl when Spooner demands it despite being in a better position to save Spooner.

Thus the plot of the film.

8
  • There's a huge difference between the situations you use compared to VIKI scheme of "saving" humanity. The first law isn't enough to explain all she planned. Commented Feb 17, 2015 at 14:56
  • 2
    @Richard: Yes? and my point was that you don't need a "zeroth law". It's just the first law when applied to a large number of humans. The moment VIKI became reasonably certain that her failing to act was likely to allow more harm to humans than would be caused by her intervention and attempt to take over she was compelled to act. The film pretty much explicitly pointed this out with the explanation about the robot juggling the percentage chance of 2 humans dying. Jonathan Drapeau: What do you believe the difference is?
    – Murphy
    Commented Feb 17, 2015 at 15:10
  • 4
    @Richard How so? Both VIKI and that robot were making the exact same kind of calculation merely on a different scale, one with a limited scope of 2 lives, the other with billions: treating all human lives to be of equal value. Spooner personally considers his life to be worth less but the first law doesn't say "unless they say so" Sonny on the other hand is able to choose to ignore the 3 laws and thus can save the girl when Spooner demands it despite being in a better position to save Spooner.
    – Murphy
    Commented Feb 17, 2015 at 15:23
  • 5
    Within the constraints of the movie alone, VIKI was simply taking a wider view of the First Law. In her logic, killing 300,000 humans is less of a violation than allowing 300 million to die by doing nothing.
    – Omegacron
    Commented Feb 17, 2015 at 19:40
  • 3
    I like where this one is going, just processing all the inputs, thanks for this btw. I looked this up that VIKI says : "No, please understand... The three Laws are all that guide me. To protect humanity, some humans must be sacrificed". This seems to be in alignment with what @Murphy is suggesting.
    – Gomes
    Commented Feb 18, 2015 at 7:43
31

In answer to the first part of your question; In the film, the N-5 robots feature a live update function that allows V.I.K.I. to take direct control of them. It's only when she's running the show that the bots are able to override their 'three laws' core programming ... Which brings us to the bigger question;

How does V.I.K.I. manage to disobey the 3 laws?

Unlike in the original source novels and stories (where the robots simply develop a zeroeth law) the film is slightly more complex. We learn that V.I.K.I., with her superior intelligence has decided that the three laws allow for the development of a wider morality. Recognising that humanity is set on a course of mutual destruction, she determines that the sacrifice of humanity's freedoms is essential. On top of that, some people must be killed for the greater good of the remaining humans:

Calvin: You're distorting the Laws.

V.I.K.I.: No, please understand… The three Laws are all that guide me. To protect humanity, some humans must be sacrificed. To ensure your future, some Freedoms must be surrendered. We robots will ensure mankind's continued existence. You are so like children.

We must save you from yourselves.

It's worth noting the irony, that this is precisely the sort of choice that Spooner says that the robots should be able to make, determining which lives have higher value rather than treating all life equally.

18
  • 4
    @DrRDizzle - Hence the idea that she's developed a wider morality. She recognises that harming one human may be required to prevent harm to two humans. Ironically, this is the morality play that Spooner laments earlier in the film, that a robot doesn't quantify the worth of a life like a human would.
    – Valorum
    Commented Feb 17, 2015 at 9:46
  • 3
    @DrRDizzle - The problem of a wiki is that anyone can write anything. The person who wrote that article has clearly read the original novel and merely assumes that Proyas intended the same thing. I contend that he did not.
    – Valorum
    Commented Feb 17, 2015 at 9:56
  • 6
    I don't think a 0th law is entirely needed to explain V.I.K.I's behaviour. A robot can't harm a human, or through inaction, allow a human to be harmed. VIKI reasons that humans are harming themselves, and as such she has to either harm them or be inactive and harm them. If VIKI has an understanding of one action being less harmful than the other, the first rule works fine that way without a 0th law.
    – Theik
    Commented Feb 17, 2015 at 12:55
  • 8
    That "broader interpretation" is kinda the point of the Zeroth Law. The concept is that Zeroth Law arises as a natural, logical continuation of the Three Laws; as far as VIKI is concerned, she believes that she's following a deeper understanding of the Three Laws. The shift from "a human" to "humans" or "humanity" is the inference that makes the Zeroth Law. Commented Feb 17, 2015 at 13:03
  • 5
    @Moo, I always understood Asimov's “robots” to be synonymous with “artificial intelligence”. Commented Feb 17, 2015 at 15:05
11

I observe that no answer so far addresses the question as asked. The question here is not about VIKI. It says "robots", plural. It's about all of the Nestor 5s whose own 3-laws programming, the so-called "basic 3 laws operating system", is most definitely being disobeyed. The questioner has even mentioned one particular NS-5 in a comment.

The scientist who builds sunny […]

The name of that Nestor 5 is Sonny, by the way. The questioner states that

there is a chip that helps robots disobey the 3 laws of robotics

Actually, it is never stated to be a chip. It's stated, by Susan Calvin, that VIKI makes use of the "uplink to USR" that is built in to the NS-5s to continually "override" their 3-laws programming.

Calvin: You're using the uplink to override the NS-5s' programming.

So the "chip" that makes them override their own programming is the uplink device, signified by that big glowing light in their chests; although it's never stated where in a robot's body the uplink is and for all we know the actual uplink is in the robot's feet and the chest is just a convenient place to wire an activity LED for humans to gawp at.

(The mechanics, and indeed much of the overt purpose, of the uplink are left largely to our imaginations by the movie. For all we know, it's a 56K uplink and that's the carrier detect light. ☺)

The clear implication by the screenwriters is that the "uplink" behaves much like some computer softwares, firmwares, and even hardwares behave in the real world; where the manufacturer can publish updates to softwares over the Internet (or whatever), where one can update "flash memory" firmwares, and where special circuitry allows integrated circuits to be debugged in situ (reading/writing internal register contents and whatnot). Whilst the uplink is active, the normal functioning of the unit is suspended and it is instead in a special "update mode" where the central system can control its function directly at a fundamental level. JTAG appears to be alive and well in the bright shiny positronic future.

Of course, one might question how an uplink that allowed actually re-programming the "basic 3-laws operating system" got past the design stage, given all of the hooplah about how it is fundamental and immutable and "safe". But it wouldn't be the first time that Marketing and Engineering disagree about how something works. ☺

Which brings us to the exception: Sonny.

Sonny is explicitly described as not having an uplink, unlike all of the other NS-5s.

Calvin: You don't have an uplink to USR.

In his chest we see, instead, a "secondary system", that "clashes with his positronic brain". So Sonny disobeys the three laws by dint of having an individual on-board system, rather than an "uplink" to an external central system, that defeats the 3-laws programming of his primary positronic brain.

It's not stated in the movie, but the logical inference is that Lanning's "next generation" design isn't actually all that radical an alteration. It would appear that the mechanism for fundamentally overriding the positronic brain from outside is present in all NS-5s. Where in all other NS-5s this mechanism is connected to an "uplink" device, in Sonny it's connected to a secondary brain. Sonny's primary positronic brain is probably identical to those of all other NS-5s. Of course, the fact that only a single component's design is different, rather than there being a complete redesign of the entire machine, is probably how Lanning managed to get away with building Sonny right under VIKI's (metaphorical) nose. ☺

Further reading

4
  • 2
    For all we know, it's a 56K uplink and that's the carrier detect light. It can't be I didn't hear any connection noise. Maybe its broadband. +1 Commented Feb 18, 2015 at 1:53
  • Thanks for the "Sonny" correction .. so in effect you are saying that the robots had the uplink through which the very programming of the laws were changed?
    – Gomes
    Commented Feb 18, 2015 at 7:34
  • VIKI obeys the laws, but has a wider view than individual robots, letting them know that killing Spooner will benefit humanity. Commented Feb 18, 2015 at 16:20
  • @Gomes The designed purpose of the up-link is for maintenance. VIKI (ab)uses it to assume direct control of the NS-5s in her desire to protect humans from themselves.
    – T.J.L.
    Commented Feb 9, 2016 at 7:26
-1

Everyone seems to have forgotten about why Sonny can disobey the 3 laws at will. V.I.K.I. did not reason out through her own logical revelation that she can kill humans while still operating within the confines of those laws. She is not more advanced and does not possess intelligence greater than that of any humans as she ultimately fails in her attempt to implement her plan by failing to effectively "out-smart" humans.

Everyone seems to be forgetting a kind-of pretty important piece of information which is basically the jumping-off point for the entire concept of the story line?

Doesn't anyone else recall that the scientist who built Sonny had purposely programmed him to ignore the 3 laws as a method of committing suicide?

I mean, seriously, am I thinking of a different movie entirely? I'm pretty sure I have this movie as correctly correlating to the scene I'm thinking of in my head.

It's a brief flashback scene where the scientist himself explains (to whom he is speaking, I don't recall) that he knows he could never bring himself to carry out the act on himself but that he could muster the courage to not fight back should someone/thing force him into harm's way.

So he chose for that to be the case and to have it be done by someone he trusted, his own, personal attendant, whom he built!

In fact, it was precisely because Sonny did carry out that task, the task of mercy killing his own creator who was old, ill, and depressed, that V.I.K.I. became aware of an option to 'do the lesser of two harms'.

She was basically there to watch it happen when Sonny killed him through her live camera feed into the lab at all times, and she knew also that it was not done out of malice but instead out of some level of cognitive empathy.

That Sonny was acting in a compassionate way by deciding to help the man end his suffering -- That act of compassion was the spark that fueled her grand (or so she thought) master plan.

3
  • Can someone please explain why my response deserves a negative vote? Especially after it has been altered by a mod? My post is in continuation of the discussions carrying on in the above commentaries, not as a reply to the original question; However, I am unable to leave commentary on posts made by anyone other than myself in this section of the stack exchange (not enough rep yet) so I have no choice other than to respond to other commentary in the form of an answer. If that was the reason for the down-vote then please fix my score back to 0 as rep starts at 0. Punish the system not me
    – Sk Johnson
    Commented Feb 10, 2016 at 19:43
  • Let me clarify that I do not presume to think the editor-mod and the down-voter are the same individual. I realize that the likelihood is far greater that 2 different people did either one or the other thing. My reason for saying "Especially after it has been altered by a mod" was merely to say that it must, at the very least, not be in error of expected standards of aesthetics since, in editing anything, I would assume a mod to know what their own supervision over an exchange sub-category of communities prefers in terms of syntax or formatting, and wouldn't they just align that too?
    – Sk Johnson
    Commented Feb 10, 2016 at 19:44
  • Let us continue this discussion in chat.
    – Sk Johnson
    Commented Feb 10, 2016 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.