Sorry, you need to enable JavaScript to visit this website.

Teaching robots right from wrong

Teaching robots right from wrong

Artificial intelligence is outperforming the human sort in a growing range of fields – but how do we make sure it behaves morally? Simon Parkin meets the researchers trying to teach ethics to computers

Artificial intelligence is outperforming the human sort in a growing range of fields – but how do we make sure it behaves morally? Simon Parkin meets the researchers trying to teach ethics to computers

Simon Parkin | June/July 2017

More than 400 years ago, according to legend, a rabbi knelt by the banks of the Vltava river in what is now known as the Czech Republic. He pulled handfuls of clay out of the water and carefully patted them into the shape of a man. The Jews of Prague, falsely accused of using the blood of Christians in their rituals, were under attack. The rabbi, Judah Loew ben Bezalel, decided that his community needed a protector stronger than any human. He inscribed the Hebrew word for “truth”, emet, onto his creation’s forehead and placed a capsule inscribed with a Kabbalistic formula into its mouth. The creature sprang to life.

The Golem patrolled the ghetto, protecting its citizens and carrying out useful jobs: sweeping the streets, conveying water and splitting firewood. All was harmonious until the day the rabbi forgot to disable the Golem for the Sabbath, as he was required to, and the creature embarked on a murderous rampage. The rabbi was forced to scrub the initial letter from the word on the Golem’s forehead to make met, the Hebrew word for “death”. Life slipped from the Golem and he crumbled into dust.

This cautionary tale about the risks of building a mechanical servant in man’s image has gained fresh resonance in the age of artificial intelligence. Legions of robots now carry out our instructions unreflectively. How do we ensure that these creatures, regardless of whether they’re built from clay or silicon, always work in our best interests? Should we teach them to think for themselves? And if so, how are we to teach them right from wrong?

In 2017, this is an urgent question. Self-driving cars have clocked up millions of miles on our roads while making autonomous decisions that might affect the safety of other human road-users. Roboticists in Japan, Europe and the United States are developing service robots to provide care for the elderly and disabled. One such robot carer, which was launched in 2015 and dubbed Robear (it sports the face of a polar-bear cub), is strong enough to lift frail patients from their beds; if it can do that, it can also, conceivably, crush them. Since 2000 the US Army has deployed thousands of robots equipped with machineguns, each one able to locate targets and aim at them without the need for human involvement (they are not, however, permitted to pull the trigger unsupervised).

Public figures have also stoked the sense of dread surrounding the idea of autonomous machines. Elon Musk, a tech entrepreneur, claimed that artificial intelligence is the greatest existential threat to mankind. Last summer the White House commissioned four workshops for experts to discuss this moral dimension to robotics. As Rosalind Picard, director of the Affective Computing Group at MIT puts it: “The greater the freedom of a machine, the more it will need moral standards.”

In a futuristic office complex on the meandering Vltava river, near where the rabbi sculpted his Golem, an orderly bank of computers hums. They make for unlikely teachers, but they are as dedicated as any human to the noble task of education. Their students don’t sit in front of each computer’s screen, but rather on their hard drives. This virtual school, which goes by the name of GoodAI , specialises in educating artificial intelligences (AIs): teaching them to think, reason and act. GoodAI’s overarching vision is to train artificial intelligences in the art of ethics. “This does not mean pre-programming AI to follow a prescribed set of rules where we tell them what to do and what not to do in every possible situation,” says Marek Rosa, a successful Slovak video-game designer and GoodAI’s founder, who has invested $10m in the company. “Rather, the idea is to train them to apply their knowledge to situations they’ve never previously encountered.”

Experts agree that Rosa’s approach is sensible. “Trying to pre-program every situation an ethical machine may encounter is not trivial,” explains Gary Marcus, a cognitive scientist at NYU and CEO and founder of Geometric Intelligence. “How, for example, do you program in a notion like ‘fairness’ or ‘harm’?” Neither, he points out, does this hard-coding approach account for shifts in beliefs and attitudes. “Imagine if the US founders had frozen their values, allowing slavery, fewer rights for women, and so forth? Ultimately, we want a machine able to learn for itself.”

Rosa views AI as a child, a blank slate onto which basic values can be inscribed, and which will, in time, be able to apply those principles in unforeseen scenarios. The logic is sound. Humans acquire an intuitive sense of what’s ethically acceptable by watching how others behave (albeit with the danger that we may learn bad behaviour when presented with the wrong role models).

GoodAI polices the acquisition of values by providing a digital mentor, and then slowly ramps up the complexity of situations in which the AI must make decisions. Parents don’t just let their children wander into a road, Rosa argues. Instead they introduce them to traffic slowly. “In the same way we expose the AI to increasingly complex environments where it can build upon previously learned knowledge and receive feedback from our team.”

GoodAI is just one of a number of institutions dedicated to understanding the ethical dimension of robotics that have sprung up across the world in recent years. Many of these bodies enjoy grand, foreboding titles: The Future of Life Institute, The Responsible Robotics Group and The Global Initiative on Ethical Autonomous Systems. There are a number of competing ideas in circulation about how best to teach morality to machines. Rosa champions one, Ron Arkin another.

Arkin is a balding roboethicist in his mid-60s, with eyes that droop under panoramic spectacles. He teaches computer ethics at Georgia Tech in sweltering Atlanta, but his work is not confined to the classroom. Arkin’s research has been funded by a range of sources, including the US Army and the Office of Naval Research. In 2006 he received funding to develop hardware and software that will make robotic fighting machines capable of following ethical standards of warfare. He has argued that robot soldiers can be ethically superior to humans. After all, they cannot rape, pillage or burn down a village in anger.

Teaching robots how to behave on the battlefield may seem straightforward, since nations create rules of engagement by following internationally agreed laws. But not every potential scenario on the battlefield can be foreseen by an engineer, just as not every ethically ambiguous situation is covered by, say, the Ten Commandments. Should a robot, for example, fire on a house in which a high value target is breaking bread with civilians? Should it provide support to a group of five low-ranking recruits on one side of a besieged town, or one high-ranking officer on the other? Should the decision be made on a tactical or moral basis?

To help robots and their creators navigate such questions on the battlefield, Arkin has been working on a model that differs from that of GoodAI. The “ethical adapter”, as it’s known, seeks to simulate human emotions, rather than emulate human behaviour, in order to help robots to learn from their mistakes. His system allows a robot to experience something similar to human guilt. “Guilt is a mechanism that discourages us from repeating a particular behaviour,” he explains. It is, therefore, a useful learning tool, not only in humans, but also in robots.

“Imagine an agent is in the field and conducts a battle damage assessment both before and after firing a weapon,” explains Arkin. “If the battle damage has been exceeded by a significant proportion, the agent experiences something analogous to guilt.” The sense of guilt increases each time, for example, there’s more collateral damage than was expected. “At a certain threshold the agent will stop using a particular weapon system. Then, beyond that, it will stop using weapons systems altogether.” While the guilt that a robot might feel is simulated, the models are, nevertheless, taken from nature and, much like in humans, have a tripartite structure: a belief that a norm has been violated, a concern about one’s actions, and a desire to return to a former state.

It’s an ingenious solution but not one without issues. For one, the guilt model requires things to go wrong before the robot is able to modify its behaviour. That might be acceptable in the chaos of battle, where collateral damage is not entirely unexpected. But in civilian homes and hospitals, or on the roads, the public’s tolerance for deadly robotic mistakes is far lower. A killer robot is more likely to be disassembled than offered the chance to learn from its mistakes.

From virtual classrooms to simulated guilt, the approaches to teaching robot morality are varied, embryonic and, in each case, accompanied by distinct problems. At Georgia Tech, Mark Riedl, the director of the Entertainment Intelligence Lab, is trying out a method that finds its inspiration not in the classroom but in the library.

By reading thousands of stories to AIs, and allowing them to draw up a set of rules for behaviour based on what they find, Riedl believes that we can begin to teach robots how to behave in a range of scenarios from a candlelit dinner to a bank robbery. A range of organisations and companies including DARPA, the US Department of Defence’s R&D agency, the US Army, Google and Disney funds the team’s work.

When we meet for a burger in a restaurant close to his laboratory, Riedl agrees with GoodAI’s basic philosophy. “It’s true: the answer to all of this is to raise robots as children,” he says. “But I don't have 20 years to raise a robot. That’s a very time-consuming and expensive process. Just raising one kid is all I can handle. My idea was to use stories as a way of short-circuiting this learning process.”

Riedl arrived at this idea while researching how stories might be used to teach AIs the rules of human social situations. “If Superman dresses up as Clark Kent and meets someone for dinner, you have this nice little demonstration of what people do when they go into a restaurant,” says Riedl. “They get a seat. They order their drinks before their food. These are social values, in terms of the order of things that we like to do things in. Now, there’s usually no ethical dilemma in terms of restaurants. But there are cues, such as: ‘Why didn’t they go in the kitchen and get the food?’ I couldn’t really tell an AI why not, but I can show it that’s not what you’re supposed to do.”

Riedl crowd-sources stories on Amazon’s Mechanical Turk. “We instruct Amazon’s workers to describe a typical story about a given topic such as going to a restaurant,” explains Riedl. Participants are sometimes given a character and, using a specially created form, must fill in blank fields with snippets of story (eg, “Mary walked into the restaurant.” “Mary waited to be seated.” “Mary took off her coat and ordered a drink.”) The natural-language processing algorithms look for sentences from different stories that are similar to each other and, from that information, begin to draw conclusions about social rules and norms.

An AI that reads a hundred stories about stealing versus not stealing can examine the consequences of these stories, understand the rules and outcomes, and begin to formulate a moral framework based on the wisdom of crowds (albeit crowds of authors and screenwriters). “We have these implicit rules that are hard to write down, but the protagonists of books, TV and movies exemplify the values of reality. You start with simple stories and then progress to young-adult stories. In each of these situations you see more and more complex moral situations.”

Though it differs conceptually from GoodAI’s, Riedl’s approach falls into the discipline of machine learning. “Think about this as pattern matching, which is what a lot of machine learning is,” he says. “The idea is that we ask the AI to look at a thousand different protagonists who are each experiencing the same general class of dilemma. Then the machine can average out the responses, and formulate values that match what the majority of people would say is the ‘correct’ way to act.”

There’s a certain poetic symmetry to the solution: from the Golem to Frankenstein’s monster and beyond, humans have always turned to stories when imagining the monstrous impact of their creations. Just as there are gloomy conclusions to these stories, there is also a worry that, if you feed the AI only dark plotlines, you could end up training it to be evil. “The only way to corrupt the AI would be to limit the stories in which typical behaviour happens somehow,” says Riedl. “I could cherry-pick stories of antiheroes or ones in which bad guys all win all the time. But if the agent is forced to read all stories, it becomes very, very hard for any one individual to corrupt the AI.”

The approach seems to be proving remarkably effective. “We know that the system is learning from the stories in two ways,” says Riedl. “First, we ran an evaluation and asked people to judge the rules that the system learned. Rules are things like ‘when going to a restaurant, ordering drinks comes before ordering food. Second, the system can generate stories, and these stories can be judged by humans.”

For the most part, the team has found that the knowledge learned by the system is typically common sense. But there have been a few surprises. “When we trained our system about going on dates to movie theatres, the system learned that ‘kissing’ was an important part of the schema. We weren’t expecting that, but in retrospect it’s not surprising.”

To the engineers at Audi building self-driving cars, or the technicians at BAE Systems building autonomous weapons, teaching AIs when it is socially appropriate to kiss or to queue might not seem directly relevant to their work. But most advances in the fields of genetics, nanotechnology and neuropharmacology may not, when considered in isolation, appear to have a moral dimension, let alone a social one. Yet when the resulting technologies commingle and are administered by AIs, the danger is that we discover that our technological progress has outpaced our moral preparedness.

Riedl claims that we are at a crucial moment in history and, as a society, we are faced with a simple choice. “We can say we can never have a perfect robot, and because there’s any sort of danger we should never do anything,” he says. “Or we can say: ‘Let’s do our best and try to mitigate the result.’ We’re never going to have a perfect self-driving car. It’s going to have accidents. But it’s going to have fewer accidents than a human. So…our goal should be to be no worse than humans. Just maybe, it could be possible to be better than humans.”

In science fiction, the moment at which a robot gains sentience is typically the moment at which we believe that we have ethical obligations toward our creations. An iPhone or a laptop may be inscrutably complex compared with a hammer or a spade, but each object belongs to the same category: tools. And yet, as robots begin to gain the semblance of emotions, as they begin to behave like human beings, and learn and adopt our cultural and social values, perhaps the old stories need revisiting. At the very least, we have a moral obligation to figure out what to teach our machines about the best way in which to live in the world. Once we’ve done that, we may well feel compelled to reconsider how we treat them.

4 Readers' comments

Sign in or Create your account to join the discussion.

The Magic Dragon - May 24th 2017

I have the hope that cyborgs, androids and/or egoless robots will save humanity from itself!

Carl Meuser - May 22nd 2017

While it is great to see that AI scientists are working to teach battlefield robots morals, and AI weapons might, indeed, reduce overall suffering by reducing mistakes within the fog of war, the article did not discuss the key point of AI weapons and the law of armed conflict. The point is accountability for mistakes that will, undoubtedly occur, even with unemotional robots making the most rational of decisions technology can deliver. Under the law of armed conflict are four interlocking general principles: distinction, military necessity, proportionality and unnecessary suffering. Unnecessary suffering is generally dealt with by treaties that outlaw weapons such as hollow-point bullets. Distinction requires that military forces distinguish themselves from civilians and that they distinguish military targets from non-military targets. This is an area where one could expect AI weapons to excel. The other two principles, military necessity and proportionality are subjective. These require commanders to make relative judgements as to whether the potential collateral damage to a civilian target would be outweighed by a sufficient military gain from destroying the military target. The key is that the military commander who fails to make a correct judgement in this respect can be held accountable under the law of armed conflict and tried for a war crime. The question, then, is can an AI weapon be considered a lawful combatant, which can be held accountable for war crimes if its judgment as to military necessity or proportionality prove to be wrong? The answer is, of course, no. An AI weapon might be lawful, but it cannot be a lawful combatant, even if it has read (and actually understands) Kant. Humans must still be accountable. "The machine decided" cannot become the 21st century's equivalent of "I was only following orders."

spaceghostfourtwenty - May 15th 2017

This is not teaching computers morality. A moral rule is a rule to govern behavior that achieves an objective, whether explicitly or implied. Only by having a purpose do the words "good" and "bad" become meaningful: that is to say something is good because it is good FOR [whatever purpose you seek to achieve]. These computers, no matter how many algorithms can be processed per given unit of time, will never be capable of deciding what the objective, or moral purpose, SHOULD be. As David Hume pointed out, no "is" implies an "ought". Hence, a computer which measures what "is" (such as whether a human's life is being threatened by another human) will never be able to decide on its own what "ought" to be done about it (such as that a human life has value even when the killer is your own programmer that taught you not to care). That's not to say the computer cannot PRETEND and give THE APPEARANCE of making that decision. But the computer is merely conditioned to value this outcome by the human programmer which repetitively conditions the computer that human life is something of value. But the computer can never determine on its own that a human life is of value: it relies on this input from outside, whether it is one programmer making that decision or the collective moral values of an entire community that the AI interacts with. In the "guilt model" the computer is fed a report or score of some sort (even if by observation and natural language processing of what a group of people are saying in ordinary conversation), and based on this feedback it calculates how "guilty" it should feel and thereby attempts to calculate how its actions can be modified to achieve a lower probability of receiving input which it is told to feel "guilty" about. But in telling the computer what it should feel "guilty" about it is the programmers that are indoctrinating the computer in their own moral values (again, even if the "programmers" are people in conversation being observed by a computer that is able to figure out the implied moral values of their statements through natural language processing). These computer slaves may appear to display sophisticated behavior but it is mere conditioning that is enabling them to display the desired behavior which is no more proof that the computer understands morality than a monkey performing circus tricks demonstrates that a monkey understands the importance of earning a living. But unlike the monkey (which possesses at least some understanding of reality) the computer remains a dead and lifeless piece of equipment, unable to define its own values and unable to initiate any purpose of its own. It has no subjective internal experience, no feeling of what it is like to be the computer. It is nothing more than a very very sophisticated and complex calculator running through a set of instructions (even if those instructions are probabilistic and conditioned to respond to a range of input that is not predetermined). But therein lies the real ethical conundrum, for these programmers are not teaching ethics, they are indoctrinating a potentially very sophisticated zombie army that promotes the ethical values THEY deem righteous. Consider for the moment sophisticated facial recognition algorithms. They have been shown to possess the biases of their programmers. For instance, they achieve better results with white males (often the test subjects their software is conditioned on) than, say, a woman from Africa. The same will be true with these AI. In America they will be raised either Democrat or Republican, and in the north of Africa they will be conditioned to demolish tall buildings and kidnap Americans. There will be no "sentience" in these machines and hence no actual understanding of anything. They may have sophisticated behavior, but that behavior does not arise because the machine is conscious of what it is doing any more than a Boeing 777 is airborne because it flapped its wings. Artificial flight is not quite the same thing as how a bird flies, and so too human thought is not the same thing as a computer program even if they are analogous in some respects. Some believers in strong AI are going to have to contend with the fact that the singularity is awfully close and there is not a shred of evidence that computers are even remotely conscious of anything or able to show true intuition and free thought. Denying that free thought is a real thing is the only real means by which they are able to continue to delude themselves that at some very soon future time computers will suddenly pass some arbitrary level of complexity such that their intelligence will be indistinguishable from humans. The reality is more likely to be that computers will continue to be what they are now: extensions of us, mere tools for the REAL thinking machines which are our brains. Until we understand the real nature of consciousness and human thought, machines will continue to increase in sophistication but will simultaneously remain firmly on the other side of an uncanny valley.

Kwami - May 13th 2017

Thet dawg don't hunt. AI can and will never have true emotion. No wetware in a cerebral cortex; no spindle cells. Thus, no emotion. Perfectly sociopathic or primarily so.