You are moving along the highway, when suddenly a man runs onto a busy road. Cars are moving around you, and you have a split second to decide: try to drive around a person and create a risk of an accident? Continue in the hope that he will have time? To brake? How would you rate the odds if a child is fastened in your back seat? In many ways, this is the classic “moral dilemma”, the problem of the trolley. She has a million very different options that allow us to reveal human prejudice, but the essence is the same.
You are in a situation where life and death are at stake, there is no simple choice, and your decision will, in essence, determine who will live and who will die.
Trolley dilemma and artificial intelligence
New work MIT, published last week in the journal Nature, is trying to come up with a working solution to the problem of the trolley, attracting millions of volunteers. The experiment began in 2014 and was completely successful, having received more than 40 million responses from 233 countries, which makes it one of the largest moral studies conducted.
A person can make such decisions unconsciously. It is difficult to weigh all the ethical systems and moral prerequisites when your car is racing along the road. But in our world, decisions are increasingly being made by algorithms, and computers can easily react faster than us.
Hypothetical situations with self-driving cars are not the only moral decisions to be made to algorithms. Medical algorithms will choose who gets treated with limited resources. Automated drones will choose how much “collateral damage” is permissible in a separate military encounter.
Not all moral principles are equal.
The “solutions” to the problem of the trolley are as varied as the problems themselves. How will machines make moral decisions when the foundations of morality and ethics are not universally accepted and may not have decisions? To determine whether the algorithm is correct or incorrect?
The crowdsourcing approach adopted by the Moral Machine scientists is quite pragmatic. In the end, in order for the public to adopt self-driving cars, it must take the moral foundation behind their decisions. It will not be very good if ethicists or lawyers come to a decision that is unacceptable or unacceptable for ordinary drivers.
The results lead to the curious conclusion that moral priorities (and therefore algorithmic decisions that can be made by people) depend on where in the world you are.
First of all, scientists recognize that it is impossible to know the frequency or nature of these situations in real life. Wrecked people very often cannot say what exactly happened, and the range of possible situations rules out a simple classification. Therefore, in order to track down the problem, it has to be broken down into simplified scenarios, to look for universal moral rules and principles.
When you pass a survey, you are offered thirteen questions that require a simple choice: yes or no, trying to narrow the answers to nine factors.
Should the car turn into another lane or keep moving? Should you save young people, not old ones? Women or men? Animals or people? Should you try to save as many lives as possible, or is one child “worth” two elderly? Save passengers in the car, not pedestrians? Those who cross the road not by the rules, or those by the rules? Should you save people who are physically stronger? What about people with higher social status, such as doctors or businessmen?
In this harsh hypothetical world, someone must die, and you will answer each of these questions with varying degrees of enthusiasm. However, making these decisions also reveals deeply rooted cultural norms and prejudices.
Processing a huge set of data obtained by scientists during the survey, gives universal rules, as well as interesting exceptions. The three most prevailing factors, averaged over the entire population, were expressed in the fact that everyone preferred to save more lives than fewer people, not animals, and young, not old ones.
You may agree with these points, but the deeper you think about them, the more disturbing will be the moral conclusions. More respondents preferred to save the criminal instead of the cat, but in general they preferred to save the dog, not the criminal. On average, the world being old is rated higher than being homeless, but homeless people were saved less often than fatty people.
And these rules were not universal: respondents from France, Great Britain and the United States preferred the young, while respondents from China and Taiwan were more willing to save the elderly. Respondents from Japan preferred to save pedestrians, not passengers, while in China they prefer passengers to pedestrians.
The researchers found that they could group the responses by country into three categories: “The West”, mainly North America and Europe, where morality is based mainly on Christian doctrine; “East” – Japan, Taiwan, the Middle East, dominated by Confucianism and Islam; “Southern” countries, including Central and South America, along with strong French cultural influence. In the Southern segment, stronger preferences to sacrifice women than anywhere else. In the Eastern segment, there is a greater tendency to save young people.
Filtering on different attributes of the respondent gives endless interesting options. “Very religious” respondents are less likely to prefer saving an animal, but both religious and non-religious respondents express an approximately equal preference for saving people with high social status (although it can be said that this contradicts some religious doctrines). Men and women prefer to save women, but men are less inclined to this.
No one argues that this research somehow “solves” all these weighty moral problems. The study authors note that crowdsourcing online data does include bias. But even with a large sample size, the number of questions was limited. What happens if the risks change depending on your decision? What if the algorithm can figure out that you only had a 50 percent chance of killing pedestrians, given the speed at which you moved?
Edmond Awad, one of the authors of the study, expressed caution regarding over-interpretation of the results. The discussion, in his opinion, should flow into the risk analysis – who is more or less at risk – instead of deciding who will die and who will not.
But the most important result of the study was the discussion that broke out on his soil. As the algorithms begin to make more and more important decisions affecting people’s lives, it is imperative that we have a constant discussion of the ethics of AI. Designing an “artificial conscience” should include the opinion of everyone. Although the answers will not always be easy to find, it is better to try to form a moral framework for the algorithms, not allowing the algorithms to independently form the world without human control.