Real-world Ethics for Self-Driving Cars
ICSE 2020 Poster

Real-world Ethics for Self-Driving Cars

Interactive Poster Presentation by Tobias Holstein

If your web-browser, does not support JavaScript, you can look at the slides below.

 
 
 
 
 
 
 
 
 
 
 
 
Poster Presentation Slides

Real-world Ethics for Self-Driving Cars

Presentation for the Virtual ICSE Poster Session

Tobias Holstein,
Gordana Dodig-Crnkovic,
Patrizio Pelliccione

Cars over the last decades

Cars over the last decades: starting with a 3 wheel car in the beginning of 1900 to a Tesla in 2017.

Complex driving tasks are successively replaced by
advanced driving assistant systems.

With Self-Driving Cars into the Future

Different Self-Driving Cars from Google/Waymo and General Motors

No Steering-Wheel or other primary driving controls,
the former driver becomes solely passenger.

The (Trolley-)Problem

The Trolley Problem is an unsolvable thought experiment and often presented as a key issue for self-driving cars.

Depiction of the Trolley Problem Scenario - Source: The New York Times; Illustration by Frank O’Connell
Source: The New York Times; Illustration by Frank O’Connell

The question often posed in context of self-driving cars is:
whom will the Self-Driving Car kill, when it has to decide?.

Irrelevance of the Trolley Problem - Aspect 1

Currently, there are no fully autonomous cars and humans are still responsible for the decision making. The process of increasing automation goes via step-wise improved driving performance of a car based on machine learning.

Two recent studies that compare crash experiences of automated vs. conventional vehicles show that automated vehicles perform better.

The behaviour of the car is based on learning from experience and not on programming predefined hypothetical scenarios.

Irrelevance of the Trolley Problem - Aspect 2

The assumption made in the Trolley Problem about the deterministic nature of all the involved processes is wrong.

It means that all the objects have perfectly known positions from which only one perfectly calculable consequence will follow.

In the real world, we have a complex system under uncertainty that is not possible to predict exactly in real-time, and the way those phenomena are handled is by machine learning.

Learning from real-world driving experiences leads to improved data, which constantly improve the capabilities of the self-driving cars.

Irrelevance of the Trolley Problem - Aspect 3

The Moral Machine experiment, asking people all over the world about what they would do in a Trolley Problem situation, is often mentioned in connection to autonomous cars, in spite of the fact that it is about people and not about self-driving cars.

It is not a way to understand what cars should do, as humans are known to be the main cause of car accidents. According to The National Highway Traffic Safety Administration, 94% of serious crashes are due to human errors.

Irrelevance of the Trolley Problem - Conclusion

Trolley Problem scenarios explore differences among people in what they believe they would do in certain traffic situations.

But, the relevance for the development of Self-Driving Cars lies in the techno-social and ethical aspects of real-world engineering.

Our Approach

Instead of pointing towards the unsolvable Trolley Problem,
we use a hybrid interdisciplinary methodology to identify
relevant ethical and societal challenges
for the development of self-driving cars.

Combining literature on value-based design and regulations, guidelines and standards, with the technical characteristics of present-day automated cars and their anticipated developments, allows to extract a list of most important topics.

Our Results

We identified ...

technical challenges and their manifestations grouped by requirements: safety, security, privacy, trust, transparency, reliability, responsibility, and accountability, quality assurance, and sustainability.

social challenges and their manifestations grouped by requirements: social challenges of disruptive technology, stakeholders/general public interests, and legislation, norms, policies, and standards.

The list and given examples can be used for discussion with experts, stakeholders and for further validation studies.

Developing Self-Driving Cars is an Iterative Process

Self-driving cars development through the iterative process involving technology R&D, society, laws, and manufacturing.

Example of a Technical Challenge

Sensors and Recognition software, aim to detect objects (cars, buildings, etc.) and living beings (cyclists, pedestrians, etc.).

There are different stages of recognition:
3 examples of recognition: just different sized objects (moving or not moving), objects versus persons, and detailed information about objects or persons.

In the last stage, if it were technically possible, there is a privacy problem.
This specific problem is for example solved in Europe by the GDPR.

Example of a Technical Challenge

By analyzing photos of pedestrians, for example,
a neural network can learn to identify a pedestrian.

Challenge of recognizing extraordinary objects, person in darth vader costum and a person with a tree costum are depicted as an example.

How safe, reliable, or precise must a recognition be? In other words, how good is good enough?

Conclusion

It is of key importance to include ethical thinking and reasoning into the design and development process in every phase from requirements, till testing, maintenance, and evolution.

Architectural and design decisions should be taken through a process that includes ethics,
for example as a required non-functional requirement.

Transparency will be key to be able to observe and evaluate processes and software independently.

More Information and Resources

You can find a list of references and papers
on our project page: https://ethics.se