Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Ava Thomas Wright
Postdoctoral Research Fellow in AI and Data Ethics here at Northeastern University
JD, MS (Artificial Intelligence), PhD (Philosophy)
I am here today to talk about “Value Sensitive Design” in AI systems. The goal of Value Sensitive Design is to make socially-informed and thoughtful value-based choices in the technology design process
Appreciating that technology design is a value-laden practice
Recognizing the value-relevant choice points in the design process
Identifying and analyzing the values at issue in particular design choices
Reflecting on those values and how they can or should inform technology design
AI Ethics (I): Value-
embeddedness in
AI system design
Group Activity: Moral Machine http://moralmachine.mit.edu/
Students will work through the scenarios presented in moral
machine as a group and decide which option to choose.
The instructor might ask students to discuss as a group
which choice they should make and then decide by vote.
In a larger class, students might break into small groups and
work through the activity together. It is important that the
group make a choice rather than have everyone do it on
their own to highlight an important point in the lesson plan.
It is also important to show what happens once all the
cases are decided: MM outputs which factors the user
takes to be morally relevant and to what extent.
I. Descriptive
vs. Prescriptive
(Normative)
Claims
Set-up: Review the findings on popular ethical preferences in the MM paper in Nature (see, for example, Figure 2)
The distinction between descriptive and
prescriptive ethical questions:
Descriptive: How do people think AVs should
behave in accident scenarios? (describes
what people's preferences are)
Prescriptive: How shouldAVs behave in accident
scenarios? (prescribeswhat AVs should do, or
what AV system designers should do)
Some descriptive and prescriptive questions the MM experiment raises:
Descriptive:
•Does the MM platform accurately capture people's preferences about how AVs should behave in accident scenarios?
•Can the MM platform help its users clarify how they reason about how AVs should behave?
Prescriptive:
•Should designers use the moral machine platform to make decisions about how to program autonomous vehicles to behave in accident scenarios?
•How should designers determine how to program AVs to behave in accident scenarios?
•When (if ever) should designers use surveys of ethical preferences to decide how to program autonomous systems such as AVs?
Group Discussion
Answer the prescriptive and descriptive questions just
raised. This serves to set up the rest of the lesson plan.
Suggestions
10 minutes: Have students break into small groups to try to
answer these questions
5 minutes: Have students write down their individual answers
10 minutes: Have a general group discussion about people’s
answers to these questions
Aims of Discussion
Dependence relationships between the questions:
If MM is a bad descriptive tool, then we shouldn’t look to it
to answer moral questions
Even if MM is a good descriptive tool, nothing immediately
follows from that about the answer to prescriptive questions
about what you ought to do (sometimes referred to loosely
as the "is-ought" gap in moral theory).
The majority's preferences might be unethical or unjust
Examples: Nazi Germany; antebellum South. Or consider a
society of cannibals guided by the consensus ethical rule,
"Murder is morally permissible so long as one intends to eat
one's victim."
The MM thus makes two implicit claims
about AV system design:
descriptive claim: The MM platform does accurately capture people's ethical preferences about how an AV should behave in accident scenarios.
prescriptive claim: AVs should be programmed to act in accordance with the majority's preferences as collected by the MM platform.
Take a 5-
minute
break?
II. Challenges
for the
Descriptive
claim
Descriptive Claim: The MM platform is a good tool for accurately capturing people's ethical preferences about how an AV should behave in accident scenarios.
If the MM platform is not a good tool for
accurately capturing people's ethical
preferences about how an AV should
behave in accident scenarios., then it
should not be used as a tool for answering
prescriptive questions about how to
program autonomous vehicles.
Even if you think you should encode the
majority's preferences. you first have to
make sure to get them right!
Issues in the collection
of data
1) Representativeness
of sample
There are
few
controls on
data
collection
in MM:
For example, Is the data from our class representative of any individual user or even of the group?
Users might not take it seriously
There are no instructions letting the user know that this data might be used for the programming of AVs
The people answering questions on the MM website may not be representative of everyone
Users cannot register indifference
Potential response: With enough data, we can ignore the noise that results from the above
Issue: But we need to know a lot more about how much
noise is introduced
2) Implicit valueassumptions or blindspots in data collection practices
Some ethical features of accident scenarios in MM were selected for testing, but not others. Why?
For example, MM does not gather people's preferences
with regard to race, ethnicity, apparent LGBT status, etc.
Many other features that might have influenced results
could have been tested as well.
Potential response: Perhaps MM should disqualify discriminatory ethical preferences, if they exist.
Issue: But MM tests ethical preferences with regard to gender and
age.
Designing the experiment to capture some preferences that may be
discriminatory but not others is a normative decision that requires an
explanation and ethical justification.
III. Big-Picture
Takeaways
General
Data
Collection
Concerns
Data comes from somewhere and the quality and care
taken when collecting it will determine whether the
resulting data is useful. Data that is poorly constructed
can undermine programmers’ ability to design systems
ethically.
Other disciplines might be needed to help understand
or vet data. In the case of MM, a social scientist might
be able to tell us what kinds of results are significant
even with lots of noise. They might also tell us what sorts
of controls are needed.
Tools or practices for collecting data may be implicitly biased or contain unexamined ethical value assumptions
A more diverse design team might help reveal
blindspots or surface implicit ethical assumptions so that
they can be examined.
Such problems do not apply only when the data
collected is data concerning people's ethical
preferences.
For example, suppose a hospital with a history of
intentionally discriminating against the hiring of female
doctors naively uses its own historical data on the traits of
successful hires to train a machine learning system to
identify high-quality job applicants. The (perhaps unwitting)
result would be a sexist algorithm.
We will discuss this more in AI Ethics II module
Design of system may have hidden value assumptions
Even if there is some version of MM that provides reliable
information about users’ ethical preferences, the implicit
proposal that we should rely on such data to inform how
we should develop AVs is a (controversial) prescriptive
claim that requires defense.
Arguably this is the main issue with the MM platform and is
the topic of the next class.
Review Questions
What is the difference between a descriptive and a prescriptive claim? (the is-ought gap)
What are the main descriptive and prescriptive claims made in the MM platform? What is the logical relationship between them?
Describe some issues with how data on people’s ethical preferences was collected in MM.
Should designers program autonomous systems such as AVs to act in accordance with the ethical preferences of a majority of people as revealed by platforms like the MM? (Q for next time)
Rightful Machines
A rightful machine is an explicitly moral autonomous system that respects principles of justice and the public law of a legitimate state.
Efforts to build such systems must focus first on duties of right, or justice, which take normative priority over contestable duties of ethics in cases of conflict. (This insight resolves the “trolley problem” for purposes of rightful machines.)
Feasibility:
An adequate deontic logic of the law 1) can describe conflicts but 2) normatively requires their resolution
SDL fails, but NMRs can meet these requirements
Legal duties must be precisely specified
A rational agent architecture: 1) rat agent (LP) constraining 2) control system (ML) for 3) sensors and actuators
An implementation: answer-set (logic) programming
ob(-A) :- murder(A), not qual(r1(A)). qual(r1(A)) :- act(A), not ob(-A).
murder(A) :- intentional(A), act(A), causes_death(A, P), person(P).