Data, Responsibly: The Next Decade of Data Science

Bill Howe, PhDAssociate Professor, Information SchoolAssociate Director, eScience Institute

Adjunct Associate Professor, Computer Science & EngineeringUniversity of Washington

05/03/2023 2

My goals this evening• Describe emerging topics in data science

research and practice around a technical interpretation of ethics

• Describe some specific thrusts we are pursuing• Encourage you to get involved

Data, Responsibly / SciTech NW

05/03/2023 3

How much time do you spend “handling data” as opposed to “doing science”?

Mode answer: “90%”

Bill Howe, UW

1) Upload data “as is”Cloud-hosted; no need to install or design a database; no pre-defined schema

2) Analyze data with SQLRight in your browser, writing queries on top of queries on top of queries ...

SELECT hit, COUNT(*) FROM tigrfam_surface GROUP BY hit ORDER BY cnt DESC

3) Share the results Click on the science question, see the SQL that answers it

5

SPARQL(GEMS)Serial C++

PGAS/HPCMyriaX RDBMS

SQLDatalogMyriaL

Compiler Compiler Compiler Compiler Compiler

Hadoop (via layers)

Compiler

multiple languages

multiple big data systems

multiple GUIs/apps

May 3, 2023 6

Making it easier to do data science• SQLShare: Easier to use a database• Myria: Easier to use a bunch of different

systems at once, at scale

• Worked great in the physical sciences

• But some collaborators weren’t that excited…

05/03/2023 7Bill Howe, UW

Data Science Kickoff Session:137 posters from 30+ departments and units

Data, data, data

8

Kevin MerritCEO Socrata

Deep DhillonCTO Socrata

9

• Pursue transformative interdisciplinary urban research• Facilitate translation from UW to .gov stakeholders• Position Seattle/UW as a leader in applied urban research• 80+ faculty from 20+ departments around campus

10

Assessing Community Well-BeingThird-Place Technologies

Optimization of King County Metro ParatransitComputer Science & Engineering

Predictors of Permanent Housing for Homeless FamiliesBill and Melinda Gates Foundation

Open Sidewalk Graph for Accessible Trip PlanningComputer Science & Engineering

Inaugural 2015 program:16 spots140 applicants …from 20+ departments

11

Mining Online Data to Detect Unsafe Food ProductsElaine Nsoesie, Institute for Health Metrics and

EvaluationORCA data for improved transit system planning and operation

Washington State Transportation Center (TRAC)Global Open Sidewalks: Creating a shared open data layer

Taskar Center for Accessible TechnologyCrowdSensing Census: A tool for estimating poverty

Bell Labs, Nokia

2016 program:16 spots190 applicants

New in 2016: An explicit emphasis on data ethics

05/03/2023 12Bill Howe, UW

July 2016

“Data, Responsibly”Dagstuhl Workshop

GerhardWeikum

Serge Abiteboul

Julia Stoyanovich

GeromeMiklau

14

Cathy O’Neil

September 2016

Three properties of a WMD:

OpacityScaleDamage

First decade of Data Science research and practice:

What can we do with massive, noisy, heterogeneous datasets?

Next decade of Data Science research and practice:

What should we do with massive, noisy, heterogeneous datasets?

The way I think about this…..(1)

05/03/2023 16

The way I think about this…. (2)

Decisions are based on two sources of information:

1. Past examplese.g., “prior arrests tend to increase likelihood of future

arrests”

2. Societal constraintse.g., “we must avoid racial discrimination”

Data, Responsibly / SciTech NW

We’ve become very good at automating the use of past examples

We’ve only just started to think about incorporating societal constraints

05/03/2023 17

The way I think about this… (3)

How do we apply societal constraints to algorithmic decision-making?

Option 1: Keep a human in the loopEx: EU General Data Protection Regulation requires

that a human be involved in legally binding algorithmic decision-making

Ex: Wisconsin Supreme Court says a human must review algorithmic decisions made by recidivism models

Option 2: Build them into the algorithms themselvesI’ll talk about some approaches for this

Data, Responsibly / SciTech NW

05/03/2023 18

The way I think about this…(4)

On transparency vs. accountability:• For human decision-making, sometimes explanations

are required, improving transparency– Supreme court decisions– Employee reprimands/termination

• But when transparency is difficult, accountability takes over– medical emergencies, business decisions

• As we shift decisions to algorithms, we lose both transparency AND accountability

• “The buck stops where?”Data, Responsibly / SciTech NW

05/03/2023 19

Some Facets of “Data, Responsibly”

• Privacy• Fairness• Transparency• Reproducibility• Ethics

Data, Responsibly / SciTech NW

I won’t be talking about this

I’ll give a taste of the work here

I won’t be talking about this

Towards automatic scientific claim-checking

Vignette on teaching data ethics

05/03/2023 20

FAIRNESS

Data, Responsibly / SciTech NW

21

Ex: Staples online pricing

Reasoning: Offer deals to people that live near competitors’ storesEffect: lower prices offered to buyers who live in more affluent

neighborhoods

22

[Latanya Sweeney; CACM 2013]

Racially identifying names trigger ads suggestive of an arrest record

slide adapted from Stoyanovich, Miklau

23

Propublica, May 2016

24

The Special Committee on Criminal Justice Reform's hearing of reducing the pre-trial jail population.

Technical.ly, September 2016

Philadelphia is grappling with the prospect of a racist computer algorithm

Any background signal in the data of institutional racism is

amplified by the algorithm

operationalized by the algorithm

legitimized by the algorithm

“Should I be afraid of risk assessment tools?”

“No, you gotta tell me a lot more about yourself.At what age were you first arrested? What is the date of your most recent crime?”

“And what’s the culture of policing in the neighborhood in which I grew up in?”

26

Towards a precise characterization of fairness…

Positive Outcomes Negative Outcomes

offered employment denied employment

accepted to school rejected from school

offered a loan denied a loan

offered a discount not offered a discount

Label outcomes to individuals as positive or negative

Fairness is concerned with how outcomes are assigned to a population

slide adapted from Stoyanovich, Miklau

27

Statistical parity

race

black

white

⊕ ⊖⊖

⊕⊕⊖⊖

⊖

⊕

40% of the whole population

positiveoutcomes

⊖

Statistical paritydemographics of the individuals receiving any outcome are

the same as demographics of the underlying population

20% of black

60% of white

slide adapted from Stoyanovich, Miklau

28

First attempt: Ignore sensitive information

zip code

10025 10027

race

black

white

20% of black

60% of white

⊕⊖⊖⊖

⊕⊕ ⊖

⊖

⊖

⊕

positiveoutcomes

Removing race from the vendor’s assignment process does not prevent discrimination

Assessing disparate impactDiscrimination is assessed by the effect on the protected sub-population, not by the input or by the process that lead to the

effect.

slide adapted from Stoyanovich, Miklau

29

More directly: Impose statistical parity

credit score

good bad

black

white

⊕⊖⊖

⊖⊕⊕ ⊖

⊖

⊖⊕

positive outcomes

40% of black

40% of white

race

positive outcome: offered a loan

Tradeoff between (perceived) accuracy and fairness; may be contrary to the goals of the vendor

slide adapted from Stoyanovich, Miklau

30

A systems approach:FairTest: fairness test suite for data analysis apps

• Tests for unintentional discrimination according to several representative discrimination measures.

• Automates search for context-specific associations between protected variables and application outputs

• Report findings, ranked by association strength and affected population size

[F. Tramèr et al., arXiv:1510.02377 (2015)]

As a corporation, should I care?

Compliance

Jacobson, Scientific American, 2013

CustomerRetention

Employee Retention

Eichler, Hiffington Post, 2012

CNET, May 2016

05/03/2023 32

REPRODUCIBILITY

Bill Howe, UW

05/03/2023 33

Science is a complete mess• Reproducibility

– Begley & Ellis, Nature 2012: 6 out of 53 cancer studies reproducible – Only about half of psychology 100 studies had effect sizes that

approximated the original result (Science, 2015)– Ioannidis 2005: Why most public research findings are false– Reinhart & Rogoff: global economic policy based on spreadsheet

fuck ups

Bill Howe, UW

Science, 2015

05/03/2023 35Data, Responsibly @ Dagstuhl

Retractions are increasing…..

05/03/2023 37

Why is this happening? (1)

Bill Howe, UW

05/03/2023 38

Why is this happening? (2)

Bill Howe, UW

Why is this happening? (2)Publication Bias!

“DEEP CURATION”TOWARDS AUTOMATIC SCIENTIFIC CLAIM CHECKING

05/03/2023 41

Vision: Validate scientific claims automatically– Check for manipulation (manipulated images, Benford’s Law)– Extract claims from papers– Check claims against the authors’ data– Check claims against related data sets– Automatic meta-analysis across the literature + public

datasets

• First steps– Automatic curation: Validate and attach metadata to public

datasets– Longitudinal analysis of the visual literature

Data, Responsibly / SciTech NW

Microarray experiments

05/03/2023 43Bill Howe, UW

Microarray samples submitted to the Gene Expression Omnibus

Curation is fast becoming the bottleneck to data sharing

Maxim Gretchkin

Hoifung Poon

Maxim Gretchkin

Hoifung Poon

No growth in number of datasets used per paper!

Maxim Gretchkin

Hoifung Poon

Majority of samples are one-time-use only!

color = labels supplied as metadata

clusters = 1st two PCA dimensions on the gene expression data itself

Can we use curate algorithmically?Maxim Gretchkin

Hoifung Poon

The expression data and the text labels appear to disagree

Maxim Gretchkin

Hoifung Poon

Better Tissue Type Labels

Domain knowledge (Ontology)

Expression data

Free-text Metadata

2 Deep Networkstext

expr

SVM

Deep Curation Maxim Gretchkin

Hoifung Poon

Distant supervision and co-learning between text-based classified and expression-based classifier: Both models improve by training on each others’ results.

Free-text classifierExpression classifier

Deep Curation: Our stuff wins, with no training data

Maxim Gretchkin

Hoifung Poon

state of the art

our reimplementation of the state of the art

our dueling pianos NN

amount of training data used

05/03/2023 51

VIGNETTE ON TEACHING DATA ETHICS

Bill Howe, UW

Alcohol Study, Barrow Alaska, 1979Native leaders and city officials, worried about drinking and associated violence in their community invited a group of sociology researchers to assess the problem and work with them to devise solutions.

Methods• 10% representative sample

(N=88) of everyone over the age of 15 using a 1972 demographic survey

• Interviewed on attitudes and values about use of alcohol

• Obtained psychological histories including drinking behavior

• Given the Michigan Alcoholism Screening Test (Seltzer, 1971)

• Asked to draw a picture of a person– Used to determine cultural

identity

Results announced unilaterally and publicly

At the conclusion of the study researchers formulated a report entitled “The Inupiat, Economics and Alcohol on the Alaskan North Slope” which was released simultaneously at a press release and to the Barrow community. The press release was picked up by the New York Times, who ran a front page story entitled Alcohol Plagues Eskimos

The results of the Barrow Alcohol Study in Alaska were revealed in the context of a press conference that was held far from the Native village, and without the presence, much less the knowledge or consent, of any community member who might have been able to present any context concerning the socioeconomic conditions of the village. Study results suggested that nearly all adults in the community were alcoholics. In addition to the shame felt by community members, the town’s Standard and Poor bond rating suffered as a result, which in turn decreased the tribe’s ability to secure funding for much needed projects.

Backlash

Methodological Problems“The authors once again met with the Barrow Technical Advisory Group, who stated their concern that only Natives were studied, and that outsiders in town had not been included.”

“The estimates of the frequency of intoxication based on association with the probability of being detained were termed "ludicrous, both logically and statistically.””

Edward F. Foulks, M.D., Misalliances In The Barrow Alcohol Study

Ethical Problems• Participants were not in control of their data nor

the context in which they were presented.• Easy to demonstrate specific, significant harms:

– Social: Stigmatization– Financial: Bond rating lowered

• Important: Nothing to do with individual privacy– No PII revealed at any point, to anyone– No violations of best practices in data handling– But even those who did not participate in the study

incurred harm

Two Topics• Social Component: Codes of Conduct• Technical Component: Managing

Sensitive Data

Ethical principles vs. ethical rules• In the Barrow example, ethical

rules were generally followed• But ethical principles were violated:

The researchers appear to have placed their own interests ahead of those of the research subjects, the client, and society

Principles: Codes of Conduct

• American Statistical Association– http://www.amstat.org/committees/eth

ics/• Certified Analytics Professional

– https://www.certifiedanalytics.org/ethics.php

• Data Science Association– http://www.datascienceassn.org/code-

of-conduct.html

Recap• There’s a sea change underway in how we will

teach and practice data science• No longer only about what can be done, but

about what should be done• This is not just a policy/behavior/culture issue –

there are technical problems to solve

• If you’re not thinking about this stuff, you will be facing retention issues and compliance issues very soon– Witness privacy, which is a few years ahead