Embed Size (px)
Citation preview
ALGORITHMIC INJUSTICEHow AI Embeds Human Bias and
Distorts Our Decision Making
Antony K. HaynesAssociate Dean & Assistant Professor, Albany Law School(518) 445-2321 (w) | (202) 417-8193 (c)[email protected] (w) | [email protected] (h)
Quantification Bias
Character Algorithm Exercise
Sequential Steps1. Draw a circle for the body.2. Add two eyes.3. Add a crown.4. Add wings.5. Add four legs.
Houston Fed'n of Teachers, Local 2415 v. Houston Independent School District (HISD), 251 F. Supp. 3d 1168 (S.D. Tex. 2017)
Education Value-Added Assessment System (EVAAS)
2017: J. Smith ruled on SJ for P’s on 14A DP claimoHISD’s EVAAS poses a realistic threat to deprive P’s of employment
oSAS uses a proprietary software algorithm to calculate scores
oSAS algorithm design to prevent reverse engineering by anyoneoSAS unable to detect or correct data errors in a timely fashion
• “When a public agency adopts a policy of making high stakesemployment decisions based on secret algorithms incompatible withminimum due process, the proper remedy is to overturn the policy.”
How would you create a hiring algorithm?
AI Evaluates Candidates’ Videos
“[T]he company uses proprietary machine learning algorithms thatanalyze data points found in the video interview and then predictsfuture job performance of the candidate. As a result, it can predictwho will be the top performers that can meet the company’s businessobjectives
“HireVue interviews are video based and allow a company’s recruiter tosee non-verbal cues. These are facial expressions, eye-movements,body movements, details of clothes and nuances of the voice. Asmentioned earlier, these same nonverbal cues are collected as datapoints and processed by HireVue technology to perform meaningfulassessments.” –About HireVue Interview
AI “Interviews” Candidates
ape*
Which Auto-tags???
Which auto-tag?
Whose Faces Get Recognized?
Whose Faces Get Shown (At All)?
Kodak Shirley Card (c. 1940s)
Features
Physical Design Can Exclude and Isolate Specific Groups
• Robert Moses shaped New York City from 1930s – 60s
• His Long Island Parkway overpasses blocked buses
• Effectively blocked Long Island from poor & minorities
Low Overpass on Long Island Parkway
Features
117 million American adults are now part of a virtual, perpetual line-up
“we can predict crime”
Predictive policing uses police records to learn patterns in the occurrence of crime
Using these patterns, the computer then predicts the most likely locations of future crimes
Predictive policing uses police records to learn patterns in the occurrence of crime
all policerecordsarrest
records
Using these patterns, the computer then predicts the most likely locations of future crimes
future recordedcrimes
future recordedarrests
As police are then dispatched to the locations with the highest predicted rate of crime arrests, new recorded crimes arrests validate initial biased predictions and amplify their influence in subsequent forecasting.
runawayfeedback loop
*Adapted from slide presentation by William Isaac and Kristian Lum, To Predict and Serve? Predictive Policing with Biased Training Data (HRDAG Oct. 2016)
Predict Crime?
“Developed by our Northpointe division, COMPAS is a state-of-the-art, fourth generation risk/needs assessment tool that uses sophisticated statistical models to accurately assess both static risk factors (those things, such as prior criminal history, that can’t change) and dynamic risk factors (those things, such as criminal thinking and attitudes, that can change).”
--http://www.courtview.com/evidence-based
Was one of your parents ever sent to jail or prison How many of your friends/acquaintances are taking
drugs illegally? How often did you get in fights while at school? A hungry person has a right to steal. If people make me angry or lose my temper, I can be
dangerous.
WhiteAfrican
American
Labeled High Risk, But Didn’t Re-Offend
23.5% 44.9%
Labeled Low Risk, Yet Did Re-Offend
47.7% 28.0%
Fair?
Falsely labeled a
future criminal
Falsely labeled a
future law-abiding citizen
State v. Loomis, 371 Wis. 2d 235 (2016)
“Whether the circuit court’s consideration of a COMPAS riskassessment violated Loomis’s constitutional right to due processpresents a question of law . . . . [W]e ultimately conclude that aCOMPAS risk assessment can be used at sentencing.”
“Loomis advances initially that the proprietary nature of COMPASprevents a defendant from challenging the scientific validity of the riskassessment. . . . Although Loomis cannot review and challenge how theCOMPAS algorithm calculates risk, he can at least review and challengethe resulting risk scores set forth in the report attached to the PSI.”
It is better to make a type II error than a type I error
Zero criminal history
Arrestads
multiple criminal records
Noarrest
ads
Google Searches for ???
“three black teenagers” “three white teenagers”
Google Searches for ???
“unprofessional hairstyles for work”
“professional hairstyles for work”
Google Search for “CEO”
Percentage of women in top 100 Google image search results for CEO: 11%
Percentage of U.S. CEOs who are women: 27%Black: 3.4%
Black : 2%
Language Translation
Who Works in Tech?
Twitter and Tay
“The free speech wing ofthe free speech party.”
“What makes Twitter greatis that it’s open to everyoneand every opinion.”
How address harassment?
• Open algorithms / Audit
• Inclusive data sets
• Continuously updated
• Outputs benchmarking
• Redefine success (no bias)
• Watch Feedback loops
Inclusive Coding
•Who Codes Matters
•How We Code Matters
•Why We Code Matters
https://ainowinstitute.org/)
https://www.ajlunited.org/
http://www.blackgirlscode.com/https://www.partnershiponai.org/thematic-pillars/
IMMORAL SOFTWAREHow AI Embeds Human Bias and
Distorts Our Decision Making
Antony K. HaynesAssociate Dean for Strategic Initiatives and Information Systems, Director Online Graduate Programs,Director Cybersecurity & Privacy Law, andAssistant Professor of Law
Albany Law School | 80 New Scotland Ave | Albany, NY 12208
Phone: 518.445.2321 (w) | 202.417.8193 (c)E-mail: [email protected] | Twitter: @antonykhaynes
