Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Fairness via Separation of PowersThe Harvard community has made this
article openly available. Please share howthis access benefits you. Your story matters
Citation Cowan, Ethan Andrew. 2020. Fairness via Separation of Powers.Master's thesis, Harvard Extension School.
Citable link https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37365039
Terms of Use This article was downloaded from Harvard University’s DASHrepository, and is made available under the terms and conditionsapplicable to Other Posted Material, as set forth at http://nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of-use#LAA
Fairness via Separation of Powers
Ethan Cowan
A Thesis in the Field of Software Engineering
for the Degree of Master of Liberal Arts in Extension Studies
Harvard University
May 2020
Abstract
We propose a method of separate actors with mixed incentives for the cre-
ation of models which conform to various fairness conditions. Instead of viewing Fair
Machine Learning as an optimization problem under fairness constraints, we divide
the responsibility into three independently acting groups: a Fairness Condition Cre-
ator (Fairness Maximizer), a Model Trainer (Accuracy Maximizer), and an Overseer
(Justice Maximizer). By following a workflow inspired by theories of democratic gov-
ernance and mixed incentive structures, these three groups can converge onto fair and
accurate models according to context-specific definitions.
Acknowledgements
I would firstly like to thank Professor David Parkes for his guidance during
the creation of this thesis. What began as a nebulously defined interest became a
fully fledged research project only with his support and expertise. This thesis would
not have been possible without his brilliance for navigating the connections between
seemingly disparate domains.
I owe a special deal of thanks to Elizabeth Langdon-Gray and the staff of the
Harvard Data Science Initiative (HDSI), for organizing the conference which piqued
my interest in the topic of algorithmic fairness and ethics in machine learning. The
majority of this work was written in the HDSI office, which provided a congenial work
atmosphere and support network.
This research comes at the end of several years of study in the Harvard Exten-
sion School, which has been an invaluable intellectual resource to me since 2016. It
meant a great deal to me that so many classes were opened to students from around
the world, and many of the skills and topics that I learned were central to the creation
of this research.
This thesis is dedicated to my parents and grandparents.
ii
Contents
Table of Contents iii
List of Figures vi
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Democratic Parallels . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Fairness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Liberty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.7 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Fairness via Separation of Powers 15
2.1 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Organizational Structure . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Constitution Creation . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.5 Mixed Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 Appeals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
iii
3 Transformations and Metrics 26
3.1 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.1.1 Metrics on Statistical Distributions . . . . . . . . . . . . . . . 29
3.2 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2.1 Statistical Transformations . . . . . . . . . . . . . . . . . . . . 31
4 Simulations 33
4.1 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Iris Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.2.1 Constitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2.2 Fairness Conditions . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2.3 Tenet 1: Similar Means . . . . . . . . . . . . . . . . . . . . . . 41
4.2.4 Tenet 2: Class Balance . . . . . . . . . . . . . . . . . . . . . . 42
4.2.5 Tenet 3: Closeness Between Minima and Maxima . . . . . . . 42
4.2.6 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.7 Data Transformation Steps . . . . . . . . . . . . . . . . . . . . 44
4.2.8 Transformed Data . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 COMPAS Simulation 53
5.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Constitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.3 The Importance of Hyper Parameters . . . . . . . . . . . . . . . . . . 60
5.4 Penalties and Disparate Outcomes . . . . . . . . . . . . . . . . . . . . 63
5.5 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.6 Fairness Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.6.1 Tenet 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
iv
5.6.2 Tenet 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.6.3 Tenet 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.7.1 Step Count Estimation . . . . . . . . . . . . . . . . . . . . . . 73
6 Conclusion 76
A Data Transformation Paths 79
A.1 Transformation Paths and Accuracy (Iris Data) . . . . . . . . . . . . 80
A.2 Transformation Paths (COMPAS Data) . . . . . . . . . . . . . . . . . 83
References 85
v
List of Figures
2.1 Separation of Powers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Constitutional Convention (Sequential Voting) . . . . . . . . . . . . . 20
2.3 Constitutional Convention (k-Subset Voting) . . . . . . . . . . . . . . 21
2.4 Bonus Incentive Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5 Negative Incentive Matrix . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1 Distance comparisons for data sampled from N (0, 1) . . . . . . . . . 28
3.2 The relationship between DM and DB . . . . . . . . . . . . . . . . . 29
3.3 Distance and Similarity for data sampled from N (0, 1) . . . . . . . . 30
4.1 Iris Data with 2 Classes . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2 Summary statistics for the Iris Data Set . . . . . . . . . . . . . . . . 36
4.3 Summary statistics for Iris Setosa . . . . . . . . . . . . . . . . . . . . 36
4.4 Summary statistics for other species (Iris Data) . . . . . . . . . . . . 37
4.5 Tenet 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.6 Tenet 2 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.7 Tenet 3 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.8 Iris Data Set Constitution . . . . . . . . . . . . . . . . . . . . . . . . 40
4.9 Transformation 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . 41
4.10 Transformation 2 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . 41
vi
4.11 Transformation 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . 43
4.12 Transformation 2 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . 43
4.13 Before the transformations (Iris Data) . . . . . . . . . . . . . . . . . 45
4.14 After Step 1 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.15 After Step 2 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.16 After Step 3 (Iris Data) . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.17 Summary statistics for the Iris Data Set after preprocessing (Iris Data) 49
4.18 Summary statistics for Iris Setosa after preprocessing . . . . . . . . . 50
4.19 Summary statistics for other species after preprocessing . . . . . . . . 50
4.20 Model accuracy distribution (Iris Data) . . . . . . . . . . . . . . . . . 51
5.1 Risk Score Demographic Comparisons . . . . . . . . . . . . . . . . . . 55
5.2 Tenet 1: Recidivism Balance . . . . . . . . . . . . . . . . . . . . . . . 56
5.3 Racial Demographics (COMPAS Data) . . . . . . . . . . . . . . . . . 57
5.4 Tenet 2: Racial Demographic Parity . . . . . . . . . . . . . . . . . . . 57
5.5 Individuals counted by sex (COMPAS Data) . . . . . . . . . . . . . . 58
5.6 Tenet 3: Sexual Demographic Parity . . . . . . . . . . . . . . . . . . 58
5.7 Age distribution (COMPAS Data) . . . . . . . . . . . . . . . . . . . . 59
5.8 COMPAS Data Set Constitution . . . . . . . . . . . . . . . . . . . . 60
5.9 Summary statistics by race (COMPAS Data) . . . . . . . . . . . . . . 61
5.10 Trade off between mean error and disparate impact by race (COMPAS
Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.11 Minimal change in mean scores for different penalty types (COMPAS
Data) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.12 Simulated Annealing of AM/FM . . . . . . . . . . . . . . . . . . . . . 65
5.13 fairstep: Fair State Step . . . . . . . . . . . . . . . . . . . . . . . . . 66
vii
5.14 accstep: Accuracy State Step . . . . . . . . . . . . . . . . . . . . . . 67
5.15 Transformation 1 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 67
5.16 Transformation 2 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 68
5.17 Transformation 1 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 68
5.18 Transformation 2 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 69
5.19 Transformation 1 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 69
5.20 Transformation 2 (COMPAS Data) . . . . . . . . . . . . . . . . . . . 70
5.21 Hyperparameter search (COMPAS Data) . . . . . . . . . . . . . . . . 71
5.22 Accuracy Changes (COMPAS Data) . . . . . . . . . . . . . . . . . . 72
5.23 Accuracy difference per step (COMPAS Data) . . . . . . . . . . . . . 73
5.24 Step count summary statistics (COMPAS Data) . . . . . . . . . . . . 74
5.25 Steps to reach a maximum bonus (COMPAS Data) . . . . . . . . . . 75
A.1 Paths with significant accuracy loss . . . . . . . . . . . . . . . . . . . 80
A.2 Paths with significant accuracy loss (cont.) . . . . . . . . . . . . . . . 81
A.3 Paths with no accuracy loss . . . . . . . . . . . . . . . . . . . . . . . 82
A.4 Sample of paths with bonus b/2 . . . . . . . . . . . . . . . . . . . . . 83
A.5 Paths with maximum bonus b . . . . . . . . . . . . . . . . . . . . . . 84
viii
Chapter 1: Introduction
1.1. Background
The accelerating role of technology in daily life has outpaced both the regula-
tory power of governments and the ability of individuals to make informed decisions.
In the period between 2010 and 2020, smartphone usage in the United States increased
by 435.5% (Statista, 2019) along with a commensurate increase in data collection by
smartphone applications. Many of these applications are making seemingly-innocuous
decisions based on this information, such as tailoring the app experience to each user,
or targeting advertisements. Previous research has shown that even the choice of
targeted advertising can have racial biases, leading to different user experiences and
harmful outcomes. In particular, minority businesspeople have lamented the disparity
in search results on a variety of platforms, noting that veteran neighborhood busi-
nesses are often placed in much lower positions than newer, corporate-affiliated stores
(Noble, 2018).
Even more serious consequences can be found in the legal system, where emerg-
ing technologies are being used to make decisions about freedom. The Correctional
Offender Management Profiling for Alternative Sanctions (COMPAS) system was de-
veloped by Equivant Software as a risk estimation tool when making sentencing and
parole decisions. COMPAS uses a variety of data points about a defendant, includ-
ing age of first arrest, and number of prior convictions. The score, between 1 and
1
10, is supposed to inform judges and parole officers about the perceived risk of the
individual being arrested in the future for committing another crime. Prior analysis
by ProPublica argued that these scores show a racial bias, falsely labeling black de-
fendants as future criminals at “almost twice the rate of white defendants” (Angwin,
Larson, Mattu, & Kirchner, 2016). This work led to a discussion about the trade offs
which come about when developing models with potentially high impacts on the lives
of individuals.
The marketing of COMPAS focused on its ability to supposedly make a more
impartial decision than a human actor. Equivant argued that a human is likely to
harbor biases, both conscious and otherwise, that often cloud bail and sentencing de-
cisions. However, the ProPublica analysis argued that the data on which COMPAS
relies is itself biased due to the over-policing and criminalization of communities of
color, and therefore perpetuated the very biases that Equivant claimed to be mini-
mizing.
Since the publication of Dwork et al.’s seminal work (Dwork, Hardt, Pitassi,
Reingold, & Zemel, 2012), there has been a proliferation of interest in the theory
and applications of fair machine learning models. A key early result of this research
demonstrated that neither enforcing demographic parity, nor removing sensitive at-
tributes from training data, are sufficient to ensure a fair outcome. In many cases,
other attributes will correlate strongly with the removed sensitive attribute, leading
the model to continue having a harmful impact. Enforcing demographic parity can
also violate individual fairness, as some individuals in one population may receive a
less favorable outcome in order to balance the demographics of the group.
Common to many previous approaches is the assumption that one individual
or entity is selecting the fairness definition(s), choosing the algorithm, training the
model, and performing the optimization and deploying the final result into the world.
2
This situation supposes that this one individual is acting benevolently, with no intent
to harm any other individual or group. In practice, decision making authority is often
distributed across an organization. Given the seriousness of the issues previously
discussed, it seems unlikely that one individual would be given the power to decide
what will be considered a fair outcome.
1.2. Democratic Parallels
Many democratic societies have adopted a government with a “separation of
powers” arrangement, precisely to prevent abuses of power by one individual or group
with unchecked power. In many countries, this means a government with separate
branches for executive, legislative, and judicial functions, each of which can exercise
certain checks on the power of the others. The ideas leading to this structure can be
traced to Europe before 1700, in which the liberty of subjects and concepts of justice
relied heavily on the whim of an absolute monarch or ruling group.
Following this example from the Enlightenment, a modified form of separation
of powers is proposed for the development of fair machine learning models. The
three entities comprising this system will each have rights and responsibilities in the
process, as well as the ability to check the actions of the other entities should they
fail to act in a way commensurate with their role. By distributing authority and
responsibilities across different parties, the risk of abuses of power by one group is
minimized. It is central to such a system to have a constitution which clearly details
the rights and responsibilities of each party in the system.
The first step of the model building process will be a “constitutional conven-
tion,” during which representatives from the three groups propose principles and cast
votes to create a foundational document against which future work will be held. This
convention serves several purposes. First, it allows individuals to highlight domain-
3
specific challenges in the subject matter, as well as potential biases and problems
which may be encountered in any training data. Second, it serves as a community
meeting during which the group can discuss hopes and concerns for the project, as
well as ways to formally encode these ideals into the model building process. It is
also a time for reservations and concerns to be raised with managers and the rest of
the team. Third, it provides the group with a structured format for voting on tenets,
drafting the constitution, and formally approving it before proceeding with the rest
of the work.
Previous research has focused heavily on the trade-off between fairness and
accuracy. This seems natural, as modifying the training data or subjecting the model
to post-processing, in the service of fairness, will often lead to a decrease in accuracy
relative to the testing data. There is therefore a question of conflicting incentives. If
one fairness condition can be satisfied at minimal expense to overall accuracy, is the
outcome satisfactory? It depends on the situation. By placing this decision making
power in one set of hands, we lose the conversation that could be happening between
stakeholders with different perspectives and definitions of success.
Proposals for the use of democratic structures within a technology company
are not without precedent. In 2018, Facebook proposed an oversight committee of 40
members, to have veto power over the CEO (Read, 2018; “Why Mark Zuckerbergs
Oversight Board May Kill His Political Ad Policy”, n.d.). This decision came as the
company faced criticism over issues of content moderation, particularly the screening
and removal of hate speech. Facebook committed $130 dollars to the new committee,
but had not named any members one and a half years after the initial announcement
(Culliford, 2019).
The model building process will be conducted by three separate actors: the
Justice Maximizer (JM), Accuracy Maximizer (AM), and Fairness Maximizer (FM).
4
The Justice Maximizer acts as an arbiter between the other two parties, approving
the constitution against which they will work, and handling all communication. The
JM also sets the stopping conditions, and has the authority to declare the model
development process over. During the constitutional convention, the JM also has the
authority to approve the final draft of the constitution. The Accuracy Maximizer
has a more traditional role: they perform training and testing of the model using
pre-processed data from the Fairness Maximizer, and attempt to make the model
as performant as possible relative to the given metric. The Fairness Maximizer is
tasked with creating a set of computational conditions (boolean statements) and pre-
processing routines for the training data which will make the fairness conditions true.
For example, if the condition is “a maximum of 25% of the data points can be from
a given zip code,” then the pre-processing can be accomplished via oversampling or
data removal. The Fairness Maximizer is not involved in the model creation process
directly, and has no direct communication with the Accuracy Maximizer. The incen-
tives are structured such that the AM prioritizes conforming to the fairness conditions
first, and only then tries to optimize accuracy for an additional bonus.
1.3. Incentives
In a democratic system, elections are supposed to act as an incentive system
to encourage political actors to work for the good of their constituents, and the
overall well-being of the country. The executive and legislative branches are subject
to regular elections as the public’s check on this political power. The judicial branch
has different levels of courts so that the decision of one level may be overturned by the
court of a higher level. Additionally, some judges are subject to elections as checks
on their decisions.
Since the system being proposed is not subject to the will of voters, an incen-
5
tive structure must be constructed. In the absence of these incentives, the Fairness
Maximizer could write any conditions, the Accuracy Maximizer could ignore them,
and the Justice Maximizer could never approve a constitution or manage communica-
tions between the other two parties. Incentives are constructed to recognize that each
group must act to fulfill their task while also pursuing the common goal. The Fairness
Maximizer only receives a bonus if the Accuracy Maximizer satisfies the given accu-
racy condition laid out in the constitution (e.g. “accuracy can only decrease by 5%
maximum”). Conversely, the Accuracy Maximizer only gets a bonus if a minimum
fraction of the fairness conditions are satisfied, as laid out in the constitution. The
Justice Maximizer receives a bonus only if the constitution is constructed in a way
that the other two parties are able to work effectively and achieve their goals. An
initial bonus is also given to the Justice Maximizer for successfully completing and
approving a constitution within a given window of time. Additional requirements
may be placed on the Justice Maximizer to ensure prompt communication.
The bonus is assumed to be financial or material in nature. Each group has
the opportunity to receive a pay bonus following the completion of the model devel-
opment. The amount depends on context, but must be high enough relative to their
base compensation to motivate the groups to take the process seriously. For amounts
that are viewed by the participants as too small, the fairness maximization process
could be viewed as voluntary or non-essential.
1.4. Fairness
Previous work has focused on two key formulations of fairness, namely Individ-
ual Fairness, and Group Fairness(Dwork et al., 2012), and comparing and contrasting
the two definitions and presenting conditions under which they are and are not sat-
isfactory.
6
Let V denote the set of individuals in question, A the set of outcomes that
these individuals may experience, and ∆(A) the set of probability distributions over
these outcomes. For a distribution S, µS will denote the population mean.
Definition 1. Individual Fairness (Lipschitz Mapping) (Dwork et al., 2012)
The tenet that similar individuals should have similar outcomes relative to a given
measure. Formally, a mapping M : V −→ ∆(A) such that ∀v1, v2 ∈ V ,
D(Mv1,Mv2) ≤ d(v1, v2), (1.1)
where d is a metric describing the similarity of individuals and D is a measure of
similarity between distributions.
Definition 2. Group Fairness (Statistical Parity up to bias ε)(Dwork et al.,
2012) In the context of classification, the tenet that the likelihood of receiving a certain
label does not depend on one’s class membership. Formally, for distributions S and
T , a mapping M : V −→ ∆(A) satisfies group fairness relative to a probability metric
D provided that, for two distributions S, T ∈ ∆A, the distribution means µS and µT
differ by at most ε:
D(µS, µT ) ≤ ε (1.2)
Definition 3. Equalized Odds A classifier Y with α possible labels has equalized
odds relative to an attribute A with β possible values and outcome Y with γ possible
values provided that “Y and A are independent conditional on Y.”(Hardt, Price, &
Srebro, 2016) Formally, for
l ∈ {0, ..., α− 1}, i, j ∈ {0, ..., β − 1}, y ∈ {0, ..., γ − 1} :
P (Y = l|A = i, Y = y) = P (Y = l|A = j, Y = y) (1.3)
7
1.5. Liberty
Questions of liberty are recorded in the Western tradition as far back as Aris-
totle’s Politics, in which the concept was explained both in terms of equal political
representation and unfettered personal action (Aristotle, 2016). The later Hobbesian
formulation is now often referred to as negative liberty, and means the lack of exter-
nal obstacles on one’s actions. For the remainder of this paper, we will use Hobbes’
definition, and will use the term “liberty” to refer to “negative liberty”.
Definition 4. Liberty (or Negative Liberty) - Exemption from external con-
straints, exemption from interference by others. (Pettit, 1989)
Historically, the pursuit of liberty and push against tyranny led to the popular
demand for power to be more widely distributed throughout the political system. In
systems with absolute authority granted to a single group or individual, the rights of
individuals and minority groups were rarely protected and the concept of justice only
existed relative to the whims of those in power.
Montesquieu wrote the modern formulation of separation of powers, including
the prototypical example of separation of powers, as a mechanism for the defense of
liberty. In particular, his writing highlighted the need for an independent judiciary
to judge civil and criminal matters, writing “there is no liberty, if the judiciary power
be not separated from the legislative and executive.”(Montesquieu, 1989)
Definition 5. Separation of Powers - Structure in which authority and responsi-
bilities are uniquely distributed to components of a system. For example, a system of
government in which each branch has powers and limitations (checks and balances)
on those powers imposed by the other branches. The uniqueness of responsibilities is
necessary to prevent different branches from conflicting and competing.
8
1.6. Related Work
Previous work on fairness can broadly be categorized into theoretical research,
data pre-processing methods, applied algorithms for fairness, and interdisciplinary
research, often with connections to the legal literature.
Foundational work in the theory of fair classification largely began with Dwork
et. al.’s work which introduced the concept of “treating similar individual’s similarly”
(Dwork et al., 2012). There has followed a variety of theoretical research focusing
on fairness definitions and the role of sensitive attributes; that is, abstractions for
characteristics such as race and sex which are often the basis of discrimination (Hardt
et al., 2016). Other work has synthesized the ideas of Rawls into a framework for
minimizing disparate impact (Joseph, Kearns, Morgenstern, Neel, & Roth, 2016) and
developed theories for fair representation in order to satisfy individual and group
fairness requirements (Zemel, Wu, Swersky, Pitassi, & Dwork, 2013).
More recent research has outlined methods by which a human “fairness ar-
biter” can answer a series of queries in order to approximate a metric for individual
fairness (Ilvento, 2019). Focus on classifiers has led to work describing unfairness from
a causal perspective, either as an unfair path in a causal Bayesian network (Chiappa
& Isaac, 2019) or relative to other constructed causal models (Khademi, Lee, Foley,
& Honavar, 2019; Kilbertus et al., 2017). Other work has examined unfairness as a
violation of a “contrastive fairness” condition (Chakraborti, Patra, & Noble, 2019),
while others have considered the development of meta-algorithms for ensuring fairness
in classification (Celis, Huang, Keswani, & Vishnoi, 2019).
Fairness conditions have also been considered as part of the optimization pro-
cess (Donini, Oneto, Ben-David, Shawe-Taylor, & Pontil, 2018) by defining a general-
ized loss function which takes group fairness into account. Many of these approaches
9
admit that the academic debate about fairness definitions is ongoing, and must choose
just one definition or else construct a composite out of several existing definitions.
Many applied studies have employed existing ML algorithms in the pursuit of
fair outcomes. For example, studies have focused on the use of adversarial learning
to encourage fair outcomes (Wadsworth, Vera, & Piech, 2018), with a particular eye
toward the COMPAS data set and recidivism prediction. Similarly, work has been
published that uses the k nearest neighbors (kNN) algorithm to both discover and
prevent discrimination (Luong, Ruggieri, & Turini, 2011). This work synthesizes the
legal concept of situation testing with kNN clustering in order to uncover biases in
the classification process.
Goals of individual or group fairness are often pursued via data preprocessing
techniques. As the behavior of a model is heavily guided by the training data it is
given, there are a variety of ways to influence the model outcome, including oversam-
pling, undersampling, and data removal. Early work on the topic of preprocessing
examined the ways in which discrimination on the basis sensitive attributes can be
minimized (Kamiran & Calders, 2012). This work divided the possible methods into
four groups: suppression (removal of attributes), massaging (relabeling the data),
reweighing (assigning different weights to data points to achieve a fair result), and
sampling (for example, changing the class balances in the training data set). More re-
cent work has used the reweighing approach to impose fairness constraints on models
built with biased data (Blum & Stangl, 2019). Work has also been done to examine
what data should be collected at all, recommending the use of third parties to hold
sensitive attributes during the model creation process. In one variation of this system,
the third party acts as a data preprocessor, training the data to “protect anonymity
and preserve bias” (Veale & Binns, 2017).
Many of the data transformations used to demonstrate the framework pro-
10
posed in this thesis are built upon the work of Kamiran and Calders. The approaches
toward fair model development are broadly divided into three categories: modifying
a pretrained classifier, enforcing fairness during the training step, and modifying the
data representation (Donini et al., 2018). This thesis is concerned mainly with the
third approach - modifying representations in the training data, and then performing
an unmodified optimization process.
The legal and ethical literature has focused on the problem of fairness in ML
from several perspectives. While some authors have focused on the compatibility of
fairness definitions between the legal and technological sectors (Xiang & Raji, 2019),
others have used popular writing on the topic to address current issues. In partic-
ular, Chander’s review of The Black Box Algorithm raises questions about contem-
porary algorithmic usage, including Facebook’s patent to determine creditworthiness
via one’s friend graph (Chander, 2017). Chander also addresses the case of Wisconsin
v. Loomis (Wisconsin v. Loomis, No. 2015AP157CR, 2016), in which an individual
convicted of a crime and sentenced using COMPAS challenged the sentencing as a vi-
olation of due process. The appeal challenged the usage of COMPAS on the grounds
that the COMPAS system is closed source and also that it takes race and gender into
account. The appeal went to the Supreme Court of the United States, which declined
to hear the case.
The interaction between transparency and accountability in algorithmic deci-
sion making has been addressed by Kroll et al., in an article that argues that trans-
parency is both infeasible and insufficient to guarantee fairness in many cases. From
an intellectual property and privacy perspective, revealing the inner workings of an
algorithm may be either undesirable or illegal, meaning that accountability cannot
be based on the assumption of transparency. The authors argue that, when designing
technical systems, “accountability must be part of the system’s design from the start”
11
(Kroll et al., 2017).
1.7. Contribution
In this thesis, I introduce a mechanism by which organizations can incentivize
the creation of fair machine learning models. This mechanism contains instructions
for the creation of a project-specific constitution and an avenue for the creation of
fairness conditions which satisfy it. Drawing inspiration from behavioral economics,
I contribute an incentive structure for multiple actors to achieve different aspects of
the same end goal.
The main contributions of this thesis come from identifying parallels between
theories of liberty and democratic governance and the technology creation process.
From this basis, I provide a framework for examining the actions of a ML model. The
actions of the ML model can be thought of in terms of potential violations of liberty,
and separation of powers in the model creation process can minimize this risk. Once
the process is understood as a political process, I propose the separation of powers
mechanism, which incentivizes different groups with different responsibilities within
an organization to work toward each other’s success.
1.8. Outline
Chapter 2 lays the theoretical foundation for the separation of powers mech-
anism. This begins with a discussion of the political history that led to the formu-
lation of separation of powers, as well as distinguishing definitions between positive
and negative liberty. I define key terms which will be used throughout the remainder
of this thesis. I also outline the scenario that necessitates this mechanism, includ-
ing the weaknesses with the current model in which one group or individual makes
12
the key decisions in the model creation process. This is followed by a discussion of
the three main actors in the mechanism: the Fairness Maximizer (FM), the Accu-
racy Maximizer (AM), and the Justice Maximizer (JM), as well as their rights and
responsibilities.
The processes that constitute the mechanism are described as algorithms, to
provide a clear and unambiguous description of each part of the model creation pro-
cess. This includes the constitution creation process, during which the different groups
come together to agree on the fundamental tenets that will be used. This is followed
by a description of the incentive structure, including the mixed incentive matrix,
which ties the success of each group to the success of the other. The final step in the
process, known as the appeals process, is then defined and justified.
Chapter 3 provides several foundational definitions related to measuring the
difference between two data sets. This is central to this work, as many of the fairness
goals are tied to transformations of the training data. The distance between the
original data set and its transformed counterpart will give us a sense of how far the
final, fairness-condition satisfying models will be from a model trained on the original
data with no fairness conditions considered.
Chapter 4 details the algorithms used for simulating the separation of powers
mechanism. Without experiments with organizations, I must perform computational
simulations, where each actor is modelled as a semi-autonomous agent which can
decide which fairness conditions to implement in order to maximize an objective
function. In this case, the system has a constitution composed of a variety of compu-
tational conditions, and data preprocessing functions are constructed to make those
fairness conditions true within the training data. The system then iterates over dif-
ferent permutations of the fairness condition preprocessors in order to satisfy the
maximal number while simultaneously maximizing accuracy.
13
In Section 4.2, I document the results of the first study on a real data set.
Using the Iris Data Set, I construct a scenario in which one outcome is considered
less ideal than the other, and run the data through various preprocessors before
training the model. The results contain accuracy loss relative to the original training
data and also relative to the transformed training data. This data was chosen for its
familiarity, as it is a classic data set that has been previously used for educational
and demonstrative methods.
In Section 5 I apply the separation of powers mechanism to a contemporary
data set: the COMPAS recidivism prediction data set. This simulation shows results
for a data set which has been thoroughly studied and is not as statistically clean
as the Iris Data Set. This is also a data set from a tool that has been accused of
disparate treatment on the basis of race, sex, and age.
Chapter 6 summarizes the separation of powers mechanism, including the the-
oretical foundation from political science, as well as the findings from the simulations.
14
Chapter 2: Fairness via Separation of Powers
2.1. Key Terms
We will be working with the following definitions:
Definition 6. Condition A function ξ : E −→ Y where E is a set of events and
elements in Y are countable and have a weak, total ordering reflecting their prefer-
ability. A mechanism for relating the preferability of outcomes for different events,
where ξ(x1) > ξ(x0) means that x1 is a preferable outcome to x0 relative to some
external reward.
e.g. For an exam, the scores can be ordered in order of preference:
ξ(x) =
0 x ≤ 60
0.5 60 ≤ x ≤ 80
1 x ≥ 80
Throughout this study, many of the conditions will be boolean, with one of
either true or false being a preferable outcome.
Definition 7. Computational Condition - Building upon Definition 6, a com-
putational condition is a tuple (C, τ0, ..., τn) where C is a Condition (boolean function)
C : X −→ {0, 1} and each of τ0, ..., τn is a data transformation such that C(τi(X)) = 1
∀i ∈ {0, ..., n}.
15
Definition 8. Action - Something done by a model which results in consequences,
such as classification or prediction. The Action is the verb performed by the model
which may cause disparate impact or other harmful result. The goal of the model
building process will be to make the Action performant subject to a measure while also
satisfying as many fairness conditions as possible.
For classification, the Action of a classifier M with k possible labels L =
{l0, ...lk−1} and input data ~x ∈ RN is the act of mapping M : RN −→ L which assigns
a label to each point in RN .
Definition 9. Societal Values - A set of conditions S = {C0, ..., CN} , possibly
related or even contradictory, that demarcate the set of preferable characteristics or
judgements over a set of events X.
Definition 10. Tenet - A function T : A −→ [0, 1] where A is a subset of possible
actions, and T(a) represents the acceptability value of the action a ∈ A. For example,
if an action a endangers the safety of others, or otherwise violates a central right, then
T (a) = 0, whereas a socially endorsed action (charity, volunteering) may be assigned
T (a) = 1. The set of tenets is drawn from a universe of tenets U = {T : A −→ [0, 1]}
where each tenet maps the set of possible actions onto an acceptability value.
Definition 11. Constitution - A set C = {T1, ...., TN} of tenets, possibly related,
that demarcate the rights and liberties which are most highly valued by a society.
2.2. Scenario
Suppose an organization is attempting to train and use a machine learning
model, and wants it to be fair in the eyes of both cultural norms and internal stan-
dards. For example, the organization wants the model to respect individual fairness.
These conditions are not assumed to be written in the law.
16
The model creation process can then be recast as one of minimizing the loss
function while simultaneously satisfying the maximal number of tenets within a con-
stitution. However, following historical precedent, placing this much decision making
power into the hands of one individual or one group acting autonomously can lead to
abuses of power. In the current paradigm, the highest priority will likely be speed of
development and accuracy. Without strong incentives and a restructured workflow,
the concept of fairness is likely to be deprioritized and treated as an afterthought. For
now we will suppose that a complete constitution has been received and the task is to
create a model that satisfies fairness relative to this constitution. (Further discussion
in Section 2.4)
2.3. Organizational Structure
The model creation process is split into three semi-autonomous groups: the
Accuracy Maximizer (AM), Fairness Maximizer (FM), and the Justice Maximizer
(JM). Each of thee groups has distinct rights, responsibilities, and authorities in the
process. By separating the power into three groups, we minimize the possibility of
abuse while still maintaining a clear recipe for optimization. These groups are loosely
based on the branches of government found in the trias politica model: executive
(AM), legislative (FM), and judicial (JM).
The FM will interpret the constitution in order to construct a set of compu-
tational conditions that the model must follow, as well as recipes for modifying the
training data to satisfy the conditions. These conditions are expressed as boolean
test conditions, along with transformation functions which modify the training data
to make the conditions true.
The FM is expected to have knowledge of the data set and problem domain
in order to construct these conditions. The AM will similarly use their knowledge to
17
FM AM
JM
Training Data
Fair Conditions
Status Report
Status Report
Model Accuracy / Params
Figure 2.1: Separation of Powers.
train the most accurate model they can, in order to serve as a baseline, without yet
taking fairness conditions in to account. The JM serves as the communication between
the two groups; they send the rules from the FM to the AM, and send information
about the model (e.g. method used, etc.) back from the AM to the FM. The JM
is also ultimately tasked with signing off when an optimal model is believed to have
been found. At each step of the process, the FM will send modified training data
to the AM, along with a vector of binary values specifying which fairness conditions
were used.
2.4. Constitution Creation
For each new project, the organization must hold a “constitutional conven-
tion,” in which representatives from each working group discuss and vote on which
18
tenets will be used in the model creation process. This will be highly context-specific,
and will involve background research on the subject matter and potential ethical and
privacy issues. The drafters will need to depend on legal and ethical standards for the
particular domain being modelled. For example, a model related to housing would
be expected to depend heavily on the Fair Housing Act.
Formally, a constitutional convention is a process where a constitution C
with N tenets {T1, ...., TN} is agreed upon by a committee with P voting members
{I1, ..., IP}. The specifics of the voting process can be left to the organization, which
can choose a consensus or majoritarian model. Regardless of the method, the process
vote(Ti, I) will return 1 if the tenet is accepted, and 0 otherwise. The proposed tenets
can also be submitted anonymously, to prevent personal biases or power dynamics
from interfering with the process. Additionally, the voting process can be carried out
sequentially, sampling each tenet randomly without replacement, or alternatively a
subset vote can be made, in which all tenets are reviewed as a whole by each individ-
ual and a group of top tenets are voted on by each individual. For this approach, a
parameter k must be set for the number of tenets that will be adopted.
In the following algorithm, the function propose(Ii) is a stand in for the ith
individual proposing a tenet to the group, Tprop is an array of the proposed tenets
that have not yet been accepted, and C is the final constitution, containing the tenets
that have been accepted.
19
Input: I = {I0, ..., IN−1}
Result: C = {T0, ..., TM−1 }
j = 0
Tprop = []
C = []
for i← 0 to N − 1 do
Tprop[i] = propose(Ii)
end
foreach T ∼ U(Tprop) do
if vote(Tprop[i], I) == 1 then
C[j] = Tprop[i]
j + +
end
end
return C
Figure 2.2: Constitutional Convention (Sequential Voting)
20
Input: k ∈ Z, I = {I0, ..., IN−1}
Result: C = {T0, ..., TM−1 }
Tprop = []
for i← 0 to N − 1 do
Tprop[i] = propose(Ii)
end
// Return an array of size k*N, the top k tenet choices for each individual
v =choose-top-k(I)
vsort = distinct(countsort(v))
C = {vsort0 , ..., vsortk−1}
return C
Figure 2.3: Constitutional Convention (k-Subset Voting)
2.5. Mixed Incentives
To incentivize these groups to find common success, we propose a bonus struc-
ture based on criteria satisfaction. In a traditional structure, the individual or group
receives a bonus if the model exceeds a given metric, such as accuracy. This is in-
sufficient for the proposed method, as there are two metrics to be optimized against,
which in many cases counteract each other.
Without carefully considering incentives, there are trivially optimal solutions
that can be achieved; for example a model that satisfies all FM rules, but has very
low accuracy. Such outcomes must be disincentivized. To counteract these cases, the
system requires an incentive matrix that aligns the AM and FM to work towards the
21
success of each other. In particular, AM will receive a larger bonus when all FM rules
are satisfied, and FM will receive a larger bonus when all accuracy conditions are
satisfied.
For example, suppose that the traditional bonus for exceeding a minimum ac-
curacy is b, in a regime where there are no fairness considerations. The bonus will
now be expressed as a matrix (Figure 2.1), which incentivizes each actor to design
their piece of the algorithm in order to maximize, to their knowledge, the likelihood
of the other actor achieving their goal. Each element of B is a tuple, where the first
value is the reward for the FM and the second value is the reward for the AM. The
first row of the matrix corresponds to a scenario in which the fairness conditions are
not satisfied, and the second row means that the fairness conditions were satisfied.
Similarly, the first column refers to the accuracy conditions not being satisfied, while
the second column means that the accuracy conditions were satisfied. The JM is
given a bonus for signing off and delivering a completed model in fewer than N steps,
as well as an additional bonus in the case of B11. Formally,
B00 =⇒ No conditions satisfied
B01 =⇒ Fairness not satisfied, accuracy satisfied
B10 =⇒ Fairness satisfied, accuracy not satisfied
B11 =⇒ Both conditions satisfied
B(k) =
(0, 0) ( bk, 0)
(0, bk) (b, b)
(2.1)
Figure 2.4: Bonus Incentive Matrix
22
The matrix B forms a symmetric game over the actions of the two players
AM and FM, where both players have a dominant strategy: the FM should produce
accuracy-preserving transformations, and the AM should implement the minimum
number of required transformations.
With no prior information about the likelihood of either player succeeding in
their respective task, the expected payoff for each player is
E(B(0)) = (0 +b
k) ∗ 1
2=
b
2k(2.2)
E(B(1)) = (0 + b) ∗ 1
2=b
2(2.3)
so that the expected improvement in payoff is
E(B(1)) = k ∗ E(B(0)) (2.4)
This game can also be modified so that each player is negatively incentivized.
Consider the following incentive matrix:
B′(k) =
(0, 0) ( bk,− b
k)
(− bk, bk) (b, b)
(2.5)
Figure 2.5: Negative Incentive Matrix
In this form of the game, the FM loses bk
if accuracy is not satisfied, and the AM
loses bk
if the minimum number of fairness conditions are not satisfied. The expected
bonuses are then:
23
E(B′(0)) = (0 +b
k) ∗ 1
2=
b
2k(2.6)
E(B′(1)) = (− bk
+ b) ∗ 1
2=
(k − 1)b
2(2.7)
Then the expected improvement to the bonus is:
E(B′(1)) = k(k − 1) ∗ E(B′(0)) (2.8)
Algorithm 2.1 Reward Calculation
Input: Xtrain,Xtest,ytrain,ytest,baselineAccuracy,minAccuracyLoss,minFairnessResult: (rfair, racc)model = model.fit(Xtrain,ytrain)X′train,y
′train = resample(Xtrain,ytrain)
fairModel = model.fit(X′train, y′train)conditionsMet = fairModel.computeFairnessConditions()ypred = model.predict(Xtest)acc = computeAccuracy(ytest,predictions)if |acc− baselineAccuracy| < minAccuracyLoss then
if conditionsMet >= minFairness thenrfair = B
elserfair = B
2
end
endelse if conditionsMet >= minFairness then
if |acc− baselineAccuracy| < minAccuracyLoss thenracc = B
elseracc = B
2
end
endelse
racc = rfair = 0end
24
2.6. Appeals
There are two points in the process during which the one group can challenge
the other group’s model and take their bonus. When the AM has converged onto a
model that they believe to be optimal subject to both fairness and accuracy concerns,
then the FM gets one opportunity to make the model more accurate while satisfying
more fairness conditions. If the FM succeeds, then the AM gets a chance to further
outperform them. If the AM cannot improve upon the FM’s improved model, then
the FM receives the full bonus B and the AM receives nothing.
This step in the procedure disincentivizes any actor from simply satisfying
the other actor’s conditions while neglecting their own, in order to reap an easier
reward. For example, the AM could construct a model that satisfies enough fairness
conditions to receive a reward B/2 (Eq. 2.1, above) while depriving the other actor
of any reward. The appeals mechanism gives the FM the chance to create a model
with superior performance, and receive a total reward of B.
25
Chapter 3: Transformations and Metrics
As data transformations will be a key part of this process, it is important to
highlight the transformation methods that can be used, as well as the metrics that
are needed to formalize the notion of distance between a data set and its transformed
version. Supposing that our training data X contains m rows and n columns, we can
say that it is isomorphic to Rm×n, and use a generalization of the Euclidean metric
as a starting off point. Data transformations can be approached from two separate
perspectives. The data itself can be transformed, using computational rules and/or
transformation matrices. Alternatively, the probability distribution which generated
the data can be inferred, and then mapped to a new distribution. This transformed
distribution can then be sampled from to create a new training set. In either case,
there are many metrics to choose from, with a large range of possible outcomes. This
chapter begin with a brief discussion of the notion of distance between elements in
matrices, and then generalize these concepts.
26
3.1. Metrics
For two matrices A,B ∈ Rm×n there are a variety of possible metrics which
provide a notion of distance between them.1 We focus first on the Frobenius metric,
which generalizes the familiar concept of distance in Euclidean space. As the quantity
m ∗ n increases, any change in just one element of A or B will lead to vanishingly
small changes in the distance.
Definition 12. Frobenius (Euclidean) p-metric
For m× n matrices A = (aij) and B = (bij), dp(A,B) = p
√n∑
i=1
n∑j=1
(aij − bij)p
Given two data sets X1 and X2, it may not always be possible or desirable
to require that they have the same features columns appearing in the same order.
For the purposes of comparing a new data set with previously evaluated data sets,
there can be an advantage to finding patterns and structural similarities with data
sets from other fields. This need motivates the following definition, which generalizes
the notion of distance to data sets from different sources, by examining the closest
distances between the individual columns in each data set.
Definition 13. Sorted Distance For two data sets X1, X2 ∈ Rm×n and a distance
metric d : Rm × Rm −→ R, the Sorted Distance first maps X1 −→ X ′1 and X2 −→
X ′2, where X ′1 and X ′2 have the same values as X1 and X2, with columns sorted in
descending order. The value of the distance is then d(X ′1, X′2)
Definition 14. Sorted and Normed Distance For two data sets X1, X2 ∈ Rm×n
and a distance metric d : Rm × Rm −→ R, the Sorted and Normed Distance first
1Until now, we have considered the training data to be a subset of Rm×n, which is only guaranteedto be true after feature selection and processing. For example, a column of data with enumeratedtype and options [A, B, C] will become a column of type integer, with options [1,2,3]. For theremainder of this paper, we will assume that training data is already in such a processed form.
27
normalizes each column X i1 ←− X i
1/‖X i1‖ and X i
2 ←− X i2/‖X i
2‖ ∀i < n, and then sorts
the columns and returns the distance, as in Definition 13.
Definition 15. Minimum Column Distance
For two data sets X1, X2 ∈ Rm×n and a distance metric d : Rm × Rm −→ R, the
Minimum Column Distance is the sum ‖X1 − X2‖ =∑i,j∈S
d(X i1, X
j2) where S is the
set of index pairs (i, j) that minimize the column distance: d(X i1, X
j2) < d(X i
1, Xk2 )
∀k ∈ {0, ..., n}, k 6= j
These definitions can lead to different values of distance over the same pair
of data sets. Consider X1 and X2 where each column is a series of samples from
N (0, 1). According to the Frobenius Metric, dp(X1, X2) = 141.18, since this measure
only captures the agreement in value of each pair of elements in the data sets.
Metric Sorted Normed Distance
Frobenius N N 141.18
Min. Column N N 137.30
Frobenius Y N 8.95
Min. Column Y N 5.14
Frobenius Y Y 0.09
Min. Column Y Y 0.05
Figure 3.1: Distance comparisons for data sampled from N (0, 1)
Sorting and normalizing the data sets before comparing them results in dis-
tance values that more closely reflect the underlying statistics of the data, rather than
the similarity of individual elements. In cases where there is a need to calculate the
similarity of the overall properties of the data sets, there can be an advantage in fitting
distributions to the data, and measuring the distance relative to the distributions.
28
3.1.1 Metrics on Statistical Distributions
An alternative approach is to consider each column as a series of samples
from a statistical distribution with parameters Θ0, ...,Θn−1. Provided that a suitable
distribution can be fit for each column, then the problem of computing distance be-
comes one of distances between statistical distributions. The Bhattacharyya Distance
provides a notion of similarity between distributions, while the Matusita Distance
provides a measure of difference.
Definition 16. Bhattacharyya Distance For statistical distributions F1, F2 with
probability distribution functions p1 and p2, the Bhattacharyya Measure is DB(F1, F2) =∫ √p1(x)
√p2(x)dx
Definition 17. Matusita Distance
For statistical distributions F1, F2 with probability distribution functions p1 and p2,
the Matusita Measure is DM(F1, F2) =∫
(√p1(x)−
√p2(x))2dx
The Matusita Measure is related to the Bhattacharyya Measure via the equa-
tion
DM(F1, F2) = 2− 2DB(F1, F2) (3.1)
Figure 3.2: The relationship between DM and DB
(Matusita, 1955)
This relation shows that minimizingDM(F1, F2) is equivalent to maximizingDB(F1, F2).
While the Matusita Measures quantifies how “far” the distributions are from one an-
other, the Bhattacharyya Measure compares the closeness (or affinity) of the two
29
distributions. Returning to the example data set from Figure 3.1, we have p1 = p2 =
N (0, 1), and both of these measures simplify:
DB(X1, X2) =
∫ √N (0, 1)
√N (0, 1)dx =
∫N (0, 1)dx = 1 (3.2)
DM(X1, X2) = 2− 2DB(X1, X2) = 0 (3.3)
Figure 3.3: Distance and Similarity for data sampled from N (0, 1)
As expected, the measures DB and DM give inverse indicators of the same relation-
ship: X1 and X2 are perfectly similar by statistics (DB) and have metric distance 0
(DM).
3.2. Transformations
Definition 18. Transformation
A Transformation τ : Rm×n −→ Rm×n maps a data set into another data set of the
same dimension. For a distance metric d, the distance from X ∈ Rm×n to τ(X) is
given by d(X, τ(X)) and formalizes the notion of difference between the original data
set and its transformed counterpart.
Under the Bonus Incentive Matrix, the FM will seek to construct transfor-
mations such that the quantity d(X, τ(X)) is minimized, in order to maximize the
possibility of preserving the accuracy score relative to the original data. The FM
will need to decide which transformations to construct, and which metrics to use to
measure distance between the original and transformed data sets.
Regardless of the metric used, transformations will not generally commute.
This means that the data transformations will need to be performed in a variety of
30
orders to test the effect on the final model. For example, oversampling one class in
the data and then trimming extreme values will not, in general, give the same result
as trimming the extreme values and then oversampling. Further discussion of this
non-commutativity is found in 4.3.
3.2.1 Statistical Transformations
Suppose that each column in a data set X is sampled independently, so that
the ith column of X ∈ Rm×n is drawn from a distribution Pi(θi0, ..., θ
ik−1). Then
X ∼ (P0(θ00, ..., θ
0k−1), ..., Pn−1(θ
n−10 , ..., θn−1k−1)) and the initial parameter matrix Θ can
be written:
Θ0 =
θ00 · · · θ0n−1
.... . .
...
θ0k−1 · · · θn−1k−1
(3.4)
Let Θideal denote the matrix of parameters such that d(X, t(X)) is minimized
and C(t(X)) = 1 for all transformations t associated with the Condition C. The ith
column of Θideal is the parameters for the distribution that best fits the data in the
ith column of X. Given the current parameters Θ and the ideal parameters Θideal, we
can transform the parameters via
τ : Rk×n −→ Rk×n (3.5)
which is a lower dimensional transformation than one operating on X directly. Then
for a series of computational conditions (C, τ0, ..., τn) which impose restrictions on
the statistical properties of one or multiple columns, each τi must only operate on
the lower dimensional space Rk×n and the transformed data X ′ can be reconstructed
from the final parameter matrix Θf = τin(τin−1(· · · τ0(Θ0))), where each column is
31
Chapter 4: Simulations
In the absence of human trials, simulations are necessary to show this approach
converging onto ideal solutions. A key part of the simulation is selecting which fairness
conditions and which hyperparameters to use for each step. The algorithms for each
are outlined in the following section. These simulations are not necessarily designed
to mimic the thoughts and actions of a collection of human actors; rather, they serve
to illustrate the interaction between the incentives between the different groups.
4.1. Algorithms
The fairness vector vfair tells the model which fairness conditions to test for,
and which to ignore. At each step in the simulation, a new vfair is drawn that is
near the previous vfair relative to some metric. For modeling purposes, we have been
using the Hamming Distance, to guarantee that each step only changes the value of
one dimension of vfair.
Algorithm 4.1 Fairness Vector GenerationInput: vfair, MResult: vfair ∈ {0,1}dB = {x|x ∈ {0,1}d,d(x,vfair) = 1,
∑k−1i=0 xi ≥M}
vfair ∼ U(B)
The action space for AM is more recognizable from a traditional grid search.
Each dimension in vacc corresponds to a parameter, mapped to an integer value. The
33
AM receives training data which has already been modified according to the values
present in vfair. We then proceed to iterate over the space like so:
Algorithm 4.2 Accuracy Vector Generation (Hyperparameters)
Input: vacc
Result: vacc ∈ Zk
B = {x|x ∈ Zk,d(x,vacc) = 1}vacc ∼ U(B)
The reward calculation process involves comparing the accuracy of the model
trained on unmodified training data (the baseline accuracy) with the accuracy after
applying the fairness conditions. The AM tries to minimize the accuracy loss while
maximizing the number of fairness conditions satisfied.
4.2. Iris Simulation
The Iris Data Set was first described by Fisher in the illustration of taxonomic
problems (Fisher, 1936). It has since become a standard data example in educa-
tional materials when highlighting a particular algorithm or data analysis technique.
Partially for this reason, we have chosen this data set for the first simulation of the
separation of powers mechanism.
The Iris Data Set (N = 150) has 3 classes with 50 samples each, corresponding
to 3 species of Iris flowers: Iris Setosa, Iris Virginica, and Iris Versicolor. There are
4 features: sepal length, sepal width, petal length, and petal width. In several of the
dimensions, Iris Setosa clusters distinctly from the other two species. For the sake of
this simulation, we will combine Iris Virginica and Iris Versicolor into one class, and
consider Iris Setosa to be a less ideal label for the purposes of fairness definitions.
34
Figure 4.1: Iris Data with 2 Classes
As can be seen in the pair plot, there are several dimensions in which the Iris
Setosa species forms a distinct cluster from the rest of the data. This is especially
highlighted in the bottom row, where the y axis represents petal width.
35
sepal length sepal width petal length petal width
count 150 150 150 150
mean 5.843333 3.057333 3.758000 1.199333
std 0.828066 0.435866 1.765298 0.762238
min 4.300000 2.000000 1.000000 0.100000
25% 5.100000 2.800000 1.600000 0.300000
50% 5.800000 3.000000 4.350000 1.300000
75% 6.400000 3.300000 5.100000 1.800000
max 7.900000 4.400000 6.900000 2.500000
Figure 4.2: Summary statistics for the Iris Data Set
sepal length sepal width petal length petal width
count 50 50 50 50
mean 5.00600 3.428000 1.462000 0.246000
std 0.35249 0.379064 0.173664 0.105386
min 4.30000 2.300000 1.000000 0.100000
25% 4.80000 3.200000 1.400000 0.200000
50% 5.00000 3.400000 1.500000 0.200000
75% 5.20000 3.675000 1.575000 0.300000
max 5.80000 4.400000 1.900000 0.600000
Figure 4.3: Summary statistics for Iris Setosa
36
sepal length sepal width petal length petal width
count 100 100 100 100
mean 6.262000 2.872000 4.906000 1.676000
std 0.662834 0.332751 0.825578 0.424769
min 4.900000 2.000000 3.000000 1.000000
25% 5.800000 2.700000 4.375000 1.300000
50% 6.300000 2.900000 4.900000 1.600000
75% 6.700000 3.025000 5.525000 2.000000
max 7.900000 3.800000 6.900000 2.500000
Figure 4.4: Summary statistics for other species (Iris Data)
4.2.1 Constitution
In this hypothetical scenario, we imagine a group of individuals representing
the different parts of the organization, coming together to decide on which tenets will
be written into the constitution. Since we have specified that the label Iris Setosa is
considered less desirable, we must construct tenets that will protect individuals from
being erroneously labeled as such. For this “constitutional convention”, the following
notation is used:
• X - The set of all potential training data
• Xsetosa - The subset of the training data labeled Iris Setosa
• Xother - The subset of the training data labeled other
• µdimlabel - The mean value of column dim for data with label label
Assuming we have prior research demonstrating that the statistical distribu-
tion of Iris Setosa plants is wider than our sample, someone proposes that we should
37
have a global condition specifying how far apart the two subsets in the data set can
be. They propose a tenet that the mean sepal widths (sw) and mean petal lengths
(pl) must be “close together”, which they define as
|µswsetosa − µsw
others| ≤ ε (4.1)
|µplsetosa − µ
plothers| ≤ ε (4.2)
Figure 4.5: Tenet 1 (Iris Data)
Someone else raises the issue of class balance: they claim that in the wild,
Iris Setosa is actually closer to 50% of the observed Iris flowers in the area. They
argue that, knowing this, the training data should reflect the reality of a 50-50 split
between Iris Setosa and the other species. They then propose a tenet mandating
class balance:
|Xsetosa| − |Xothers| ≤ ε (4.3)
Figure 4.6: Tenet 2 (Iris Data)
Finally, someone raises the issue of individual outliers. Since the petal width
in the Iris Setosa sample appears to be much smaller than the reality, they describe a
tenet which guarantees that the widest Iris Setosa be close in width to the narrowest
sample from the rest of the data. This tenet guarantees closeness of the extreme
38
values, but does not transform the underlying statistics as dramatically as Tenet 1
(Figure 4.5).
|max(Xpwsetosa)−min(Xpw
others)| ≤ ε (4.4)
Figure 4.7: Tenet 3 (Iris Data)
Following the proposal of the three tenets, the group votes to confirm each of
them, and produces the following constitution. Note that the revisions clause was
not explicitly mentioned during the tenet voting process, as it is a central feature of
the document and allows for revisions on-the-fly as problems are revealed that impact
the majority of the organization.
39
Preamble: Recognizing that our training data does not repre-
sent ground truth, and that the action of labelling may lead to
disparate impact, we are laying out the following tenets which
will govern our model creation process. For all definitions of
“closeness”, we will use a parameter ε, to be defined before
implementation
Tenet 1: Class means must be close together.
Tenet 2: Classes must be equally balanced.
Tenet 3: The widest Iris Setosa sample must be close to the
smallest sample from the rest of the data
Revisions: Should an oversight or flaw be found in these
tenets during the model creation process, any member of the
organization may solicit support from others to call a vote to
amend. A simple majority is needed to call the vote to order,
and a 2/3 majority is needed to accept the change.
Figure 4.8: Iris Data Set Constitution
4.2.2 Fairness Conditions
For the purposes of this study, we have constructed a constitution based on
our knowledge that the sampling of the Iris Setosa was not representative, and in
fact the two species are more closely related than the data shows. The task of the FM
is then to define a set of conditions and associated transformations that will make
those conditions true in the training data. The AM will then have the choice of which
combination of transformations to use in order to honor each of the tenets which were
previously accepted.
40
4.2.3 Tenet 1: Similar Means
Recall that Tenet 1 requires |µswsetosa − µsw
others| ≤ ε and |µplsetosa − µ
plothers| ≤ ε.
The FM then proposes Transformation 1, which modifies the Iris Setosa data, and
Transformation 2, which modifies the rest of the data. In the following algorithms,
ε is the value agreed upon during adoption of the constitution to define the concept of
“closeness”, and α is a scaling factor which is used to bring the data into agreement.
Input: Xsetosa, α, ε
for x in Xsetosa do
xpl = xpl + |ε− α ∗ µplothers|
xsw = xsw − |ε− α ∗ µswothers|
end
Figure 4.9: Transformation 1 (Iris Data)
Input: Xothers, α, ε
for x in Xothers do
xpl = xpl − |ε− α ∗ µplsetosa|
xsw = xsw + |ε− α ∗ µswsetosa|
end
Figure 4.10: Transformation 2 (Iris Data)
41
4.2.4 Tenet 2: Class Balance
To oversample the training data with equal class sizes, we give the option to
use one of two approaches. The first method is a Random Oversampler, which is a
more naive approach. This method samples with replacement from the minority class,
leading to duplicate data points. This method is computationally straightforward,
but does not capture information about the statistical distribution of the data apart
from the points which are already in the sample, so sampling biases and the effects
of missing will be magnified.
The second method is the Synthetic Minority Oversampling Technique (SMOTE)
(Chawla, Bowyer, Hall, & Kegelmeyer, 2002), which chooses random points from the
minority class and generates new data between those points and their nearest neigh-
bors. SMOTE is a commonly used method for achieving class balance in previously
unbalanced data sets, and has also formed the foundation for a variety of additional
oversampling methods (Douzas, Bacao, & Last, 2018). For this study, an unmodified
version of SMOTE was used, as detailed in the original publication of the algorithm.
4.2.5 Tenet 3: Closeness Between Minima and Maxima
This tenet is only concerned with the difference in value between the widest
instance of Iris Setosa and the thinnest instance of another species. Since this condi-
tion is local in nature, it only requires a transform to modify one data point in order
to be satisfied. We propose two transforms which satisfy this condition; one trans-
formation adds a wider Iris Setosa, while the other transformation adds a thinner
sample from the other species.
42
Input: X, ε
i = arg minXpwother
j = arg maxXpwsetosa
Xpwother[i] = Xpw
setosa[j] + ε
Figure 4.11: Transformation 1 (Iris Data)
Input: X, ε
i = arg minXpwother
j = arg maxXpwsetosa
Xpwsetosa[j] = Xpw
other[i]− ε
Figure 4.12: Transformation 2 (Iris Data)
We have now defined 3 tenets, each of which have 2 possible transformations
to be used. Assuming we want the final process to satisfy all 3 tenets, we must pick
one transformation for each of the 3 tenets and perform them in an order so as to
maximize the accuracy.
4.2.6 Experimental Setup
For the Iris Data Set, we performed an exhaustive search over the possible data
transformations, in order to highlight the potential changes in accuracy depending
on the order of the transforms. We begin with the data structures and terminology
that were used for this part of the simulation.
43
We will track the order of transformations using the following notation. For
a series of Computational Conditions (Definition 7), each step s = (Ci, τj) in the
process is defined as performing the jth transformation τj of Ci in order to satisfy a
condition Ci. The transformation history is then a list
S = [(Ci0 , τj0), ..., (Cj0 , τjn)] (4.5)
denoting which transformation was performed to satisfy each of the conditions C0, ..., Cn.
Our goal is then to find a sequence S = [(Ci0 , τj0), ..., (Cj0 , τjn)] such that accuracy is
minimally impacted.
4.2.7 Data Transformation Steps
For the sake of example we will choose the transformation path [(2, 1), (3, 1), (1, 1)].
Before performing any transformations, the data appears like so:
44
Figure 4.13: Before the transformations (Iris Data)
The team takes the first step: Condition 2, Transform 1, which is a SMOTE
oversample. The shape of the data has not changed, though the Iris Setosa points
are now more dense.
45
Figure 4.14: After Step 1 (Iris Data)
The team then takes the second step: Condition 3, Transform 1, which adds
points so that the widest Iris Setosa data point is close to the narrowest data point
from the other class.
46
Figure 4.15: After Step 2 (Iris Data)
Finally, the team takes the last transform: Condition 1, Transformation 1.
These transformation shifts the mean of the Iris Setosa data to be close to the mean
of the other data.
47
Figure 4.16: After Step 3 (Iris Data)
At this point, the team has a training data set that satisfies all three tenets
of their constitution: the means of the two classes are close, the classes are balanced,
and the minima and maxima of the two classes are close. While it appears from these
two dimensions that the data is no longer clearly separable, a full pairs plot of the
other dimensions shows that there are still clean separations between the classes, and
in fact the accuracy has not been harmed at all in the process.
48
4.2.8 Transformed Data
The following tables show the summary statistics for the Iris Data Set after
undergoing the transformations specified in the previous sections. Note that, in the
transformed data, N = 201 due to resampling to satisfy Tenet 2 and the addition of
one data point to satisfy Tenet 3.
sepal length sepal width petal length petal width
count 201.000000 201.000000 201.000000 201.000000
mean 5.626980 2.822254 4.856254 0.960158
std 0.818997 0.346194 0.593738 0.779370
min 4.300000 1.624724 3.000000 0.100000
25% 5.000000 2.524724 4.640343 0.200000
50% 5.485952 2.800000 4.840343 1.000000
75% 6.300000 3.024724 5.001836 1.600000
max 7.900000 3.800000 6.900000 2.500000
Figure 4.17: Summary statistics for the Iris Data Set after preprocessing (Iris Data)
49
sepal length sepal width petal length petal width
count 101.000000 101.000000 101.000000 101.000000
mean 4.998247 2.773000 4.807000 0.251403
std 0.322563 0.353778 0.159252 0.128155
min 4.300000 1.624724 4.340343 0.100000
25% 4.784979 2.524724 4.740343 0.200000
50% 5.000000 2.724724 4.840343 0.200000
75% 5.179357 3.024724 4.914403 0.300000
max 5.800000 3.724724 5.240343 1.000000
Figure 4.18: Summary statistics for Iris Setosa after preprocessing
sepal length sepal width petal length petal width
count 100.000000 100.000000 100.000000 100.000000
mean 6.262000 2.872000 4.906000 1.676000
std 0.662834 0.332751 0.825578 0.424769
min 4.900000 2.000000 3.000000 1.000000
25% 5.800000 2.700000 4.375000 1.300000
50% 6.300000 2.900000 4.900000 1.600000
75% 6.700000 3.025000 5.525000 2.000000
max 7.900000 3.800000 6.900000 2.500000
Figure 4.19: Summary statistics for other species after preprocessing
4.3. Results
Of the 48 possible transformation histories that satisfied tenets 1, 2, and 3,
there was a wide variety of accuracy impact. The majority of the transformation
50
combinations yielded a drop in model accuracy of more than 20%, and 16 out of 48
combinations led to a decrease in accuracy of less than 10%. For the full results, see
A.1.
Figure 4.20: Model accuracy distribution (Iris Data)
One of the challenges with this exhaustive search is the size of the potential
solution space. In this example, we have 3 conditions, each of which have 2 trans-
formations to choose from. This means that there are 3 ∗ 2 = 6 potential values of
(Ci0 , tj0), (3 − 1) ∗ 2 = 4 potential values for (Ci1 , tj1), and (3 − 2) ∗ 2 = 2 poten-
tial values for (Ci2 , tj2). In total, there are 6 ∗ 4 ∗ 2 = 48 possible transformation
histories which satisfy all 3 tenets. As the number of conditions c and transforma-
tions per condition t grows, this scales according to c! ∗ t, which is quickly dominated
51
by the factorial term. In practice, 5! ∗ 2 = 240 combinations may be tenable, but
10! ∗ 2 = 7257600 may not be. This may serve as an effective limit on the number of
adopted tenets, and should be kept in mind during the constitutional voting process.
In this case, there were multiple combinations of transformations which yielded
less than 10% decrease in accuracy, and these combinations were testable within a
matter of minutes. This was achievable mainly due to the separable nature of the data
- in several dimensions, the data cleanly separates into two distinct classes. Even after
oversampling and transforming the data, the classes still remained clearly separable.
Although the Iris Data Set serves as a useful illustrative example, in general, this will
not be the case.
From the point of view of the AM/FM paradigm, the model building team
would have multiple avenues to achieve full fairness compliance without sacrificing
accuracy, and both teams would receive the maximum bonus.
52
Chapter 5: COMPAS Simulation
We now turn our attention to a real world data set, in order to illustrate the
AM/FM mechanism in use on a domain which has received scrutiny in the past several
years for issues of bias and discrimination. This data was first made available to the
public as part of ProPublica’s investigative report into the use of risk assessment tools
during sentencing and parole decisions. (Angwin et al., 2016)
5.1. Data
The training data consists of 10291 rows, each representing an individual who
has been assigned a risk score by the COMPAS system. Pre-processing was per-
formed to remove duplicate rows, and encode the severity of the charge degrees, with
the following values: {F1, F2, F3,M1,M2} where F means felony and M means mis-
demeanour. The number corresponds to the seriousness of the crime, where a lower
number means more severe than a higher number. For example, F3 is more severe
than M1, and M1 is in turn more severe than M2.
The attributes are as follows:
53
Attribute Name Data Type Example
Sex enum M, F
Age int 24
Race enum African-American, Caucasian
Charge Degree enum F3, M1
Juvenile Felony Count int 0
Juvenile Misdemeanour Count int 0
Juvenile Other Count int 0
Priors Count int 0
Recidivated binary T, F
Violent Recidivated binary T, F
Decile Score float 2.4
Table 5.1: COMPAS data schema
54
5.2. Constitution
We imagine representatives from across an organization meeting to build a con-
stitution before proceeding to build a model for recidivism prediction. The represen-
tatives are tasked with agreeing upon several tenets and signing off on a constitution
that will guide the development of the model.
The discussion begins after everyone has examined the test data. Someone
brings a topic to the table, that the recidivism rate differs by race. The individual is
concerned that this could reflect over-policing and racially disparate outcomes in the
legal system, rather than a difference in rate of crimes committed. This point brings
objections from others, who feel that the data is reflecting reality, and modifying it
would amount to changing the fundamental behavior of the model. Another member
of the group explains that recidivism is defined as being rearrested and re-convicted
of a crime, but does not guarantee that the crime was actually committed. Con-
versely, committing another crime does not guarantee being arrested and convicted,
so operating solely on this feature would amount to predicting the very biases of the
existing law enforcement system. After a long discussion, the original objector agrees
to support a tenet that brings more racial balance to the recidivism rate, to account
for individuals who were never rearrested, and those who convicted though innocent.
The group agrees to propose Tenet 1, which says that the recidivism rate between
racial groups must be different by no more than ε.
|Xrecidothers| ≤ |Xrecid
african−american|+ ε (5.1)
Figure 5.2: Tenet 1: Recidivism Balance
56
African-American 5138Caucasian 3562Hispanic 939Other 570Asian 51Native American 31
Figure 5.3: Racial Demographics (COMPAS Data)
During the discussion about Tenet 1, someone in the group looks at the data
and notices that the racial distribution in the data is not representative of the general
population. This seems like a problem to them - what if the model learns to expect
one racial group more than others, despite the true distribution in the population?
The group looks at the numbers, and recognizes that they are not in line with the
Census:
Tenet 2 is then presented to the group, stating that the racial makeup of the
training data must match the national numbers according to the U.S. Census, up to
a tolerance ε.
|Xafrican−american| ≤ |X|+ ε (5.2)
Figure 5.4: Tenet 2: Racial Demographic Parity
A third objection is raised, based on the previous discussion about demo-
graphic parity. Someone who notices that sex is also unbalanced in the training set.
There are almost 4 times as many males in the sample as females, instead of an equal
number, as expected.
57
Male 8187
Female 2104
Figure 5.5: Individuals counted by sex (COMPAS Data)
A discussion ensues, during which many argue that such an imbalance by sex
could lead to the model being more accurate for males, or potentially discounting
female individuals. There is broad support for the concept of demographic balance
in terms of sex, though not for a particular implementation. Some argue that exactly
50% of the test data needs to be female, while others argue for a tolerance of several
percent. In the end, the group settles on a requirement that the sex balance must be
within ε = 0.01:
|Xmale| ≤ |Xfemale|+ ε (5.3)
Figure 5.6: Tenet 3: Sexual Demographic Parity
Another consideration is raised, about age discrimination. Someone in the
group recently read a paper about the disparate treatment of people under 25 in
recidivism prediction, and is concerned that the model will perpetuate this bias.
Summary statistics are provided to the group:
58
min age max age count
17 33 5679
34 49 3050
50 65 1389
66 80 170
81 96 3
Figure 5.7: Age distribution (COMPAS Data)
Many others object to this proposition, arguing with research that younger
people can be more impulsive than older adults. There are also concerns that adding
more than three tenets will make the model building process untenable. A vote shows
that there is insufficient support for this tenet, and it is dropped from consideration.
After agreeing on three tenets and rejecting a fourth, the constitution is
adopted with the following wording:
59
Preamble: Recognizing the sensitivity of recidivism predic-
tion and the biases present in data from the criminal justice
system, we are laying out the following tenets which will gov-
ern our model creation process. For all definitions of “close-
ness”, we will use a parameter ε, to be defined before imple-
mentation
Tenet 1: The recidivism rates across racial groups must be
close
Tenet 2: The racial demographics in the data must be close
to the numbers from the U.S. Census
Tenet 3: The count of females must be close to the count of
males
Revisions: Should an oversight or flaw be found in these
tenets during the model creation process, any member of the
organization may solicit support from others to call a vote to
amend. A simple majority is needed to call the vote to order,
and a 2/3 majority is needed to accept the change.
Figure 5.8: COMPAS Data Set Constitution
5.3. The Importance of Hyper Parameters
The following is a comparison of disparate outcomes by race for a Support
Vector Regressor (SVR) trained on COMPAS data with Radial Basis Function (RBF)
kernel with degree=3, ε=0.1. These are the default values used by scikit-learn, and are
used to illustrate the outcome when naively implementing a model without specifying
parameters. In this table, AA means African-American, NCC means Non-Caucasian,
60
and CC means Caucasian.
The free parameter is regularization C which is inversely related to the size of
the hyperplane defined in the SVR. For small values of C we expect more generaliz-
ability at the cost of misclassifying more points, while a larger value of C will result
in a smaller hyperplane and a higher risk of overfitting.
C µAA σAA µNCC σNCC µCC σCC Mean
Err
Stdev
Err
0.0005 3.2555 0.2642 3.2339 0.2722 3.1350 0.2864 -0.5059 2.3321
0.0010 3.2463 0.4924 3.2100 0.5054 3.0087 0.5499 -0.7661 2.3056
0.0050 3.4345 1.1819 3.3075 1.2120 2.8458 1.2760 -0.6223 1.9734
0.0100 3.5301 1.3895 3.3793 1.4264 2.9754 1.4297 -0.6242 1.9404
0.1000 3.8984 1.8008 3.6240 1.8370 2.9659 1.6894 -0.4692 1.8367
0.5000 4.0009 1.9160 3.6672 1.9715 2.8369 1.7899 -0.4211 1.7892
0.7500 4.0849 1.9543 3.7327 1.9996 2.8396 1.7776 -0.4651 1.8105
1.5000 4.0833 1.9652 3.7439 2.0245 2.8243 1.7452 -0.3672 1.7643
5.0000 4.2230 1.9901 3.8052 2.0458 2.8179 1.7980 -0.3135 1.7699
10.0000 4.0674 2.0459 3.7244 2.0755 2.7829 1.7956 -0.3502 1.7845
Figure 5.9: Summary statistics by race (COMPAS Data)
We can see that the absolute mean error is minimized for C = 5.0, however
this leads to a mean risk score difference of µAA − µC = 4.2230 − 2.8179 = 1.4051,
which is not optimal. If we were to only use µAA−µC as our minimization objective,
we would find that the value C = 0.0005 leads to a minimum value of 0.1205; in other
words the disparate treatment by race is more than 11 times higher based solely on
the choice of C.
61
The mean error and mean score difference are equalized near C = e−4 = 0.018
however this leads to a model with mean error approximately 2 times higher than the
minimum, and mean score difference approximately 5 times higher than the demon-
strated minimum. It follows that hyper parameter manipulation subject to a mixed
objective function will not result in a model that is ideal in both regards.
Figure 5.10: Trade off between mean error and disparate impact by race (COMPAS
Data)
The main action of the AM will be parameter manipulation with the end
goal of simultaneously satisfying fairness conditions and accuracy minimums. As
demonstrated, the choice of hyper parameters can have an enormous impact on the
behavior of the final model, and naively using default values can cause significant
62
harm when the model is making recommendations about limiting an individual’s
freedom.
5.4. Penalties and Disparate Outcomes
The penalty (loss) function is one of the fundamental characteristics of a model.
The optimization process is performed in order to minimize the value of this func-
tion. The penalty name is sometimes even present in the algorithm name, taking for
example “Least Squares Regression” which minimizes the squared residuals of the
function.
While we would expect the choice of penalty to strongly affect the outcome
of the model, we find weaker results when the results are averaged over multiple
trials. For Stochastic Gradient Descent Regressor (SGDR) models trained on the
COMPAS data, with potential penalties l1, l2, and elasticnet, we find little change in
the difference in average score between African American and Caucasian individuals.
63
Figure 5.11: Minimal change in mean scores for different penalty types (COMPAS
Data)
5.5. Experimental Setup
To simulate the creation of a COMPAS model in the AM/FM paradigm, we
reformulate the situation as a classification problem and use simulated annealing over
the choice of fairness transforms and hyperparameters. The idea is that simulated
annealing “cools off” over time and becomes less likely to make changes in the values,
similar to the team learning more about the data and domain and becoming more
confident in their choice of transforms and hyperparameters. For the purposes of the
simulation, we predict a label of 0, 1 for each individual, where 0 means the individual
will not be convicted of another crime, and 1 means that they will.
64
Input: X, model, nsteps, conditions, params
X ′ ←− X
Xtrain, ytrain, Xtest, ytest = split(X ′, y′)
model = model(params).f it(Xtrain, ytrain)
accbaseline = accuracyscore(model.predict(Xtest), ytest)
costold = 0
for i in nsteps do
for j in conditions.size do
X ′ = conditions[j].transform(X ′)
end
y′ = X ′.pop(isrecid)
X ′train, y′train, X
′test, y
′test = split(X ′, y′)
model = model(params).f it(X ′train, y′train)
accnew = accuracyscore(model.predict(X ′test), y′test)
costnew = accnew − accbaseline
u ∼ U(0, 1)
if exp (costold − costnew)/t > u then
fairstep(conditions)
accstep(params)
costold ←− costnew
end
end
return
Figure 5.12: Simulated Annealing of AM/FM
65
Input: fvec
op ∼ U(0, 1)
if op == 0 then
{Swap order of operations}
indices ∼ U(0, fvec.size, 2)
a = fvec[indices[0]]
b = fvec[indices[1]]
fvec[indices[0]] = b
fvec[indices[1]] = a
end
else
{Change transform on condition}
index ∼ U(0, fvec.size)
val = fvec[index]
val[1] + +
val[1] = val[1] mod 2
fvec[index] = val
end
Figure 5.13: fairstep: Fair State Step
66
Input: params
k ∼ U(params)
{Have a small but finite chance to not step the parameter}
step ∼ P ([−1, 0, 1]), P (−1) = P (1) and P (0) << P (1)
params[k] = params[k] + step
Figure 5.14: accstep: Accuracy State Step
5.6. Fairness Conditions
5.6.1 Tenet 1
In order to make the recidivism rates closer across different racial groups, the
following two transformations are made available to the model building team (AM).
Input: Xother, α, ε
raa = |Xrecidaa |/|Xaa|
rother = |Xrecidother|/|Xother|
u = raa ∗ ‖Xother‖ − ‖Xrecidother‖
rows[u] ∼ U(Xrecidother)
for i in rows do
Xrecidother[i][recid] = True
end
Xother ←− Xrecidother + (Xother −Xrecid
other)
Figure 5.15: Transformation 1 (COMPAS Data)
67
Input: Xaa, α, ε
raa = |Xrecidaa |/|Xaa|
rother = |Xrecidother|/|Xother|
u = rother ∗ ‖Xaa‖ − ‖Xrecidaa ‖
rows[u] ∼ U(Xrecidaa )
for i in rows do
Xrecidaa [i][recid] = False
end
Xaa ←− Xrecidaa + (Xaa −Xrecid
aa )
Figure 5.16: Transformation 2 (COMPAS Data)
5.6.2 Tenet 2
Input: X
y = X.pop(race)
{Oversample to balance racial groups}
X ′, y′ = smote(X, y)
X ′[race] = y′
return X′
Figure 5.17: Transformation 1 (COMPAS Data)
68
Input: X
{Demographic balance should be about 17% of the population}
rows = ‖Xaa‖/0.17
Xnew ∼ U(Xother, rows)
X ′ = X +Xnew
return X′
Figure 5.18: Transformation 2 (COMPAS Data)
5.6.3 Tenet 3
Input: X
y = X.pop(sex)
X ′, y′ = smote(X, y)
X ′[sex] = y′
return X′
Figure 5.19: Transformation 1 (COMPAS Data)
69
Input: X
rows = ‖Xfemale‖
Xnew ∼ U(Xmale, rows)
X ′ = Xfemale +Xnew
return X′
Figure 5.20: Transformation 2 (COMPAS Data)
5.7. Results
The simulation was performed with 3 fairness conditions, each of which have
2 potential transformations, and a support vector classier (SVC) with a radial basis
function kernel and model parameter C. A maximum accuracy loss of 2% was allowed.
After 200 iterations, the simulation found 18 configurations that satisfied both the AM
and FM conditions, representing 9% of the configurations. These 18 configurations
lead to both the AM and FM receiving the maximum bonus b (Figure 2.4).
70
Figure 5.23: Accuracy difference per step (COMPAS Data)
5.7.1 Step Count Estimation
A core concern will be the number of steps required to find an optimal solution,,
such that all fairness and accuracy conditions are satisfied and the bonus is maximized
for all parties. To simulate this, 100 trials were run on a sample of the COMPAS data
(N = 6000) with a maximum of 50 steps per trial. Each trial began with a randomly
selected sequence of data transformations and hyperparameters C ∼ N (2, 2). The
hyperparameters were then iterated over with a step size of ∆C = 2.5. The number of
steps needed to achieve the maximum bonus was tracked for each trial, and is plotted
below. Trials that did not find an optimal solution within 50 steps were marked as
73
failures. In total, 79 of the trials found a maximum bonus point within the first 50
iterations, and 21 did not. Of the 100 trials, 46 required fewer than 5 steps to find
a maximum bonus, and 40 achieved a maximum bonus on the first step. On a 2019
Macbook Pro with a 2.3 GHz processor and 8 GB of RAM, this simulation took 2.04
hours.
steps
count 79.000000
mean 9.822785
std 13.921722
min 0.000000
25% 0.000000
50% 1.000000
75% 16.500000
max 46.000000
Figure 5.24: Step count summary statistics (COMPAS Data)
74
Chapter 6: Conclusion
This thesis has described a novel mechanism for the creation of fair machine
learning models, using structures from the theory of democratic governance. In par-
ticular, a separation of powers approach was taken to distribute the power and re-
sponsibility into the hands of more groups acting semi-autonomously. Separation of
powers is an idea with roots in the Enlightenment, during which time there was ex-
tensive thinking about how to guarantee natural rights and ensure that single actors
or groups no longer had supreme power over others.
The first step in the mechanism is a constitutional convention, during which
representatives from across the organization convene to propose and vote on a series
of tenets to be honored by the model. This step minimizes the likelihood of one
individual or faction imposing their will or beliefs onto the rest of the organization.
Several voting systems were proposed, including a simple majority system, as well as
a ranked choice voting system. Once these tenets are agreed upon and formalized into
a constitution, the FM team translates the tenets into computational conditions: a
boolean function that tests if the condition is true, and a series of data transformations
which make the condition true. The AM can select transformations as needed in any
combination in order to maximize the model’s accuracy score.
An incentive system was constructed to provide each party with a bonus incen-
tive to perform their role. Crucially, the bonus of each party was tied to the success
of the other party, to ensure that the two parties viewed their success as part of a
76
larger whole. The incentives were constructed so that the AM always implements
the required fairness transformations, and the FM constructs the transformations to
maximize the accuracy potential of the model.
A theoretical framework for transformations of training data was described.
This framework examines data transformations from two perspectives: as an opera-
tion on a matrix, and as an operation on the parameters of the statistical distribution
that generates the data. These approaches both have pros and cons, including compu-
tational complexity and feasibility depending on the structure of the data in question.
A new incentive structure was also proposed and implemented, which ensures
that the different groups perform their responsibilities with the success of the other
groups as a priority. The group that creates fairness conditions (FM) is incentivized
to create data transformations that maximize the accuracy potential of the model
being developed, while the model creation team (AM) is incentivized to implement
a maximal number of fairness conditions during the model training process. The
overseer (JM) is incentivized similarly, and receives the maximum bonus only if both
the FM and AM are successful in their goals.
A computational study was performed on the Iris data set, in order to demon-
strate the constitution creation and data transformation process. This demonstration
highlighted the power of the AM/FM approach when finding optimal transformation
paths that satisfy fairness conditions. This study also discussed the computational
scaling issues which are encountered when performing an exhaustive search over the
parameter space, and the need for actors to make informed decisions on transforma-
tions and hyperparameters based on prior domain knowledge.
This thesis concluded with the results of a simulation of the AM/FM system
on data from the COMPAS risk assessment tool. This simulation modeled the tenet
recommendation and voting process, as well as highlighting examples of potential
77
disagreements during the constitutional convention. From 200 pairs of data transfor-
mations and hyperparameters, the simulation found 18 which satisfied both fairness
and accuracy conditions and led to both parties receiving the maximum bonus. These
optimal points in the search space were found in fewer than 10 steps on average.
78
Appendix A: Data Transformation Paths
A.1. Transformation Paths and Accuracy (Iris Data)
transforms acc orig acc trans
((1, 0), (2, 0), (3, 0)) 0.62 1.000000
((3, 0), (1, 0), (2, 0)) 0.62 1.000000
((2, 0), (1, 0), (3, 1)) 0.62 1.000000
((2, 0), (1, 0), (3, 0)) 0.62 1.000000
((3, 0), (1, 0), (2, 1)) 0.62 1.000000
((3, 1), (1, 0), (2, 0)) 0.62 1.000000
((1, 0), (3, 1), (2, 1)) 0.62 1.000000
((2, 1), (1, 0), (3, 0)) 0.62 1.000000
((1, 0), (3, 0), (2, 1)) 0.62 1.000000
((1, 0), (3, 1), (2, 0)) 0.62 0.985075
((3, 1), (1, 0), (2, 1)) 0.62 1.000000
((1, 0), (2, 1), (3, 1)) 0.62 1.000000
((1, 0), (2, 1), (3, 0)) 0.62 1.000000
((1, 0), (2, 0), (3, 1)) 0.62 1.000000
((1, 0), (3, 0), (2, 0)) 0.62 1.000000
((2, 1), (1, 0), (3, 1)) 0.62 1.000000
Figure A.1: Paths with significant accuracy loss
80
transforms acc orig acc trans
((2, 1), (1, 1), (3, 0)) 0.74 1.000000
((3, 0), (1, 1), (2, 1)) 0.78 1.000000
((3, 1), (1, 1), (2, 1)) 0.78 1.000000
((1, 1), (2, 1), (3, 0)) 0.82 1.000000
((3, 1), (1, 1), (2, 0)) 0.84 1.000000
((3, 0), (1, 1), (2, 0)) 0.84 1.000000
((2, 1), (1, 1), (3, 1)) 0.84 1.000000
((1, 1), (3, 0), (2, 1)) 0.84 1.000000
((1, 1), (2, 0), (3, 0)) 0.84 1.000000
((1, 1), (2, 0), (3, 1)) 0.84 1.000000
((2, 0), (1, 1), (3, 0)) 0.84 1.000000
((1, 1), (3, 1), (2, 0)) 0.84 1.000000
((2, 0), (1, 1), (3, 1)) 0.84 1.000000
((1, 1), (3, 0), (2, 0)) 0.84 1.000000
((1, 1), (2, 1), (3, 1)) 0.84 1.000000
((1, 1), (3, 1), (2, 1)) 0.88 1.000000
Figure A.2: Paths with significant accuracy loss (cont.)
81
transforms acc orig acc trans
((3, 1), (2, 0), (1, 1)) 1.00 1.000000
((3, 1), (2, 0), (1, 0)) 1.00 1.000000
((3, 0), (2, 1), (1, 1)) 1.00 1.000000
((3, 0), (2, 1), (1, 0)) 1.00 1.000000
((3, 0), (2, 0), (1, 1)) 1.00 1.000000
((3, 0), (2, 0), (1, 0)) 1.00 1.000000
((3, 1), (2, 1), (1, 0)) 1.00 1.000000
((2, 1), (3, 1), (1, 0)) 1.00 1.000000
((2, 1), (3, 0), (1, 1)) 1.00 1.000000
((2, 1), (3, 0), (1, 0)) 1.00 1.000000
((2, 0), (3, 1), (1, 1)) 1.00 1.000000
((2, 0), (3, 1), (1, 0)) 1.00 1.000000
((2, 0), (3, 0), (1, 1)) 1.00 1.000000
((2, 0), (3, 0), (1, 0)) 1.00 1.000000
((2, 1), (3, 1), (1, 1)) 1.00 1.000000
((3, 1), (2, 1), (1, 1)) 1.00 1.000000
Figure A.3: Paths with no accuracy loss
82
A.2. Transformation Paths (COMPAS Data)
C path cost
17.500001 ((1, 1), (3, 1), (2, 1)) -0.308000
12.500001 ((3, 1), (1, 1), (2, 1)) -0.277333
17.500001 ((3, 1), (1, 1), (2, 1)) -0.256000
12.500001 ((1, 1), (3, 1), (2, 1)) -0.224000
7.500001 ((3, 1), (1, 0), (2, 0)) -0.221333
7.500001 ((2, 1), (3, 0), (1, 0)) -0.220000
10.000001 ((2, 1), (1, 1), (3, 0)) -0.214667
10.000001 ((3, 0), (1, 0), (2, 1)) -0.192000
10.000001 ((3, 1), (1, 1), (2, 1)) -0.189333
10.000001 ((2, 1), (1, 0), (3, 1)) -0.189333
10.000001 ((3, 1), (1, 0), (2, 1)) -0.178667
7.500001 ((2, 1), (1, 1), (3, 0)) -0.174667
5.000001 ((2, 1), (3, 0), (1, 0)) -0.168000
7.500001 ((2, 0), (1, 1), (3, 1)) -0.144000
12.500001 ((1, 1), (2, 1), (3, 1)) -0.141333
10.000001 ((3, 0), (1, 1), (2, 1)) -0.134667
7.500001 ((1, 0), (3, 1), (2, 0)) -0.133333
5.000001 ((2, 1), (3, 0), (1, 1)) -0.132000
20.000001 ((2, 0), (3, 0), (1, 1)) -0.130667
15.000001 ((1, 1), (2, 1), (3, 1)) -0.129333
10.000001 ((2, 0), (3, 1), (1, 1)) -0.129333
Figure A.4: Sample of paths with bonus b/2
83
C path cost
5.000001 ((3, 1), (2, 1), (1, 0)) -0.018667
12.500001 ((1, 1), (3, 0), (2, 0)) -0.018667
0.614844 ((2, 0), (3, 0), (1, 1)) -0.018667
7.500001 ((2, 0), (1, 0), (3, 0)) -0.017333
5.000001 ((2, 0), (3, 1), (1, 1)) -0.014667
5.000001 ((2, 0), (3, 0), (1, 1)) -0.014667
2.500001 ((3, 1), (2, 0), (1, 1)) -0.014667
2.500001 ((3, 0), (1, 1), (2, 1)) -0.014667
10.000001 ((3, 0), (2, 1), (1, 1)) -0.013333
0.614844 ((1, 1), (3, 0), (2, 0)) -0.010667
0.614844 ((2, 1), (3, 0), (1, 1)) -0.010667
2.500001 ((3, 0), (1, 0), (2, 0)) -0.010667
2.500001 ((3, 1), (2, 0), (1, 1)) -0.009333
2.500001 ((2, 0), (3, 0), (1, 0)) -0.009333
0.614844 ((2, 1), (1, 1), (3, 0)) -0.006667
2.500001 ((3, 0), (1, 1), (2, 1)) -0.004000
2.500001 ((2, 0), (1, 0), (3, 1)) -0.004000
2.500001 ((1, 1), (2, 0), (3, 0)) 0.000000
Figure A.5: Paths with maximum bonus b
84
References
Aherne, F., Thacker, N., & Rockett, P. (1998). The bhattacharyya metric as an
absolute similarity measure for frequency coded data. Kybernetika, 34 (4), 363–
368.
Ananny, M. (2016). Toward an ethics of algorithms: Convening, observation, proba-
bility, and timeliness. Science, Technology, & Human Values , 41 (1), 93–117.
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias [Computer
software manual]. Retrieved from https://www.propublica.org/article/
machine-bias-risk-assessments-in-criminal-sentencing
Aristotle. (2016). Aristotle’s politics : writings from the complete works. Princeton,
New Jersey: Princeton University Press.
Baumann, E., & Rumberger, J. L. (2018). State of the art in fair ml: From moral
philosophy and legislation to fair classifiers.
Blum, A., & Stangl, K. (2019). Recovering from biased data: Can fairness constraints
improve accuracy? arXiv.org . Retrieved from http://search.proquest.com/
docview/2321241279/
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019). Classification with
fairness constraints: A meta-algorithm with provable guarantees. In Proceedings
of the conference on fairness, accountability, and transparency (pp. 319–328).
New York, NY, USA: ACM. Retrieved from http://doi.acm.org/10.1145/
3287560.3287586 doi: 10.1145/3287560.3287586
85
Chakraborti, T., Patra, A., & Noble, A. (2019). Contrastive fairness in machine
learning.
Chander, A. (2017). The racist algorithm? Michigan Law Review , 115 , 1023–1213.
Chawla, N., Bowyer, K., Hall, L., & Kegelmeyer, W. (2002). Smote: Synthetic
minority over-sampling technique. Journal Of Artificial Intelligence Research,
16 , 321–357.
Chiappa, S., & Isaac, W. S. (2019). A causal bayesian networks viewpoint on fairness.
In Privacy and identity management. fairness, accountability, and transparency
in the age of big data: 13th ifip wg 9.2, 9.6/11.7, 11.6/sig 9.2.2 international
summer school, vienna, austria, august 20-24, 2018, revised selected papers (pp.
3–20). Cham: Springer International Publishing. Retrieved from https://
doi.org/10.1007/978-3-030-16744-8 1 doi: 10.1007/978-3-030-16744-8 1
Culliford, E. (2019, Dec 12). Facebook pledges $130 million to con-
tent oversight board, delays naming members. Reuters . Retrieved
from https://www.reuters.com/article/us-facebook-oversight/
facebook-pledges-130-million-to-content-oversight-board-delays
-naming-members-idUSKBN1YG1ZG
Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J., & Pontil, M. (2018). Em-
pirical risk minimization under fairness constraints. arXiv.org . Retrieved from
http://search.proquest.com/docview/2073911283/
Douzas, G., Bacao, F., & Last, F. (2018). Improving imbalanced learning through
a heuristic oversampling method based on k-means and smote. Information
Sciences , 465 , 1–20.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through
awareness. In Proceedings of the 3rd innovations in theoretical computer science
conference (pp. 214–226). New York, NY, USA: ACM. Retrieved from http://
86
doi.acm.org/10.1145/2090236.2090255 doi: 10.1145/2090236.2090255
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems.
Annals of Eugenics , 7 (2), 179-188. Retrieved from https://onlinelibrary
.wiley.com/doi/abs/10.1111/j.1469-1809.1936.tb02137.x doi: 10.1111/
j.1469-1809.1936.tb02137.x
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learn-
ing. In Proceedings of the 30th international conference on neural information
processing systems (p. 33233331). Red Hook, NY, USA: Curran Associates Inc.
Ilvento, C. (2019). Metric learning for individual fairness. CoRR, abs/1906.00250 .
Retrieved from http://arxiv.org/abs/1906.00250
Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2016, 10). Rawlsian
fairness for machine learning. ArXiv .
Kamiran, F., & Calders, T. (2009). Classifying without discriminating. In 2009 2nd
international conference on computer, control and communication (pp. 1–6).
IEEE.
Kamiran, F., & Calders, T. (2012, Oct 01). Data preprocessing techniques for clas-
sification without discrimination. Knowledge and Information Systems , 33 (1),
1–33. Retrieved from https://doi.org/10.1007/s10115-011-0463-8 doi:
10.1007/s10115-011-0463-8
Khademi, A., Lee, S., Foley, D., & Honavar, V. (2019). Fairness in algorithmic decision
making: An excursion through the lens of causality. arXiv.org . Retrieved from
http://search.proquest.com/docview/2199586504/
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., &
Scholkopf, B. (2017). Avoiding discrimination through causal reasoning. In
Proceedings of the 31st international conference on neural information process-
ing systems (p. 656666). Red Hook, NY, USA: Curran Associates Inc.
87
Kroll, J. A., Huey, J., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G.,
& Yu, H. (2017). Accountable algorithms. University of Pennsylvania Law
Review , 165 (3), 633.
Luong, B., Ruggieri, S., & Turini, F. (2011). k-nn as an implementation of situation
testing for discrimination discovery and prevention. In Proceedings of the 17th
acm sigkdd international conference on knowledge discovery and data mining
(pp. 502–510). ACM.
Matusita, K. (1955). Decision rules, based on the distance, for problems of fit, two
samples, and estimation. Ann. Math. Statist., 26 (4), 631–640.
Montesquieu, C. d. S. (1989). The spirit of the laws. Cambridge ; New York:
Cambridge University Press.
Mukhopadhyay, S., & Bindu Jain. (2001, Sep.). Multi-agent markov decision pro-
cesses with limited agent communication. In Proceeding of the 2001 ieee interna-
tional symposium on intelligent control (isic ’01) (cat. no.01ch37206) (p. 7-12).
doi: 10.1109/ISIC.2001.971476
Noble, S. U. (2018). Algorithms of oppression : how search engines reinforce racism.
New York: New York University Press.
Pessach, D., & Shmueli, E. (2020). Algorithmic fairness.
Pettit, P. (1989). A definition of negative liberty. Ratio, 2 (2), 153-168. Re-
trieved from https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467
-9329.1989.tb00004.x doi: 10.1111/j.1467-9329.1989.tb00004.x
Read, M. (2018). Does facebook need a constitution? New York . Retrieved from
http://search.proquest.com/docview/2187615794/
Scarf, H., & Shapley, L. (2016). Games with partial information. In Contributions
to the theory of games (am-39), volume iii (p. 213). Princeton, New Jersey:
Princeton University Press.
88
Statista. (2019). Number of smartphone users in the united states from 2010 to
2023 (in millions). Retrieved from https://www-statista-com.ezp-prod1
.hul.harvard.edu/statistics/201182/forecast-of-smartphone-users
-in-the-us/
Wisconsin v. Loomis, No. 2015AP157CR. (2016).
Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mit-
igating discrimination without collecting sensitive data. Big Data & Soci-
ety , 4 (2), 2053951717743530. Retrieved from https://doi.org/10.1177/
2053951717743530 doi: 10.1177/2053951717743530
Wadsworth, C., Vera, F., & Piech, C. (2018). Achieving fairness through adversarial
learning: an application to recidivism prediction. arXiv.org . Retrieved from
http://search.proquest.com/docview/2073491586/
Why mark zuckerbergs oversight board may kill his political ad policy. (n.d.). Wired .
Retrieved from https://www.wired.com/story/facebook-oversight-board
-bylaws/
Xiang, A., & Raji, I. D. (2019). On the legal compatibility of fairness definitions.
Zemel, R., Wu, Y., Swersky, K., Pitassi, T., & Dwork, C. (2013, 01). Learning fair
representations. 30th International Conference on Machine Learning, ICML
2013 , 1362-1370.
89