Embed Size (px)
Citation preview
Steffen Staab Bias in the Social Web 1Institute for Web Science and Technologies · University of Koblenz-Landau, GermanyWeb and Internet Science Group · ECS · University of Southampton, UK &
Bias in the Social Web
Steffen Staab, Christoph Kling & Team
University of Southampton &
Universität Koblenz-Landau
Steffen Staab Bias in the Social Web 2
Produce
Consume
Cognition
EmotionBehavior
SocialisationKnowledge
Observable Micro-
interactions in the Web
AppsProtocols
Data & InformationGovernance
WWW
Observable Macro-
effects in the Web
Web Science
Steffen Staab Bias in the Social Web 3
Web Observatories Konect
Steffen Staab Bias in the Social Web 4
Bias in the Data
Bias in the Algorithm
Bias in the Social Machine
Web
Obs
erva
tory
Steffen Staab Bias in the Social Web 5
Bias in the Data
Steffen Staab Bias in the Social Web 6
Observing Bias in Data
Credit Hire Sex Ethnic Zip Height ... ...
+ +
+ -
- +
+ +
- -
correlated
Data protection laws suggest not to process sensitive data attributes
like „sex“ or „ethnic“
Steffen Staab Bias in the Social Web 7
Example:
Notable women described by „has husband“
Notable men not described by „has wife“
Steffen Staab Bias in the Social Web 8
Observing Bias in Social Networks(Lerman et al 15)
Steffen Staab Bias in the Social Web 9
Geographic Bias in the Algorithm
Steffen Staab Bias in the Social Web 10
fish, rice
seafood, fish seafood, shrimp lobster, wine
seafood, fish, salmon
fish, salmon, wine
rice, fish
lobster, seafood, shrimp
coffee
coffee, wine
coffee
wine
wine
pizza, wine
pizza, wine
pasta, wine
pasta, shrimplobster, shrimp
seafood, shrimp
Tagged photos with geo-coordinates from Flickr
Steffen Staab Bias in the Social Web 11
fish, rice
seafood, fish seafood, shrimp lobster, wine
seafood, fish, salmon
fish, salmon, wine
seafood, shrimp
lobster, seafood, shrimp
coffee
coffee, wine
coffeeitalian, wine
wine
pizza, wine
italian, pizza, wine
pasta, wine
pasta, shrimp
seafoodfishlobstershrimpcrabwinesalmon
winepizzacoffeeitalianpasta
seafood, shrimp
lobster, shrimp
Tasks: Discovering topics, finding clusters
Steffen Staab Bias in the Social Web 12
Cultural areas, country borders, geographical features and other geographical observations exhibit complex spatial distributions
wikipedia.org
Challenge
Steffen Staab Bias in the Social Web 13
fish, rice
lobster, shrimp
seafood, fish seafood, shrimp lobster, wine
seafood, fish, salmon
seafood, shrimp
fish, salmon, wine
seafood, shrimp
lobster, seafood, shrimp
coffee
coffee, wine
coffeeitalian, wine
wine
pizza, wine
italian, pizza, wine
pasta, wine
pasta, shrimp
seafoodfishlobstershrimpcrabwinesalmon
winepizzacoffeeitalianpasta
A. Ahmed, L. Hong and A. Smola, 2013 (following (Yin et al 2011; Sizov 2010))
Existing approaches: Gaussian regions
Steffen Staab Bias in the Social Web 14
fish, rice
lobster, shrimp
seafood, fish seafood, shrimp lobster, wine
seafood, fish, salmon
seafood, shrimp
fish, salmon, wine
seafood, shrimp
lobster, seafood, shrimp
coffee
coffee, wine
coffeeitalian, wine
wine
pizza, wine
italian, pizza, wine
pasta, wine
pasta, shrimp
seafoodfishlobstershrimpcrabwinesalmon
winepizzacoffeeitalianpasta
MGTM 1: Global Topic Clustering
Steffen Staab Bias in the Social Web 15
fish, rice
lobster, shrimp
seafood, fish seafood, shrimp lobster, wine
seafood, fish, salmon
seafood, shrimp
fish, salmon, wine
seafood, shrimp
lobster, seafood, shrimp
coffee
coffee, wine
coffeeitalian, wine
wine
pizza, wine
italian, pizza, wine
pasta, wine
pasta, shrimp
seafoodfishlobstershrimpcrabwinesalmon
winepizzacoffeeitalianpasta
MGTM 2: Determining Neighbourhoods
Steffen Staab Bias in the Social Web 16
Cluster adjacency Dependencies of document-specific topic distributions
Exchange of topic information between clusters
MGTM 3: Derived Topic Model
Steffen Staab Bias in the Social Web 17
Exchange of topic information between clusters
MGTM 4: Exchange of Topic Information
Steffen Staab Bias in the Social Web 18
Exchange of topic information between clusters
MGTM 4: Exchange of Topic Information
Steffen Staab Bias in the Social Web 19
Exchange of topic information between clusters
MGTM 4: Exchange of Topic Information
Steffen Staab Bias in the Social Web 20
γ
M N
L
H
G
G
α0
G
Al
j
0
θjn
w
η s
d
l
δl
L: #regionsM: #documents in clusterN: #words in documentG :⁰ Global topic distributionG : Cluster-topic distributionG : Document-topic distribution
sd
MGTM
MGTM 5: Composed Model
Steffen Staab Bias in the Social Web 21
Evaluation: Anectodal, Perplexity, Gaming
Gaming study: intrusion detection
Precision 8 topicsavg / median
LGTA 0.60 / 0.58
Basic model 0.64 / 0.58
MGTM 0.78 / 0.75
Steffen Staab Bias in the Social Web 22
Biases in the Social Machine:The Case of Liquid Feedback
Steffen Staab Bias in the Social Web 23
...
Steffen Staab Bias in the Social Web 24
Online Delegative Democracy
CC-BY-SA Ilmari Karonen
Steffen Staab Bias in the Social Web 25
Delegative Democracy
• Between direct and representative democracy
CC-BY-SA Ilmari Karonen
Steffen Staab Bias in the Social Web 26
Delegative Democracy
• Between direct and representative democracy
• Voters can delegate their vote to other voters
CC-BY-SA Ilmari Karonen
Steffen Staab Bias in the Social Web 27
CC-BY-SA Ilmari Karonen
Steffen Staab Bias in the Social Web 28
CC-BY-SA Ilmari Karonen
Steffen Staab Bias in the Social Web 29
CC-BY-SA Ilmari Karonen
Delegative Democracy
• Between direct and representative democracy
• Voters can delegate their vote to other voters
• Delegations can be revoked at any time
Steffen Staab Bias in the Social Web 30
CC-BY-SA Ilmari Karonen
Delegative Democracy
• Between direct and representative democracy
• Voters can delegate their vote to other voters
• Delegations can be revoked at any time
• Votes are public!
Steffen Staab Bias in the Social Web 31
Dataset:LiquidFeedback
(German Pirate Party)
Steffen Staab Bias in the Social Web 32
LiquidFeedback – Pirate Party• Observation: 08/2010 – 11/2013
• 13,836 Members• 14,964 Delegations
• 499,009 Votes
Steffen Staab Bias in the Social Web 33
LiquidFeedback – German Pirate Party
•
Users create initiatives, which are grouped by issues and belong to areas
Steffen Staab Bias in the Social Web 34
LiquidFeedback – German Pirate Party
•
Users create initiatives, which are grouped by issues and belong to areas
Area: Environmental issuesIssue: CO2 output has to be reduced.Initiative: Subsidise wind turbines!
Steffen Staab Bias in the Social Web 35
LiquidFeedback – German Pirate Party
•
Users create initiatives, which are grouped by issues and belong to areas
Area: Environmental issuesIssue: CO2 output has to be reduced.Initiative: Subsidise wind turbines!
Areas: 22Issues: 3,565Initiatives: 6,517
Steffen Staab Bias in the Social Web 36
LiquidFeedback – German Pirate Party
• Users create initiatives, which are grouped by issues and belong to areas
Delegations on global, initiative, issue and area level
→ “Back-delegations” possible
Steffen Staab Bias in the Social Web 37
Dataset – First Impressions
•
Steffen Staab Bias in the Social Web 38
Dataset – First Impressions
•
Voting Weight
Steffen Staab Bias in the Social Web 39
Dataset – First Impressions
•
3,658 members > 10 votes1,156 members > 100 votes 54 members > 1,000 votesMedian all: 8 votes
Median delegating: 42 votesMedian delegates: 64 votes
Steffen Staab Bias in the Social Web 40
Delegation Network• Temporal analysis
•
Steffen Staab Bias in the Social Web 41
Delegation Network• Temporal analysis
•
Steffen Staab Bias in the Social Web 42
Delegation Network• Temporal analysis
•
Steffen Staab Bias in the Social Web 43
Delegation Network• Temporal analysis
•
Steffen Staab Bias in the Social Web 44
Delegation Network• Temporal analysis
•
Steffen Staab Bias in the Social Web 47
The Power of Voters
Steffen Staab Bias in the Social Web 48
Power
•
Ability to influence the outcome of a vote
Steffen Staab Bias in the Social Web 49
Power
•
Ability to influence the outcome of a vote
5
4
1
Steffen Staab Bias in the Social Web 50
Power
•
Ability to influence the outcome of a vote
5
4 same power
1
Steffen Staab Bias in the Social Web 51
Power Indices
•
Given voting weights of all voters in a vote:Predict the probability that a given user will be able determine the outcome of a vote
Banzhaf power index: Votes are independent
Shapley power index:Votes are homogeneous
Steffen Staab Bias in the Social Web 52
Power
•
Banzhaf power index: Votes are independent
Shapley power index:Votes are homogeneous
Potential Power:Measured power in the dataset
Exercised Power:Power used to actually turn votes
Steffen Staab Bias in the Social Web 53
Power Indices
•
20 40 60 80 10000.10.20.30.40.50.60.70.80.9
1
Delegations d
Powe
r p
Potential PowerExercised Power
Steffen Staab Bias in the Social Web 54
Power Indices
•
Steffen Staab Bias in the Social Web 55
Average Approval Rate
•
(How many users agree with x% of all voted proposals?)
Steffen Staab Bias in the Social Web 56
Average Approval Rate
•
Powerful voters tend to vote positive and to agree with the majority
Steffen Staab Bias in the Social Web 57
Power
•
Potential Power:Measured power in the dataset
Beta power index: Beta distributed approval rate for Banzhaf index
Regression power index:Logistic regression for predicting the approval rate – given the voting weight – of the Banzhaf index
Beta2 power index:Beta distributed approval rate for Shapley index
Steffen Staab Bias in the Social Web 58
Novel Power Indices
•
Steffen Staab Bias in the Social Web 59
Novel Power Indices
•
(Perplexity ~ normalised log likelihood)
Steffen Staab Bias in the Social Web 60
The Impact of Delegations
Steffen Staab Bias in the Social Web 61
Approval Rate
•
→ Approval rate decreases with voting experience
Steffen Staab Bias in the Social Web 62
Approval Rate
•
→ Delegates stabilise the approval rate!
Steffen Staab Bias in the Social Web 63
Results
•
Including voting bias in power indices improves the prediction
First evaluation of power indices on a large voting history
Delegates stabilise the system
Steffen Staab Bias in the Social Web 64
Conclusions
Steffen Staab Bias in the Social Web 65
Bias in the Data
Bias in the Algorithm
Bias in the Social Machine
Web
Obs
erva
tory
Steffen Staab Bias in the Social Web 66
Bias in the Data
Bias in the Algorithm
Bias in the Social Machine
Story telling
Under-standing
Modelling
Steffen Staab Bias in the Social Web 67
Institute for Web Science & Technologies
Semantic Web
Web Search & Data Mining
Computational Social Science
Interactive Web
Software & Services
