View
168
Download
2
Category
Preview:
DESCRIPTION
Mahmood Hosseini, Keith Phalp, Jacqui Taylor, Raian Ali. The Four Pillars of Crowdsourcing: A Reference Model. The IEEE Eighth International Conference on Research Challenges in Information Science (RCIS 2014). Marrakesh, Morocco. 28-30 May 2014. This work is the first attempt to provide a taxonomy for the pervasive phenomenon of crowdsourcing.
Citation preview
THE FOUR PILLARS OF
CROWDSOURCING
A Reference Model
Mahmood HosseiniKeith PhalpJacqui TaylorRaian Ali
Faculty of Science and TechnologyBournemouth University, UK
2
OUTLINEIntroduction
Why a taxonomy?
Research method
The four pillars of crowdsourcing
Crosscutting dependencies
Information systems and crowdsourcing
Conclusion
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
3
INTRODUCTION
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
4
CROWDSOURCING DEFINITION The original definition by Jeff Howe
(2006): “Crowdsourcing is the act of taking a job
traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”
Crowdsourcing has gained much attention since!
A variety of applications A variety of platforms A variety of definitions
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
5
BUT WHAT IS CROWDSOURCING REALLY? No agreed definition
(E. Estellés-Arolas, 2012, “Towards an integrated crowdsourcing definition”)
No established ontological and conceptual foundation
No taxonomy for crowdsourcing
Do we need an agreed definition? Do we need a conceptual foundation? Do we need taxonomy for
crowdsourcing?
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
6
WHY A TAXONOMY?
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
7
WHAT IS A TAXONOMY? Taxonomy brings order to a body of
information or a collection of objects under study.
Taxonomy very often enhances one's ability to understand complex concepts and ideas.
Taxonomy classifies or categorizes a body of knowledge or collection of objects in a hierarchical manner.
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
8
WHY IS A TAXONOMY NEEDED IN CROWDSOURCING? Taxonomies are useful:
Taxonomy of reverse engineering Chikofsky and Cross, “Reverse engineering and
design recovery: a taxonomy”Taxonomy of autonomous agents
Franklin and Graesser, “Is it an agent, or just a program?: a taxonomy for autonomous agents”
Taxonomy of web search Broder, “A taxonomy of web search”
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
9
WHY IS A TAXONOMY NEEDED IN CROWDSOURCING? Crowdsourcing definitions are partial, diverse,
and sometimes even contradicting. Take these examples:
“By ‘crowdsourcing’ I mean: Tapping the perceptual, cognitive or enactive abilities of many people to achieve a well-defined result such as solving a problem, classifying a data set, or producing a decision.” Erickson, 2011
“Crowdsourcing has recently emerged as a powerful alternative. It outsources tasks to a distributed group of people (usually called workers) who might be inexperienced on these tasks.” Li et al., 2013
“A crowdsourcing market is the place, usually an online website, where workers find and perform tasks often for a financial reward.” Fardani et al., 2011
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
10
RESEARCH METHOD
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
11
HOW WE CONDUCTED THIS RESEARCH Literature study in crowdsourcing
To get a better understanding of the concept Multidisciplinary (computer science, business
and management, law, medicine, environmental sciences, etc.)
Papers with an explicit definition of crowdsourcing
Content analysis Extracting crowdsourcing features
Three reviewers involved Several passes to ensure no features are left
unnoticed Building up on our taxonomy
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
12
THE FOUR PILLARS OF CROWDSOURCING
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
13
WHAT ARE THE FOUR PILLARS? The Crowd
These are the people who participate in a crowdsourcing activity
The Crowdsourcer These are the people who are seeking for a
solution to a problem, an innovation, or co-creation
The Crowdsourced Task This is what needs to be performed, solved,
created or innovated by the crowd
The Crowdsourcing Platform This is where it all takes place
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
14
TAXONOMY OF THE CROWD Diversity
Spatial diversity Gender diversity Age diversity Expertise diversity
Unknown-ness Not known to
crowdsourcer Not known to each
other Largeness
Number fulfils the task
Number not abundant
Undefined-ness Suitability
CompetenceCollaborationVolunteeringMotivation
Mental satisfaction Self-esteem Personal skill
development Knowledge sharing Love of
community
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
15
TAXONOMY OF THE CROWDSOURCER Incentives provision
Financial incentivesSocial incentivesEntertainment incentives
Open call Ethicality provision
Opt-out provisionFeedback to crowdNo harm to crowd
Privacy provision
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
16
TAXONOMY OF THE CROWDSOURCED TASK Traditional operation
In-house Outsourced
Outsourcing task Modularity
Atomic tasks Divisible to micro
tasks Complexity
Simple tasks Complex tasks
Solvability Simple for humans Complex for
computers
Automation characteristics Difficult to
automate Expensive to
automate User-driven
Problem solving Innovation Co-creation
Contribution Type Individual
contribution Collaborative
contribution
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
17
TAXONOMY OF THE CROWDSOURCING PLATFORM Crowd-related Interactions
Provide Enrolment Provide Authentication Provide Skill Declaration Provide Task Assignment Provide Assistance Provide Result Submission Coordinate Crowd Supervise Crowd Provide Feedback Loops
Crowdsourcer-related Interactions Provide Enrolment Provide Authentication Provide Task Broadcast Provide Assistance Provide Time Negotiation Provide Price Negotiation Provide Result Verification Provide Feedback Loops
Task-related Facilities Aggregate Results Hide Results from Others Store History of Completed Tasks Provide Quality Threshold Provide Quantity Threshold
Platform-related Facilities Online Environment Manage Platform Misuse Provide Ease of Use Provide Attraction Provide Interaction Provide Payment Mechanism
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
18
CROSS-CUTTING DEPENDENCIES
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
19
WHAT DO WE NEED TO KNOW? There are clearly a set of inter-
dependencies between some crowdsourcing features
They help developers and decision-makers
We use four relationship notionsRequireExcludeSupport (a softer version of require)Hinder (a softer version of exclude)
Some crowdsourcing features are social and qualitative!
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
20
INTER-DEPENDENCIES EXAMPLES Co-creation, a task feature, requires Collaboration,
a crowd feature. Largeness, a crowd feature, supports Diversity, a
crowd feature. It hinders Coordinate Crowd, a platform facility.
Not Known to Each Other, a crowd feature, excludes, Collaboration another crowd feature. It also excludes Collaborative Contribution, a task feature. It hinders Social Incentives, a crowdsourcer feature, as the participants’ identities will be hidden from each other.
Provide Skill Declaration, a platform facility, supports Provide Task Assignment, another platform facility. It hinders Not Known to Crowdsourcer, a crowd feature.
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
21
INFORMATION SYSTEMS AND CROWDSOURCING
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
22
CHALLENGES Metrics
Fuzzy concepts (e.g. largeness) Diversity
Trade-off between quality and diversity Recruitment
Trade-off between quality and open call Interaction and aggregation
Regulations needed because of the intensive human factor
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
23
WHERE CAN WE UTILIZE CROWDSOURCING IN INFORMATION SYSTEMS? Adaptive systems engineering
Ali et al. “Social adaptation: when software gives users a voice”
User feedback for requirements knowledge Pagano and Maalej “User feedback in the appstore: an empirical study”
Stakeholders discovery Lim et al. “StakeNet: using social networks to analyse the stakeholders
of large-scale software projects”
Testing Bell et al. “Secret ninja testing with HALO software engineering”
Requirements elicitation Knauss “On the usage of context for requirements elicitation: end-user
involvement in IT ecosystems”
Validation Sayeed et al. “Crowdsourcing the evaluation of a domain-adapted named
entity recognition system”
More
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
24
CONCLUSION
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
25
CONCLUSION We provide an initial taxonomy for
crowdsourcing The taxonomy will help researchers to
better understand crowdsourcing and its features
The taxonomy will help practitioners to design crowdsourcing platforms more efficiently
The taxonomy will better clarify the usage areas of crowdsourcing in any domain, including information systems
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
26
FUTURE WORK
Taxonomy revision and extension
Capturing the crowd
Developing tools
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
27
ACKNOWLEDGMENT
The research was supported by an FP7 Marie Curie CIG grant (the SOCIAD Project) and by Bournemouth University through the Fusion Investment Fund (the BBB and VolaComp and BUUU projects) and the Graduate School Santander Grant for PGR Development.
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
28
THANK YOU!
RC
IS, M
arr
ake
ch, M
oro
cco, 2
8-3
0 M
ay,
20
14
mh
oss
ein
i@b
ou
rnem
ou
th.a
c.u
k
Recommended