Amazon mechanical turk intro to govt partners v2

Crowdsourcing for improving

Business Process

October, 2012 - Optimized for 1280x960 - © Amazon Web Services

amazon web services

2

Before we start ] [ 2

_ Presented by

John Hoskins, hoskins @ amazon.com

Mechanical Turk, Amazon

3 3

Detail

High Level

What is MTurk?

How does it

work Crowdsourcing

How is it used

Questions

Our plan for today ] [

Examples

4

What is Crowdsourcing? ] [

4

5

Crowd Labor Trends [ ]

5

6 6

1. Crowdsourcing

7

] [

7

Crowdsourcing: Key Advantages

8

1. Elastic Capacity ] [

8

9

2. On Demand Availability ] [

9

10

On and Off Fast Growth

Variable peaks Predictable peaks

Staffing Usage Patterns

] [

10

11

On and Off Fast Growth

Predictable peaks

Missed Opportunity

WASTE

Staffing Usage Patterns ] [

11

Variable peaks

12

Fast Growth

Predictable peaks

On and Off

Usage Patterns: Crowdsourcing ] [

12

Variable peaks

13

[ ]

13

3. Pay as you go, for what you use

14

Business Benefits of Crowdsourcing ] [

Improve productivity

Lower costs

$ Capital efficiency

$

14

Focus on your business

15 15

2. What is MTurk?

16

Marketplace for Work _ Access to global workforce, on demand

_ Quality controls and workflow “building blocks”

_ Pay as you go, only when satisfied with results

_ Programmatic Access (API)

] [

16

What is Mechanical Turk?

17

Global Workforce ] [

17 (as of Apr 19th, 2012)

_ 500,000 Workers

_ 190+ Countries

18

Mturk Workforce [ ]

MASTERS

Highest Performing

Average

Unknown

18

19

Custom Workforce [ ]

MASTERS

Highest Performing

Average

Unknown

19

Your Custom Workforce

20 20

3. How it works

21

How it works [ ]

21

22

[ ]

22

Balances & Levers

Accuracy

Price

Speed

23 23

[ ] Choose your Workers

Accuracy

[ ] Assess your Workers

[ ] Gain Consensus

24 24

[ ] Worker Ergonomics

Speed

[ ] Minimize Inefficiencies

[ ] Instant Feedback

25 25

[ ] Consistency

Price

[ ] Predictability

[ ] Reputation

26 26

4. How is it used?

27

Common Use Cases [ ]

27

Data Management _ Verification

_ Entry & Collection

_ De-dupe

_ Algorithm Training

Annotation _ Tagging

_ Classification

Content Management _ Moderation (Photos, Content)

_ Transcription

_ Localization/Translation

Analysis/Research _ Sentiment

_ Relevance

_ Online Research

28 28

[ ] Examples

How can we verify/annotate 365,000 videos quickly?

How can we train our Machine Translation to understand spoken language?

How can we make our assets searchable?

29 29

[ ] Examples

Moderation

Content Creation

Categorization and keywording

30

http://www.mturk.com

30

John Hoskins, Mechanical Turk

hoskins@amazon.com

amazon web services