AMAZON MECHANICAL TURK

JOAN ROIG-ARDERIU

AGENDA

HUMAN COMPUTATION

SERVICE OVERVIEW

THE NAME: “TURK”

HISTORY

TERMINOLOGY

HOW DOES IT WORK? (DEMONSTRATION)

WHO USES THIS?

CONCLUSIONS

HUMAN COMPUTATION

CERTAIN TASKS ARE BETTER PERFORMED BY HUMANS THAN BY COMPUTERS (I.E. PICTURE TAGGING)

INVERTED THINKING: INSTEAD OF CALLING A COMPUTER SERVICE TO PERFORM AN ACTION, THE COMPUTER CALLS A HUMAN.

WHAT IF WE COULD COORDINATE MANY HUMANS TO PERFORM A CERTAIN TASK?

SERVICE OVERVIEW

BASED ON HUMAN COMPUTATION CONCEPT: HUMANS OUTPERFORM COMPUTERS IN MANY TASKS

AMAZON MECHANICAL TURK PROVIDES AN “ON-DEMAND, SCALABLE, HUMAN WORKFORCE TO COMPLETE JOBS THAT HUMANS CAN DO BETTER THAN COMPUTERS”.

DEVELOPERS NEEDING TASKS THAT REQUIRE HUMAN INTELLIGENCE CAN USE MECHANICAL TURK AND INTEGRATE THE RESULTS WITH THEIR SOFTWARE (WEB) APPLICATIONS

THE NAME: “TURK”

IN THE 18TH CENTURY, W. VON KEMPELEN CREATED A CHESS-PLAYING AUTOMATON CALLED “THE TURK”

“THE TURK” TRAVELED AROUND EUROPE, BEATING GOOD CHESS PLAYERS

IT WAS REVEALED THAT “THE TURK” WAS NOT AN AUTOMATON, BUT A BOX WITH A CHESS MASTER HIDDEN INSIDE

THE TURKTHE MECHANICAL TURK: FAKE CHESS-PLAYING MACHINE

HISTORY

INITIALLY CREATED FOR SELF-USAGE BY AMAZON, TO SOLVE PROBLEMS THAT REQUIRED HUMANS: FIND PAGES THAT SHOW THE SAME PRODUCT

AMAZON REALIZED THIS WAS A UNIQUE SERVICE

LAUNCHED TO GENERAL PUBLIC IN 2005

IN MARCH 2007, IT WAS REPORTED THAT THERE WERE MORE THAN 100,000 WORKERS IN OVER 100 COUNTRIES

TERMINOLOGY: HIT

ANY TASK THAT CAN BE COMPLETED BY USING A COMPUTER CONNECTED TO INTERNET

DEVELOPERS (OR REQUESTERS) PUBLISH HITS, AND WORKERS WORK ON THEM

EXAMPLES

SPAM IDENTIFICATION

IMAGE TAGGING

TERMINOLOGY: HIT

ANY TASK THAT CAN BE COMPLETED BY USING A COMPUTER CONNECTED TO INTERNET

DEVELOPERS (OR REQUESTERS) PUBLISH HITS, AND WORKERS WORK ON THEM

EXAMPLES

SPAM IDENTIFICATION

IMAGE TAGGING

EXAMPLE OF EXCLUSIVELY HUMAN TASK:CAPTCHA

TERMINOLOGY: REQUESTER

THE INDIVIDUAL THAT CREATES THE HIT

MUST HAVE MONEY IN AN AMAZON ACCOUNT

CAN REQUEST QUALIFICATIONS FROM THE WORKERS

CAN SPECIFY HOW MANY WORKERS CAN WORK ON A TASK

CAN APPROVE OR REJECT THE RESULTS

CAN TEST HITS IN A “SANDBOX” : PLATFORM THAT ALLOWS REQUESTERS TO TEST THEIR HITS WITHOUT PAYING

TERMINOLOGY: QUALIFICATION

THE REQUESTER CAN POST A TEST THAT THE WORKERS MUST PASS IN ORDER TO WORK ON A PARTICULAR HIT

QUALIFICATIONS ALLOW REQUESTERS TO SELECT COMPETENT WORKERS

TERMINOLOGY: WORKER

THE INDIVIDUAL THAT PERFORMS THE TASKS SUBMITTED BY A REQUESTER

MUST COMPLETE QUALIFICATION TEST FOR SOME TASKS

WORKER CAN PREVIEW THE HIT AND ACCEPT IT OR SKIP IT. ONCE ACCEPTED, THE HIT CAN BE SUBMITTED OR REJECTED

HOW DOES IT WORK?

REQUESTER CAN MANAGE HITS BY:

WEB INTERFACE

COMMAND LINE TOOL

SDK OR USING THE API (SUPPORTS SOAP AND REST)

WORKER CAN LOOK FOR HITS ON THE WEBSITE

LET’S DO IT!

USING THE WEB INTERFACE OF AMAZON TURK.....

AS A REQUESTER: WE ARE GOING TO CREATE A HIT THAT ASKS WHAT KIND OF EMOTIONS A PICTURE PRODUCES.

2 WORKERS PER HIT

PAY $0.03 PER HIT COMPLETED

AS A WORKER: FIND A JOB, AND MAKE SOME MONEY...

WHO USES THIS?

AMAZON TURK:

EXAMPLE OF CLOTHING COMPANY THAT USES MECHANICAL TURK TO CATEGORIZE THEIR CLOTHES: HTTP://WWW.STYLIGHT.DE/

SORTFOLIO USES MECHANICAL TURK TO PREVENT NSFW IMAGES: HTTP://SORTFOLIO.COM/

HUMAN COMPUTATION:

GWAP (GAMES WITH A PURPOSE) HTTP://WWW.GWAP.COM/

CONCLUSIONS

NEW WAY OF THINKING ABOUT GETTING WORK DONE

FAST: TASKS ARE DONE IN PARALLEL (24 X 7)

ACCURATE: MONITOR AND MANAGE QUALITY OF WORK

COST EFFECTIVE: BIG SCALE WORK DONE QUICKLY. REQUESTER CAN REVIEW RESULTS BEFORE PAYING

CONS:

PAYOUTS ARE LOW

MISUSED : I.E. WRITE GOOD REVIEWS ON PRODUCTS

REFERENCES

HTTPS://WWW.MTURK.COM/MTURK/WELCOME

HTTP://BEHIND-THE-ENEMY-LINES.BLOGSPOT.COM/SEARCH/LABEL/MECHANICAL%20TURK

HTTP://WWW.CS.CMU.EDU/~BIGLOU/RESEARCH.HTML

HTTP://VIDEO.GOOGLE.COM/VIDEOPLAY?DOCID=-8246463980976635143#

HTTP://AWS.AMAZON.COM/DOCUMENTATION/MTURK/

HTTP://WWW.NYTIMES.COM/2007/03/25/BUSINESS/YOURMONEY/25STREAM.HTML?_R=1

GRÀCIES! ~ THANK YOU! ~