Department of Computer Science Institute for System Architecture, Chair for Computer Networks

On Providing Crowdsourcing as a Service (Position Paper)

Thomas Springer, Tenshi Hara, Alexander Schill

Technische Universität Dresden Computer Networks Group

MapBiquitous Scenario

Integrated indoor/outdoor location-based system

Decentralized client/server architecture

• Building servers manage floorplans and data for positioning and routing

Thomas Springer 2 CGCloud 2013 - On Providing Crowdsourcing as a Service

Multiple ways to use crowdsourcing

update floor plans and navigation pathes

updating wifi fingerprint db (e.g. by scanning barcodes

distributed inside the building)

sharing calibration data of barometer for building level

detection

Application level crowdsourcing, e.g. is the mensa currently

empty or crowded

Current State

Long lists of self-contained crowdsourcing projects

Cloud-based Crowdsourcing Platforms

• Mechanical Turk (Amazon)

• Open Turk

Thomas Springer 4 CGCloud 2013 - On Providing Crowdsourcing as a Service

Crowdsourcing Platform

Contributer Requester

1. register

2. upload task

1. register

3. submit

4. process

5. get results5. get reward

Major drawbacks

Submission processing at the 3rd party platform

Crowdfunders don‘t have full control over submission processing

Limited to types of crowdsourcing tasks and processing capabilities

supported by the crowdsourcing platform

Crowdsourcers and crowdfunders have to trust the platform provider

Users have to maintain multiple logins

Challenges

Thomas Springer 5 CGCloud 2013 - On Providing Crowdsourcing as a Service

Contributors want to:

contribute using different

(mobile) devices independent

of their location

track, correct and withdraw

submissions

be motivated

protect privacy

Requester want to:

address a broad base of

contributors

use specific mechanisms

to process submissions

provide rewards

ban malicious users

System should

be easily adoptable

be highly scalable and fault tolerant

operate in a global setup

SANE Approach

Introducing a crowdsourcing proxy • Server Access Network Entity (SANE)

• Separation of user/submission management and submission processing

Thomas Springer 6 CGCloud 2013 - On Providing Crowdsourcing as a Service

6.2: Submission 6.1: Submission

3: Get Client Runtime

Contributor

Requester

SANE

1: Deploy Crowdsourcing Plugin 2: Exchange Keys

4: Get Server Key

5: Exchange Keys

Submission Forwarding

Thomas Springer 7 CGCloud 2013 - On Providing Crowdsourcing as a Service

Scalability

Crowdsourcing as a Service

(provided by distributed proxies)

Contributor

Contributor

Contributor

∤

∤

∤

Requester

∤

∤

∤

Requester

Requester

m n

HTTPs

Distributed

Hash

Table HTTPs

SANE

SANE

SANE

SANE

SANE

Thomas Springer 8 CGCloud 2013 - On Providing Crowdsourcing as a Service

multiple SANE instances cooperate to offer Crowdsourcing as a service

self-organization of proxies to ensure scalability and fault tolerance by using a Distributed Hash Table (DHT)

Assigning clients to SANE instances

client identifier -> hash -> managed by particular SANE instance

neighbouring SANE instances act as backup

Thomas Springer 9 CGCloud 2013 - On Providing Crowdsourcing as a Service

Client

myID: PeterMustermann

ID-hash: bbe29c00e7661de0

fd6ab795f49bda54

9ffcae5dd63d96d8

670b712360528c32

SANE 1

myArea: [0-3]

lower: [c-f]

upper: [4-7]

handle: [c-7]

SANE 2

myArea: [4-7]

lower: [0-3]

upper: [8-b]

handle: [0-b]

SANE 3

myArea: [8-b]

lower: [4-7]

upper: [c-f]

handle: [4-f]

SANE 4

myArea: [c-f]

lower: [8-b]

upper: [0-3]

handle: [8-3]

responsible SANE lo

wer

Bac

kup

upper Backup

40-57.de.sane.

example.com

58-7f.de.sane.

example.com

b8-cf.de.sane.

example.com

CH DHT DE DHT

00-1f.ch.sane.

example.com

20-37.ch.sane.

example.com

a8-bf.ch.sane.

example.com

ClientID-hash: bbe29c0...

4

1

5 6 7

Country-API

2

3

DNS

Implementation

Thomas Springer 10 CGCloud 2013 - On Providing Crowdsourcing as a Service

Building ServerBuilding Server

Android

generic SANE communication library

SANE Building Server

JSON/HTTPs JSON/HTTPs

Linux, Apache HTTPd, MySQL, PHP

Cro

wd

sou

rcin

g D

rive

r

Client Manager

Server Manager

DHT Maintainer

Security Module(Crypto/

Signature)

Web Feature Service

Routing

Fingerprinting

Security Module(Crypto/

Signature)

Submission Processing

generic SANEcommunication library

Crowdsourcing logic

submission type registered as PHP module

Submission types:

• Wifi and GSM fingerprints

• Position correction

• Crowdedness of student mensa

• Road surface state for bicycles

SANE Evaluation

performance and scalability comparable to website with DB usage

linear correlation between requests, replies and packet-loss (99.8% of packets send are being received) -> saturation @ about 5000 requests

DB at no time a bottleneck

SANE communication introduces overhead

Thomas Springer 11 CGCloud 2013 - On Providing Crowdsourcing as a Service

500 – 1000 simulated Clients on Lenovo Thinkpad X220 Tablet (Intel Core i7-2640M @2.80 GHz/8GB), Windows 7 Professional SP 1, x64

SANE 1 on VM (Intel Xeon E5620 @2.40 GHz/2 GB), Windows Server 2003 SE SP 2, x32, Apache httpd 2.2.14, PHP 5.2.6, MySQL 5.1.

SANE 2/3 VMs at all-inkl.com

connectivity: Client - DSL-6000 (5880 Mbit/s), SANE - Gigabit-Ethernet Full-Duplex (1 GBit/s), all-inkl.com - Fast-Ethernet Full-Duplex (100 Mbit/s)

average submission size 384 Byte

Conclusion and Outlook

SANE approach

• reuse of basic crowdsourcing functionality

• protect contributors privacy (submitterID, submission encryption)

• arbitrary mechanisms for submission processing applicable

• scalability and fault tolerance based on cooperating SANE instances

• standard web technologies used for implementation

o Performance and scalability follow characteristics of website with DB usage

o Proxy delay time inevitably

Future work

• test SANE with other scenarios

• rethinking DHT approach

• add submission and user rating

• gamification mechanisms and rewards to motivate contributors

Thomas Springer 12 CGCloud 2013 - On Providing Crowdsourcing as a Service

Thank you for your attention!

MapBiquitous project information

mapbiquitous.inf.tu-dresden.de

Thomas Springer 13 CGCloud 2013 - On Providing Crowdsourcing as a Service