Task Planning and Incentives in Mobile Sensing

KSE652 Social Computing Systems Design and Analysis

Uichin Lee

Recruitment Framework for Participatory Sensing Data Collections

Sasank Reddy, Deborah Estrin , Mani SrivastavaPervasive 2010

Participatory Sensing

• Allowing people to investigate processes with mobile phones

• Community based data collection and citizen science; offering automation, scalability, and real-time processing and feedback

• Examples: taking photos of assets that document recycling behavior, flora variety, and green resources in a university

Participatory Sensing: Challenges

• Diverse users and participatory sensing projects

• How to identify participants to projects?• Goal: devise a new recruitment framework

using availability and reputation – Spatio-temporal availability based on mobility and

transport mode– Reputation of data collection performance

Sustainability Campaigns• GarbageWatch: The campus needs to divert 75% of its waste stream from

landfills, and effective recycling can help reach this goal. By analyzing photos, one can determine if recyclables (paper, plastic, glass, or aluminum) are being disposed of in waste bins, and then identify regions and time periods with low recycling rates.

• What's Bloomin: Water conservation is a high priority issue for the campus and efficient landscaping can help. By collecting (geo-tagged) photos of blooming flora, facilities could later replace high water usage plants with ones that are drought tolerant.

• AssetLog: For sustainable practices to thrive on a campus, the existence and locations of up-to-date “green" resources needs to be documented (e.g., bicycle racks, recycle bins, and charge stations).

Sustainability Campaigns

Framework Overview

Recruitment Framework• Qualifier: minimum requirements

– Availability: destinations and routes within space, time, and mode of transport constraints

– Reputation: sampling likelihood, quality, and validity over several campaigns or by campaign-specific calibration exercises

• Assessment: participant selection– Identify a subset of individuals who could maximize coverage over a

campaign area and time period while adhering to the required mode of transport

– Cost may be considered when selecting participants• Progress review: checking “consistency”

– Review coverage and data collection performance periodically– If participants are below a certain threshold, provide feedback, or recruit

more participants

Related Work• Mobility models

– Location summarization for personal analytics: from location traces to places (e.g., spatio-temporal clustering, density-based clustering, reverse geo-coding)

– Location prediction to adapt applications: mostly for location-based services (LBS); prediction methods include Markov models, time-series analysis, etc.

• Reputation systems:– Summation and average (e.g., Amazon review)– Bayesian systems (e.g., Beta reputation system)

• Selection services:– Online labor markets: M-Turk, GURU.com– Sensor systems: traditional sensor networks focused on coverage (or

sensing in a predefined zone)

Coverage Based Recruitment

• Mobility traces (say for every 30 seconds)• Density-based clustering to find “destinations” (or places)• Routes are points between destinations • Mode of transport is inferred (e.g., still, walking, running,

biking, or diving)

Coverage Based Recruitment• Qualifier filters:

– e.g., selecting individuals with at least 5 destinations in a certain area in a week or individuals with at least 7 unique walking routes during day time

• Assessment: – Given (1) a set of participants with associated costs

and spatial blocks w/ mode of transport over time, and (2) block associated with certain utilities

– Goal is to find a subset of participants such that the utility is maximal under budget constraints (NP-hard)

– Greedy algorithm works OK (at least 63% from the optimum )

Coverage Based Recruitment

• Progress review:– Checks whether participant mobility is consistent with the profile

used for recruitment? – Reviewing M*N spatio-temporal association matrix

• M rows: spatial blocks (100m*100m)• N cols: distinct time slots in a day (cumulated over a week)• Entry: the proportion of time spent in a spatial block (that satisfies mode

of transport and monitoring period constraints)

– Comparing two consecutive weeks (to check deviation)• Singular Value Decomposition (SVD): U*∑*Vt

– U: patterns common across different time periods (days)– ∑: singular values (σ1…σrank) show variance represented by each pattern

where

Participation and Performance Based Recruitment

• Divide data collector reputation into two classes: cross-campaign vs. campaign-specific (focus of this paper)

• Campaign-specific metrics – Timeliness (latency)– Relevancy (falls in phenomenon of interests)– Quality– Participation likelihood: whether an individual took a

sample when given the opportunity

Participation and Performance Based Recruitment

• Campaign specific reputation modeling– Beta reputation model w/ α (#success) and β (#failure)– Expected reputation of a user: E(α, β) = α/(α+β)

– Exponential averaging of α and β over time

w: aging factor

Participation and Performance Based Recruitment

• Confidence in this reputation score is the posterior probability given that the actual expectation value lies within an acceptable level of error (e.g., 0.1)

Evaluation

• Campaign deployment information:

• Ground-truth: experts traversed the routes

Coverage Based Recruitment• Evaluated assessment methods: – Random: select users from campaigns arbitrarily– Naïve: select users who cover the most blocks overall

without considering coverage of existing participants– Greedy: select users who maximize utility by

considering coverage of existing participants

Coverage Based Recruitment

• Algorithm comparison for GarbageWatch campaign coverage

Coverage Based Recruitment

• Consistency check for campaign coverage (progress review)

(changed mode of transport: from walking to driving)

Participation and Performance Based Recruitment

• Evaluated participation likelihood– Other metrics not considered (e.g., timeliness, relevancy, quality due

to the nature of projects, i.e., auto uploading)

• A user’s reputation after “AssetLog calibration exercise”

Participation and Performance Based Recruitment

• Re-evaluating reputation over two weeks– With (c) and without (d) exponential aging

Discussion

• Greedy vs. naïve: if users’ coverage overlaps more, there will be much difference..

• Across campaign consideration (due to individual’s preference, performance may be different)

• Participants grew tried of collecting samples• Participants reported that the act of data capture should be

streamlined so that it can be repeated rapidly • Participants wanted visualization (e.g., map)• Participants were generally OK with “minor” deviation from

their routes, but drastic change may require some incentives

Dynamic Pricing Incentive for Participatory Sensing

Juong-Sik Lee and Baik HohNokia Research

Pervasive and Mobile Computing 2010

Introduction

• Dynamic return of investment (ROI) of participatory sensing applications (different data types, users’ context, etc)

• Fixed price incentives may not work well; further, it’s hard to come up with an optimal price

• Reverse auction: – Users bid for selling their data– Buyer selects multiple users and purchases their data

Reverse Auction

• A user’s utility: U(b) = (b-t)*p(b)– b: received credit – t: base value of the data (that a user believes)– p(b): winning probability

Problems with Reverse Auction

• Lost users may drop out of the system• Incentive cost explosion happens when a system has

below a threshold number users (here, m)– Those users can increase their bid as much as possible

• Solution: for each loss, buyer gives virtual participation credit (of fixed amount α); credit cumulates over time– Seller can use the credit to lower the bid (thus, increasing

winning probability)

Incentive cost explosion BID

Give “virtual credit” α to losers

winners losers

Credit-based Incentives

Summary

• Random Selection based Fixed Pricing (RSFP):– (+)Simple to implement – (+) Easy to predict total incentive cost – (-) Difficult optimal incentive price decision – (-) Unable to adapt to dynamic environments

• Reverse auction dynamic pricing w/ virtual credit (RADP-VPC)– (+) Eliminate complexity of incentive price decision – (+) Able to adapt to dynamic environments– (+) Minimize incentive cost– (+) Better fairness of incentive distribution– (+) Higher social welfare– (-) Relatively harder to implement than RSFP

Evaluation Model

• ROI till round r – ROI(r) = [earning so far] / [# of participation till round r]*[min reward]

• If ROI(r) drops below 0.5, a user drops out of the system

• A user’s valuation is randomly generated based on some distribution

• Evaluation items– Incentive cost reduction– Fairness against true valuation– Service quality

Incentive Cost Comparison• Reverse auction dynamic pricing with virtual

participation credit (RADP-VPC)• Random Selection based Fixed Pricing (RSFP)

Incentive Cost Reduction

• Virtual credit stabilizes incentive cost by keeping price competitions via preventing higher true value users from dropping out of the reverse auction

Reverse auction dynamic pricing with virtual participation credit (RADP-VPC)

Fairness Against True Valuation• Wining probability is not a function of true value in RSFP, whereas

in RADP-VPC lower true value has higher winning probability

Service Quality Guarantee

Discussion

• Privacy leak: one has to send data with bid– Data encryption prevents the buyer from validating

the quality (how about using homomorphic crypto?)• Data broker in between seller and buyer– Data collection, maintenance, processing/mining

• Handling different types of apps (e.g., real-time vs. asynchronous)

• How to guarantee data integrity and to maintain seller’s reputation?

Plans

• Exam:– Papers: Social computing systems design + social network

analytics– Types: Mostly short free-form questions or multiple choice

questions; – Exam duration: 20 minutes

• Final presentation:– Dec 18 (TUE) or 20 (THR)

• Each team will have a 20 minute slot (including Q&A)

– Final report due in Dec 23 23:59.59• SIG CHI format (up to 10 pages, double column)

– Submit along with the updated slides