1

Making Continuous Mobile Sensing More Energy-Efficient

Archan MisraSingapore Management University

archanm@smu.edu.sg

Sep 27, 2012

Karl Aberer (EPFL), Rajesh Balan (SMU), Dipanjan Chakraborty (IBM),Lipyeow Lim (U. Hawaii), Sougata Sen (SMU), Vigneshwaran Subbaraju(SMU), Zhixian Yan (EPFL)

The Opportunity

2

Smartphones: A Beehive of sensing activity

*Reproduced from: “Sensing Your World”, Mike Thompson, blogs.synopsys.com

Increasing penetration ofsensors in mobile devices/tablets.

• Accelerometers• Compass• Gyroscope• Barometer

Projected Penetration of Inertial Sensors, reproduced from Yole Developpement Report, 2011

The Applications of Sensing

3

Location-based ServicesIndoor Location

Wellness & Lifestyle

PureRunner BeWell

Social Networking & Interaction

WalkBase

Color Postural Recognition (ETH)

ShopKick

The Problem

4

The Energy Overheads:• Activating/Sampling the Sensor• Processing the Sensor Data Stream•Transmitting the Results to the “Cloud”

Power Consumption Observed on a Test Samsung Galaxy S3

• OK for ‘intermittent’ sensing.• Need research to address“continuous sensing”!

20-30% increase in overhead of motion-

related activity recognition

The Different Steps in “Making Sense”

5

Sensing

Feature Extraction

Classification

Context Deduction

High-Level Query

Mean, Fourier Coefficients, Entropy

Stand, Walk

Queuing, Exercising

Is Archan ‘queuing’ alone or with friends?

Accel(x,y,z)

The Research Threads

6

Make the Sensing Process More Adaptive andEfficient

Make the Querying Logic Smarter on anIndividual Phone

Make the Querying Logic Smarter Across ManyPhones

A3R: Adaptive Sensing & Feature Extraction

(accelerometer)

ACQUA : query optimization

Cloud Query Coordination Service

High-Level Query

Research #1: A3R

7

A3R: Adaptive Accelerometer-based Activity Recognition

8

Key Idea: Adjust accelerometer “parameters” based on the current activity of the individual.

Two parameters:• Accelerometer sampling frequency (SF)• Classification Features (CF)

Goal: reduce energy overhead of activity recognition without sacrificing accuracy

Energy Overhead Variation

• Energy overhead increases with SF.• Non-linear increase when frequency-domain features (CF)

are selected along with time-domain features.

9

Graph of Energy Consumptionon Samsung S2

Classification Accuracy• Different combination of Activites (Aggregated)

– <sampling frequency, feature choice>

10

• Activity-specific– separate study for each activity

A3R Algorithm for Continuous Activity Recognition

11

Activity Unknown<Fmax, T+F>

Stand<16,T>

Sit-relax<5,T>

Slow Walk

<16,T>

Escalator down

<100,T+F>

Ave_Conf> ∆

Ave_Conf< ∆

Ave_Conf> ∆

Ave_Conf> ∆

Ave_Conf> ∆

Ave_Conf< ∆

Ave_Conf< ∆

Ave_Conf< ∆

• The initial (SF, CF) default: highest sampling freq (SF) & richest feature set (CF)

• Classifier confidence: conf_vector = [p1, p2, …, pn]Average the confidence over a window & find the max

If ave-conf < Δconf

– use maximum (SF,CF)else

– use smart (SF,CF)

A3R: Insight into Activity Behavior of Real Users

user1 user2 user3 user4 user5 user60

2

4

6

8

10

12

14

16

x 104

coun

t of a

ctiv

ities

sit sitRelax normalWalk slowWalk stand stairs

Nokia N95 data: 6 users, 2-4 weeks each Non-adaptive vs. A3R

Full100, Full50, Full16: both {time + freq} features A3R: adaptive feature & sampling frequency

12

Evaluation II: In-Situ Study for Android Users

Continuous study on two android phones User 1: Samsung Galaxy II User 2: HTC Nexus I

Battery Remaining Each day (3 cases) A3R: adaptive feature & sampling frequency Non-adaptive: {50}Hz + {time+freq} features No activity recognition

13

Research #2: ACQUA:

14

ACQUA= Acquisition-Cost Aware Continuous Query Adaptation

ACQUA: Historical Scenario

SPO2

ECG

HR

Temp.

Acc.

...

IF Avg(Window(HR)) > 100AND Avg(Window(Acc)) < 2 AND AVG(Window(Tep))>80F

THEN SMS(caregiver)

Body-worn health & wellness sensors

Phone runs a complex event processing (CEP)

engine with rules for alerts

15

Can we save ‘energy’ by avoiding the need to download all data from all sensors, all the time?

ACQUA: Dynamically Changing Order of Retrieving Sensor Data

Predicate Avg(HR,5)>100

Max(SpO2,10)<90

Acquisition 5 * .02 = 0.1 nJ 10 * .008 = 0.08 nJPr(false) 0.95 0.8

if Avg(Heart-Rate, 5)>100 AND Max(Sp02,10)<90 then ALERT (STRESS).

Acq./Pr(f) 0.1/0.95 0.08/0.8

16

Query Sp02 first or HR first?• Different events are less “likely” (at present)• Different sensors need different amount of

energy to transfer data

#1: Evaluate predicates with lowest energy consumption first:

{Sp02, HR}

#2: Evaluate predicates with highest false probability first

{HR, Sp02}

#3: Evaluate predicate with lowest normalized acquisition cost first (#1/#2): {Sp02, HR}

ACQUA Architectural Components

17

Asynchronous Event Engine • Maintains partial query evaluation state

Dynamic QueryEvaluation Optimizer• Determines retrieval sequence for sensor streams

Query Logic Specification Module• Subset of Stream-SQL query syntax

Cost Modeler• External specification of sensor-specific trx. Cost model• Dynamic evaluation of stream selectivity

C(.); P(.)Normalized Query Syntax

Push/Pull, Batch commands

Dynamic Sensor Control (DSC)

ACQUA Components on a

Single Phone

Example: ω(Evaluation Period)=3

• Time 5: P2,P1,P3• Time 8: acquisition cost for A becomes

cheaper, because some tuples are already in buffer P1, P2, P3

Acquisition cost depends on state of the buffer at time t

18

P1

P2 P3

Dynamics of SelectivityThe sequence decision is made at EVERY

evaluation instant!

Performance Results (Simulations)Bluetooth 802.11

Ener

gyBy

tes

19

Research #3: Cloud Coordinated Mobile Sensing

20

Illustrating the Coordination Service

21

Inform when – “At least 3 PhD students of 2012 taking ISM are co-located”, so that – “I can discuss assignment with them”

Let me know if – “when at least one of 3 persons, A, B and C, are back in office and not using their cellphone, so that – “I can discuss assignment with them”

Can we save ‘energy’ by better coordinating the queries across large number of phones?

Sensing Coordination Service

Application 1

<M1,Q1>

Application 2<M2,Q2>

Q1 Q2

Logic Flow of the Coordination Service

Optimize JOINTLY (Q1, Q2):

ACQUA on Multiple Phones

Optimization Energy Savings (compared to ‘all transfer)

Individual Initial 48%

Jointly Additional 27%

23

Q1: Inform when – “A is standing and near the security gate”

Q2: Inform when – “One of A or B is standing”

Sensing Coordination Service

StandingA(Accel)StandingB(Accel)

LocationA(Wi-Fi)

Optimize individually:Q1: {locationA(Wi-Fi), StandingA(Accel)}Q2: {StandingB(Accel), StandingA(Accel) } {StandingA(Accel)

locationA(Wi-Fi)

StandingB(Accel)

Preliminary Numerical Investigation

24

LiveLabs Cloud Service

5 minutes later

Lifestyle Company

If a group of 4 or more people exit from Café after sitting down for 10 minutes, send

SMS with a “Movie Discount”

10 minutes later

4 in a group sitting down at

a Café

4 in a group left after 10 mins

LiveLabs software continuously monitors (location, activity, …)

Show this notification and get 20% on all

Movies

LiveLabs: A Future Testbed for Mobile Sensing(Jointly with Prof. Rajesh Balan)

LiveLabs: a large-scale research testbed for “mobile systems innovation” let’s companies & researchers run LARGE-SCALE trials and experiments with novel context-based mobile applications and services testbeds at 3 key locations with 30,000 users

Conclusions• Energy is the most critical resource constraint in

mobile sensing.• Advances include

– Adaptive adjustment of individual sensor– Query optimization of queries on individual phone– Joint optimization of queries across multiple phones.

• Ability to predict ‘context’ is the key.– Currently, context is inferred on per-phone basis.– Future—context itself inferred ‘collectively’?– Open issue: scaling context to citizen-scale

environments.

25

The End!

Thanks! Questions?

26

• “Energy-Efficient Continuous Activity Recognition on Mobile Phones: An Activity-Adaptive Approach” by Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra and K. Aberer, 16th Annual International Symposium on Wearable Computers (ISWC), June 2012.

• "Optimizing Sensor Data Acquisition for Energy-Efficient Smartphone-based Continuous Event Processing", by Archan MISRA and Lipyeow LIM, IEEE Int. Conference on Mobile DataManagement (MDM), May 2011.

• "The Case for Cloud-Enabled Mobile Sensing Services", by Sougata SEN, Archan MISRA, Rajesh Krishna BALAN, and Lipyeow LIM,, Mobile Cloud Computing Workshop (MCC'12), in conjunction with ACM SIGCOMM, August 2012.