49
1 SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache Co-authors: Martin Azizyan and Romit Roy Choudhury

SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

  • Upload
    lot

  • View
    29

  • Download
    2

Embed Size (px)

DESCRIPTION

SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache Co-authors: Martin Azizyan and Romit Roy Choudhury. Context. Pervasive wireless connectivity + Localization technology = Location-based applications. Location-Based Applications (LBAs). - PowerPoint PPT Presentation

Citation preview

Page 1: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

1

SurroundSense: Mobile Phone Localization via Ambience Fingerprinting

Ionut Constandache

Co-authors: Martin Azizyan and Romit Roy Choudhury

Page 2: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

2

Context

Pervasive wireless connectivity+

Localization technology=

Location-based applications

Page 3: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

3

Location-Based Applications (LBAs)

For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks

iPhone AppStore: 3000 LBAs, Android: 500 LBAs

Page 4: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

4

Location-Based Applications (LBAs)

For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks

iPhone AppStore: 3000 LBAs, Android: 500 LBAs

Location expresses context of user Facilitates content delivery

Page 5: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

5

Location is an IP addressLocation is an IP addressAs if for content delivery

Page 6: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

6

Thinking about Localizationfrom an application perspective…

Page 7: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

7

Emerging location based apps need place of user, not physical location

Starbucks, RadioShack, Museum, Library

Latitude, Longitude

Page 8: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

8

Emerging location based apps need place of user, not physical location

Starbucks, RadioShack, Museum, Library

Latitude, Longitude

We call this Logical Localization …We call this Logical Localization …

Page 9: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

9

Can we convert from Physical to Logical Localization?

Page 10: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

10

Can we convert from Physical to Logical Localization?

State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-

100m

State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-

100m

Page 11: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

11

Widely-deployable localization technologies have errors in the range of several meters

Widely-deployable localization technologies have errors in the range of several meters

Can we convert from Physical to Logical Localization?

State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-

100m

State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-

100m

Page 12: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

12

Several meters of error is inadequate to logically localize a phone

Physical LocationError

Page 13: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

13

Several meters of error is inadequate to logically localize a phone

RadioShackStarbucks

Physical LocationError

The dividing-wall problem The dividing-wall problem

Page 14: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

14

Contents

SurroundSense

Evaluation

Limitations and Future Work

Conclusion

Page 15: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

15

Contents

SurroundSense

Evaluation

Limitations and Future Work

Conclusion

Page 16: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

16

It is possible to localize phones by sensing the ambience

Hypothesis

such as sound, light, color, movement, WiFi …

Page 17: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

17

Sensing over multiple dimensions extracts more information from the ambience

Each dimension may not be unique, but put together, they may provide a

unique fingerprint

Page 18: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

18

SurroundSense

Multi-dimensional fingerprint Based on ambient sound/light/color/movement/WiFi

Starbucks

Wall

RadioShack

Page 19: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

19

BB AACC DDEE

Should Ambiences be Unique Worldwide?

FFGG

HHJJ

II

LLMMNN

OO

PPQQ

QQ

RR

KK

Page 20: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

20

Should Ambiences be Unique Worldwide?

BB AACC DDEE

FFGG

HHJJ

II

KKLL

MMNNOO

PPQQ

QQ

RR

GSM provides macro location (strip mall) SurroundSense refines to Starbucks

Page 21: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

21

++

Ambience Fingerprinting

Test Fingerprint

Sound

Acc.

Color/Light

WiFi

LogicalLocation

Matching

FingerprintDatabase

==

Candidate Fingerprints

GSM Macro Location

SurroundSense Architecture

Page 22: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

22

Fingerprints

Sound:(via phone microphone)

Color:(via phone camera)

Amplitude Values-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

N

orm

aliz

ed

C

ount

0.14

0.12

0.1

0.08

0.06

0.04 0.02

0

Acoustic fingerprint (amplitude distribution)

Color and light fingerprints on HSL space

Lightn

ess

1

0.5

0

Hue

0

0.5

1 00.2

0.4

0.6

0.8

1

Saturation

Page 23: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

23

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Moving

Page 24: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

24

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Moving

Queuing

Page 25: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

25

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Queuing Seated

Moving

Page 26: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

26

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Pause for product browsing

Moving

Page 27: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

27

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Pause for product browsing

Short walks between product browsing

Moving

Page 28: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

28

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Walk more

Moving

Page 29: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

29

Fingerprints

Movement: (via phone accelerometer)

Cafeteria Clothes Store Grocery Store

Static

Walk more Quicker stops

Moving

Page 30: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

30

Fingerprints

Movement: (via phone accelerometer)

WiFi: (via phone wireless card)

Cafeteria Clothes Store Grocery Store

Static

ƒ(overheard WiFi APs)

ƒ(overheard WiFi APs)

Moving

Page 31: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

31

Discussion

Time varying ambience Collect ambience fingerprints over different time

windows

What if phones are in pockets? Use sound/WiFi/movement Opportunistically take pictures

Fingerprint Database War-sensing

Page 32: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

32

Contents

SurroundSense

Evaluation

Limitations and Future Work

Conclusion

Page 33: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

33

Evaluation Methodology

51 business locations 46 in Durham, NC 5 in India

Data collected by 4 people 12 tests per location

Mimicked customer behavior

Page 34: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

34

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

A

ccura

cy (

%)

Cluster

Localization accuracy per cluster

Page 35: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

35

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

A

ccura

cy (

%)

Cluster

Localization accuracy per cluster

Multidimensional sensing

Page 36: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

36

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

Fault tolerance

A

ccura

cy (

%)

Cluster

Localization accuracy per cluster

Page 37: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

37

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

A

ccura

cy (

%)

Cluster

Localization accuracy per cluster

Sparse WiFi APs

Page 38: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

38

Evaluation: Per-Cluster Accuracy

Cluster

No. of Shops

1 2 3 4 5 6 7 8 9 10

4 7 3 7 4 5 5 6 5 5

No WiFi APs

Acc

ura

cy (

%)

Cluster

Localization accuracy per cluster

Page 39: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

39

Evaluation: Per-Scheme Accuracy

Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS

Accuracy 70%

74% 76% 87%

Page 40: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

40

Evaluation: User Experience

Random Person Accuracy

Average Accuracy (%)0 10 20 30 40 50 60 70 80 90 100

1 0.9 0.8 0.7 0.6 0.5

CD

F

0.4 0.3 0.2 0.1 0

WiFISnd-Acc-WiFiSnd-Acc-Clr-LtSurroundSense

Page 41: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

41

Economics forces nearby businesses to be different

Not profitable to have 3 coffee shopswith same lighting, music, color, layout, etc.

SurroundSense exploits this ambience diversity

Why does it work?

The Intuition:

Page 42: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

42

Contents

SurroundSense

Evaluation

Limitations and Future Work

Conclusion

Page 43: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

43

Limitations and Future Work

Energy-Efficiency

Localization in Real Time

Non-business locations

Page 44: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

44

Limitations and Future Work

Energy-Efficiency Continuous sensing likely to have a large energy draw

Localization in Real Time

Non-business locations

Page 45: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

45

Limitations and Future Work

Energy-Efficiency Continuous sensing likely to have a large energy draw

Localization in Real Time User’s movement requires time to converge

Non-business locations

Page 46: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

46

Limitations and Future Work

Energy-Efficiency Continuous sensing likely to have a large energy draw

Localization in Real Time User’s movement requires time to converge

Non-business locations Ambiences may be less diverse

Page 47: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

47

Contents

SurroundSense

Evaluation

Limitations and Future Work

Conclusion

Page 48: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

48

SurroundSense

Today’s technologies cannot provide logical localization

Ambience contains information for logical localization

Mobile Phones can harness the ambience through sensors

Evaluation results: 51 business locations, 87% accuracy

SurroundSense can scale to any part of the world

Page 49: SurroundSense: Mobile Phone Localization via Ambience Fingerprinting Ionut Constandache

49

Questions?

Thank You!

Visit the SyNRG research group @http://synrg.ee.duke.edu/