Upload
francis-thornton
View
215
Download
1
Embed Size (px)
Citation preview
1
SurroundSense: Mobile Phone Localization via Ambience Fingerprinting
2
Context
Pervasive wireless connectivity+
Localization technology=
Location-based applications
3
Location-Based Applications (LBAs)
For Example: GeoLife shows grocery list when near Walmart MicroBlog queries users at a museum Location-based ad: Phone gets coupon at Starbucks
iPhone AppStore: 3000 LBAs, Android: 500 LBAs
4
Most emerging location based apps do not care about the physical location
GPS: Latitude, Longitude
5
Most emerging location based apps do not care about the physical location
Instead, they need the user’s logical location
GPS: Latitude, Longitude
Starbucks, RadioShack, Museum, Library
6
Physical Vs Logical
Unfortunately, most existing solutions are physical
GPS GSM based SkyHook Google Latitude
RADAR Cricket …
7
Given this rich literature,
Why not convert from Physical to Logical Locations?
8
Physical LocationError
9
Pizza HutStarbucks
Physical LocationError
10
Pizza HutStarbucks
Physical LocationError
The dividing-wall problem The dividing-wall problem
11
SurroundSense:A Logical Localization Solution
12
It is possible to localize phones by sensing the ambience
It is possible to localize phones by sensing the ambience
Hypothesis
such as sound, light, color, movement, WiFi …
13
It is possible to localize phones by sensing the ambience
It is possible to localize phones by sensing the ambience
Hypothesis
such as sound, light, color, movement, WiFi …
14
Multi-dimensional sensing extracts more ambient information
Any one dimension may not be unique, but put together, they may provide a
unique fingerprint
15
SurroundSense
Multi-dimensional fingerprint Based on ambient
sound/light/color/movement/WiFi
Starbucks
Wall
Pizza Hut
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
16
BB AACC DDEE
Should Ambiences be Unique Worldwide?
FFGG
HHJJ
II
LLMMNN
OO
PPQQ
RR
KK
17
Should Ambiences be Unique Worldwide?
BB AACC DDEE
FFGG
HHJJ
II
KKLL
MMNNOO
PPQQ
RR
GSM provides macro location (strip mall) SurroundSense refines to Starbucks
18
Economics forces nearby businesses to be diverse
Not profitable to have 3 adjascent coffee shopswith same lighting, music, color, layout, etc.
SurroundSense exploits this ambience diversity
Why does it work?
The Intuition:
19
++
Ambience Fingerprinting
Test Fingerprint
Sound
Acc.
Color/Light
WiFi
LogicalLocation
Matching
FingerprintDatabase
==
Candidate Fingerprints
GSM Macro Location
SurroundSense Architecture
20
Fingerprints
Sound:(via phone microphone)
Color:(via phone camera)
Amplitude Values-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
N
orm
aliz
ed
C
ount
0.14
0.12
0.1
0.08
0.06
0.04 0.02
0
Acoustic fingerprint (amplitude distribution)
Color and light fingerprints on HSL space
Lightn
ess
1
0.5
0
Hue
0
0.5
1 00.2
0.4
0.6
0.8
1
Saturation
21
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
22
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Queuing Seated
Moving
23
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product browsing
Short walks between product browsing
Moving
24
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more Quicker stops
Moving
25
Fingerprints
Movement: (via phone accelerometer)
WiFi: (via phone wireless card)
Cafeteria Clothes Store Grocery Store
Static
ƒ(overheard WiFi APs)
ƒ(overheard WiFi APs)
Moving
26
Discussion
Time varying ambience Collect ambience fingerprints over different time
windows
What if phones are in pockets? Use sound/WiFi/movement Opportunistically take pictures
Fingerprint Database War-sensing
27
Evaluation Methodology
51 business locations 46 in Durham, NC 5 in India
Data collected by 4 people 12 tests per location
Mimicked customer behavior
28
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
29
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Fault tolerance
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
30
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
Sparse WiFi APs
31
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
No WiFi APs
Acc
ura
cy (
%)
Cluster
Localization accuracy per cluster
32
Evaluation: Per-Scheme Accuracy
Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS
Accuracy 70%
74% 76% 87%
33
Evaluation: User Experience
Random Person Accuracy
Average Accuracy (%)0 10 20 30 40 50 60 70 80 90 100
1 0.9 0.8 0.7 0.6 0.5
CD
F
0.4 0.3 0.2 0.1 0
WiFISnd-Acc-WiFiSnd-Acc-Clr-LtSurroundSense
34
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy
draw
35
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy
draw
Localization in Real Time User’s movement requires time to converge
36
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy
draw
Localization in Real Time User’s movement requires time to converge
Non-business locations Ambiences may be less diverse
37
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Electroic compasses can fingerprint layout Tables and shelves laid out in different orientations Users forced to orient in those ways
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Limitations and Future Work
38
Ambience can be a great clue about locationAmbient Sound, light, color, movement …
None of the individual sensors good enoughCombined they may be unique
Uniqueness facilitated by economic incentive
Businesses benefit if they are mutually diverse in ambience
Ambience diversity helps SurroundSenseCurrent accuracy of 89%
Conclusion
39
Questions?
40
SurroundSense evaluated in 51 business locationsAchieved 89% accuracy
But perhaps more importantly, SurroundSense scales to any part of the world
Summary
41
SurroundSense
Today’s technologies cannot provide logical localization
Ambience contains information for logical localization
Mobile Phones can harness the ambience via sensors More sensors, beter reliability
Evaluation results: 51 business locations, 87% accuracy
SurroundSense can scale to any part of the world
42
43
Evaluation: Per-Shop Accuracy
Localization Accuracy per Shop
Average Accuracy (%) 0 10 20 30 40 50 60 70 80 90 100
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
C
DF
WiFiSnd-Acc-WiFiSnd-Acc-Clr-LtSurroundSense
44
Architecture: Filtering & Matching
Candidate Fingerprints
45
Fingerprinting Sound
Fingerprint generation : Signal amplitude Amplitude values divided in 100 equal intervals Sound Fingerprint = 100 normalized values
• valueX = # of samples in interval x / total # of samples
Filter Metric: Euclidean distance Discard candidate fingerprint if metric > threshold г
Threshold г Multiple 1 minute recordings at the same location di = max dist ( any two recordings )
г = max ( di of candidate locations )
46
Fingerprinting Motion
Fingerprint generation: stationary/moving periods Sitting (restaurants, cafes, haircutters ) Slow Browsing (bookstores, music stores, clothing) Speed-Walking (groceries)
Filter Metric:
Discard candidate fingerprints with different classification
static
moving
t
tR =
0.0 ≤ R ≤ 0.2 sitting0.2 ≤ R ≤ 2.0 slow browsing
2.0 ≤ R ≤ ∞ speed-walking
47
Fingerprinting WiFi
Fingerprint generation: fraction of time each unique address was overheard
Filter/Ranking Metric Discard candidate fingerprints which do not have similar
MAC frequencies
48
Fingerprinting Color
Floor Pictures Rich diversity across different locations Uniformity at the same location
Fingerprint generation: pictures in HSL space K-means clustering algorithm Cluster’s centers + sizes
Ranking metric
49
Thinking about Localizationfrom an application perspective…
50
Emerging location based apps need place of user, not physical location
Starbucks, RadioShack, Museum, Library
Latitude, Longitude
51
Emerging location based apps need place of user, not physical location
Starbucks, RadioShack, Museum, Library
Latitude, Longitude
We call this Logical Localization …We call this Logical Localization …
52
Can we convert from Physical to Logical Localization?
53
Can we convert from Physical to Logical Localization?
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
54
Widely-deployable localization technologies have errors in the range of several meters
Widely-deployable localization technologies have errors in the range of several meters
Can we convert from Physical to Logical Localization?
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
State of the Art in Physical Localization:1. GPS Accuracy: 10m 2. GSM Accuracy: 100m3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-
100m
55
Physical Vs Logical
Lot of work in physical localization
GPS (5m accuracy, outdoor only) GSM based (high coverage, 200m accuracy) SkyHook (40m accuracy, need war-driving) Google Latitude (similar)
RADAR (5m accuracy, need indoor mapping) Cricket (install per-room devices) …
Each of them has distinct problems … Each of them has distinct problems …
56
Physical Vs Logical
Lot of work in physical localization
GPS (5m accuracy, outdoor only) GSM based (high coverage, 200m accuracy) SkyHook (40m accuracy, need war-driving) Google Latitude (similar)
RADAR (5m accuracy, need indoor mapping) Cricket (install per-room devices) …
Each of them has distinct problems … Each of them has distinct problems …
57
Even if we assume the best of all solutions
(5m accuracy indoors)
logical localization still remains a challenge.
58
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
59
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
60
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
61
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
62
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
63
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
A
ccura
cy (
%)
Cluster
Localization accuracy per cluster
Multidimensional sensing
64
Limitations and Future Work
Energy-Efficiency
Localization in Real Time
Non-business locations
65
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy
draw
Localization in Real Time
Non-business locations
66
Limitations and Future Work
Energy-Efficiency Continuous sensing likely to have a large energy
draw
Localization in Real Time User’s movement requires time to converge
Non-business locations
67
Limitations and Future Work
Camera inside pockets Detect phone when out of pocket Takes pictures when camera pointing downward