SurroundSense: Mobile Phone Localization via Ambience

Fingerprinting

MARTIN AZIZYAN, IONUT CONSTANDACHE, ROMIT ROY CHOUDHURY

Presented by Lingfei Wu

Outline

Introduction SurroundSense Architecture System Design Implementation Evaluation Conclusion

Introduction

Notion of location Physical coordinates(latitude/longitude) Logical labels(like Starbucks, Mcdonalds)

Many applications based on logical location

Application of logical localization

Introduction Physical coordinate can be reversed to logical location. However, it often causes error ! Why not compute logical location directly?

Relative work

Active RF Install special hardware Ultrasound, Bluetooth

Passive RF GPS, GSM or WIFI based

Behavior Sensing Imaging matching

1. Lack accuracy

2. Need pre-installed infrastructure

Motivation Combine effect of ambient sound, light, color, user motion

Sound (microphone) Starbucks VS Bookstore

Light / Color (camera) Different thematic light, colors and floors.

Human movement (accelerometer) Wal-Mart VS McDonald

Place may not be unique based on any one attribute The combination can be unique enough for localization

In this paper, we propose SurroundSense for logical localization. Starbucks

McDonald’s

Bookstore

Wal-Mart

SurroundSense Architecture

1.Xxx2.Yyy3.zzz

Candidate list

1.Xxx2.Yyy

1.Xxx2.Yyy

1.Xxx2.Yyy

1.Xxx

System Design

Fingerprint generation Fingerprinting sound Fingerprinting motion using accelerometers Fingerprinting color/light using cameras Fingerprinting Wi-Fi

Fingerprint matching Wi-Fi filter Sound filter Motion filter Color/light Match

Fingerprinting sound Convert signals to time domain

100 normalized values as feature of sound Similarity of fingerprints

Compute 100 pair-wise distance between test fingerprint and all candidate fingerprint

50

0

-50

Normalized amplitude value

N

orm

aliz

ed o

ccur

renc

e co

unt

time

ampl

itud

e va

lue

time

Dim 1 2 3 … … 100

A 0.1 0.2 0.1 … … 0.05

B 0.6 0.3 0.2 … … 0.1

Fingerprinting Sound

Unreliable to be a matching scheme

Sound from the same place can vary over time.

Only use as a filter If distance > threshold τ

then discard from the candidate list

Fingerprinting Motion

Use support vector machine(SVM) as classifier

Sequence of states as user’s moving pattern Movement is prone to fluctuation

In a clothing store, Some users browse for a long time while others purchase clothes in haste.

Only use as a filter

SVM

Raw data

moving

stationary

1

-1

Fingerprinting Motion

Compute motion fingerprint: Ratio = tmoving / tstatic Bucket 1: 0.0 <= Ratio <= 0.2 Sitting (cafe) Bucket 2: 0.2 <= Ratio <= 2.0 Browsing (clothing) Bucket 3: 2.0 <= Ratio <= ∞ Walking (grocery)

Sitting Browsing Walking

Fingerprinting Color / Light

Thematic color and lighting in different stores Where to capture the picture?

random picture of surrounding floor

Advantages of taking floor pictures Privacy concern Less noisy Rich diversity in floor color Easy to obtain

too much noise

Fingerprinting Color / Light How to extract colors and light intensity?

RGB HSL(Hue-Saturation-Lightness)

Find color cluster and its size using K-means clustering algorithm

k=2

sk-sk-1 < t

k-mean clusteringk++

no

yes

sk: the sum of distance from all pixels to their (own cluster’s) centroid.

t: convergence threshold

< c1, c2 …, ck >Bean Trader’s Coffee shop

too much noise

Fingerprinting Color / Light

Similarity of fingerprints Assume C1 = {c11, c12, …, c1n}; C2 = {c21, c22, …, c2m}

Fingerprint matching The candidate list with maximum similarity is

declared to the matching fingerprint

Total size in C1 or C2distance of centroid

Fingerprinting Wi-Fi

Wi-Fi fingerprint Record MAC address from APs every 5 second

Fingerprint tuple:<{AP1_MAC_Addr, AP1_fraction_time}, {AP2_MAC_Addr, AP2_fraction_time}, {AP3_MAC_Addr, AP3_fraction_time}>

Fingerprinting Wi-Fi

Similarity of fingerprints

Use as filter/matching module In the absence of light/color, we use it as matching module. Accuracy depend on location of shops.

M: union of MAC address of fingerprints f1 and f2fraction of time

Implementation

Client and server Client: Nokia 95 phones using Python as client Server: Matlab and Python code and some data

mining tools for fingerprinting algorithms. Fingerprint database

Labor-intensive war-sensing at 51 stores Store location: 46 business location in

university town, 5 location in India

Implementation

Evaluation

SurroundSense(SS) test environment War-sensed 51 shops organized in 10 clusters

4 students visited the first nine clusters in university town, while 2 students visited the tenth cluster in India.

4 localization models: Wi-Fi only (Wi-Fi) Sound, Accelerometer, Light and color ( Snd-Acc-Lt-Clr) Sound, Accelerometer, Wi-Fi (Snd-Acc-Wi-Fi) SurroundSense (SS)

Evaluation – Per-Cluster Accuracy

Best represented

Restaurant

Similar hardwood floor in strip mall Same AP False negative

Snd and Acc

No Wi-Fi

Evaluation – Per-Shop Accuracy

To understand the localization accuracy on a per-shop basis

47% shops30% shops

SS: 92%Snd-Acc-WiFi: 92%Snd-Acc-Lt-Clr: 75%WiFi: 75%

Evaluation – Per-User Accuracy

Simulate 100 virtual user, each assign 4~8 stores from cluster 1~9

Evaluation – Per-Sensor Accuracy

Hand-picked 6 samples to exhibit the merits and demerits of each sensor

false negativePercentage localized using special sensors

Number of shops left after special filter

Conclusion

Presented SurroundSense, a non-conventional approach for logical localization.

Created fingerprints about ambient sound, light, color, movement and Wi-Fi and match them with fingerprint database to realize accurate logical localization.

The evaluation achieved an average location accuracy of over 85% using all sensors.

Discussion The GPS 10 m, Wi-Fi and GSM 40m and 400m respectively.

Why not use Wi-Fi to get initial location instead of using GSM?

Support vector machines (SVM), K-means clustering algorithm are used in paper, do you have any better machine learning methods? Such as Kalman filter, Particle filter, and Wavelet Transform?

Can other sensors help? Such as compass and Bluetooth? Energy consideration? Non-business location?