E-eyes : Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures Presenter: Yan Wang Yan Wang †, Jian Liu †, Yingying

DAISYData Analysis and Information SecuritY Lab

E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained

WiFi Signatures

Presenter: Yan Wang

Yan Wang†, Jian Liu†, Yingying Chen†, Marco Gruteser‡, Jie Yang#, Hongbo Liu*

†Dept. of ECE, Stevens Institute of Technology‡ WINLAB, Rutgers University

# Dept. of CS, Florida State University* Indiana University-Purdue University Indianapolis

MobiCom 2014Maui, Hawaii

Sept. 7th – 11th 2014

Motivation and Applications

2

Our Goal: Low-Cost Fine-Grained Activity Identification

3

Scalability / Infrastructural costLow cost High cost

Fin

e-g

rain

edC

oar

se-g

rain

ed

Gra

nu

lari

ty o

f th

e so

luti

on

s

Activity sensorsAttached sensors

Non-attached sensors

Localization/classificationusing specialized devices

WiSee, WiTrack

Localization

RTI

Localization using off-the-shelf WiFi

Device-free passive localization

RSS-based approach

Optimal solution

Our E-eyes

Nonintrusive Intrusive

Intuition and Basic Idea

4

Increasing availability of WiFi signals in home environments WiFi provides fine-grained channel state information (CSI) Use CSI to capture changes of multipath environment

Wall

Accesspoint

WallSmart

Appliance

Direct pathWiFi

deviceReflected

rays

Reflected rays

5 10 15 20 25 300

10

20

30

40

Subcarrier Index

SN

R

5 10 15 20 25 300

10

20

30

40

Subcarrier Index

SN

R

5 10 15 20 25 300

10

20

30

40

Subcarrier Index

SN

R

0 5000 100001280

1285

1290

1295

CSI.rate

Pac

ket

inde

x

CSI.rate

CS

I Am

plit

ude

...

Sub-carrier 1 Sub-carrier 2 Sub-carrier P-1 Sub-carrier P

Wall

Accesspoint

Wall

Direct pathWiFi

deviceReflected

rays

Reflected rays

5

Uniqueness of CSI Comparing to RSS

0

5

10

15

20

25Washing dishes

Talking on the phone

RS

S a

mpl

itude

0

5

10

15

20

25

30Washing dishes

Talking on the phone

CS

I Am

plitu

de

E-eyes System Challenges

Profile uniqueness and Robustness Generality to different types of activities Assisting the profile generation and updating

6

0 50 100 150 200 2500

5

10

15

20

25

30

Packet index

Sub

-car

riers

0 50 100 150 200 2500

5

10

15

20

25

30

Packet index

Sub

-car

riers

Activity Identification

Coarse Activity Determination Generality to different Activities

System Overview

7

Access Point Signal Time Series

Data Pre-processingIncrease robustness to real environments

Assisting the profile generation and updating

Profile Construction and Updating

User Feedback

Profile matching

None Profile Based

Construction

Adaptive

Updating

Walking Activity

Tracking using MD-

DTW

In-place Activity

Identification using EMD

Data FusionCrossing

Links

Known Activity Unknown Activity

Walking activity In-place activity

Coarse Activity Determination

8

Walking activity Large moving variance due to significant body movements and

location changes In-place activity

Small moving variance due to smaller body movements

… …

CS

I Am

pli

tud

e

Time

Subcarrier p

CS

I Am

pli

tud

e

Time

Subcarrier P

CS

I Am

pli

tud

e

Time

Subcarrier 1

)1(V … …)( pV )(PV

P

ppV

1)(V

CS

I Am

plit

ud

e

Time

In-place activity

Walking activity

Characteristics of CSI Measurements from Walking Activity

9

Trajectory 1 Trajectory 2

CSI pattern is dominated by walking activities’ path Doorway profile can facilitate walking activity tracking

0 50 100 150 200 2500

5

10

15

20

25

30

Packet index

Sub

-car

riers

0 50 100 150 200 2500

5

10

15

20

25

30

Packet index

Sub

-car

riers

Walking Activity Tracking

10

Walking ActivityClassifier

Multi-Dimensional Dynamic Time Warping Distance

Derivation

CSI measurements

DTW distance

Activity Profiles

CS

I Am

pli

tud

e

Time

Subcarrier P

CS

I Am

pli

tud

e

Time

Subcarrier 1

… …C

SI A

mp

litu

de

Time

Subcarrier p

CS

I Am

plit

ud

e

Time

Subcarrier P

CS

I Am

plit

ud

e

Time

Subcarrier 1

… …

CS

I Am

plit

ud

e

Time

Subcarrier p

Characteristics of CSI Measurements from In-Place Activity

11

Different in-place activities cause different distributions of CSI Different rounds of same in-place activities result in similar

distributions of CSI

0 2 4 6 8 10 12 14 16 18 200

5

10

15

CSI Amplitude Bins

Co

un

ts

0 2 4 6 8 10 12 14 16 18 200

5

10

15

CSI Amplitude Bins

Co

un

ts

Raw dataQuantized dataC

SI A

mp

litu

de

Time

In-place ActivityClassifier

CSI Amplitude BinsC

ou

nts

Distribution

In-Place Activity Identification

12

Distribution of CSI Amplitudes Extraction

Subcarrier Earth Mover’s Distance Derivation

CSI measurements

EMD distance

Activity Profiles

CSI measurements

CSI Amplitude Bins

Co

un

ts

Activity profile

CSI Amplitude Bins

Co

un

ts

Activity Identification

Profile Construction and Updating

Non-profiling Clustering

Semi-supervised approach to cluster daily activities and update CSI profiles

Construct CSI profiles when our system starts

13

Constructing profiles

Unknown Activity

Non-profiling Clusterin

gAdaptive Updating

User Feedback

Questions

How robust is the system in typical indoor environments?

Can two different activities be distinguished at the same location?

Is WiFi traffic in home environment feasible to identify activities?

14

Experimental Setup

WiFi devices Intel 5300 NIC + Thinkpad T500 and T 51 Cisco E2500

Scenarios Small apartment with one bedroom Large apartment with two beddoms

15

Questions


Can different activities be distinguished at the same location?


16

Performance of In-place Activity Identification in Two Different Apartments

17

2-bedroom apartment

Activity typesT

PR

1-bedroom apartment

Activity types

TP

R

a b c d e f g o0

0.2

0.4

0.6

0.8

1

Single WiFi device Mult. WiFi device

a b f g h i j o0

0.2

0.4

0.6

0.8

1

Single WiFi device Mult. WiFi device

False positive rate: less than 5%

Performance of Walking Activity Tracking and Doorway Identification

18

1-bedroom apt. A B C D Unknown Door Door1 Door2 Door3

A 1 0 0 0 0Door1 1 0 0

B 0 1 0 0 0

C 0 0 0.95 0.05 0Door2 0 0.975 0.025

D 0 0 0 1 0

O 0 0 0.1 0 0.9 None 0 0 1

2-bedroom apt. E F G H Unknown Door Door1 Door2 Door3

E 1 0 0 0 0Door3 1 0 0

F 0.15 0.85 0 0 0

G 0 0 0.9 0.1 0Door4 0 0.875 0.125

H 0 0 0 1 0

O 0.05 0 0.1 0 0.9t None 0 0 1

Questions




19

Performance of Identifying Different Activities at the Same Location

20

Four in-place activities Sleeping on the bed Sitting on the bed Receiving calls nearby the sink Washing dishes nearby the sink

4 5 6 7 8 9 10 11 120

0.20.40.60.8

1

True Positive Rate

Number of EMD bins

TP

R

4 5 6 7 8 9 10 11 120

0.10.20.30.40.5

False Positive Rate

Number of EMD bins

FP

R

Questions




21

Performance of Different Packet Rate

22

Packet transmission rate (PTR): 5 pkts/s - 20 pkts/s

5 10 15 200.75

0.8

0.85

0.9

0.95

1

Average true positive rate

5 10 15 200

0.02

0.04

0.06

0.08

0.1

Average false positive rate

Conclusion

Show that the channel state information (CSI) from off-the-shelf 802.11n devices can be utilized to identify and distinguish in-place activities inside a home

Develop a monitoring framework that can run on a single WiFi AP and use the associated profile matching algorithms to compare amplitude profiles against those from known activities

Explore dynamic profile construction to accommodate the movement or replacement of wireless devices and day-to-day profile calibration

Extensive experiments in two apartments of different size demonstrates the generality of our approach

23

24

Yan [email protected]

Documents

E-eyes : Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures Presenter: Yan Wang Yan Wang †, Jian Liu †, Yingying