University of Rome Tor Vergata Department of Electronics ... · Dr. Simone Di Domenico 27 / 39 University of Rome "Tor Vergata" Department of Electronics Engineering. ... Spectral

;

University of Rome ”Tor Vergata”Department of Electronics Engineering

Physical analytics: Internet of Things e Data Analytics

Passive Device-Free Radio Frequency-based Human Sensing

Dr. Simone Di Domenico

ISCOM - 22/06/2018

Introduction Problem and Motivation Background RF sensing applications and experimental results

Presentation outline

• Introduction;

• Problem and Motivation;

• Background;

• RF sensing applications and experimental results:

◦ Activity recognition and people counting based on the passive use ofWiFi signals.

◦ Through-The-Wall presence detection using WiFi signals.

◦ Algorithms to reduce the need of training for activity recognition andpeople counting.

◦ People counting through the opportunistic use of LTE signals.

Dr. Simone Di Domenico 2 / 39

University of Rome ”Tor Vergata” Department of Electronics Engineering


Human Activity Recognition

Activity recognition is the process which aims to recognize the activityperformed by one or more users in a given area using the data collectedfrom one or multiple sensors.




Activity Recognition Approaches

• Device-based: user wears on-body sensors and is cooperative.

• Device-free: user does not carry any sensors on his body.

◦ Active: transmitter is under control of AR system.

◦ Passive: RF signals already present in the environment are exploited.




Passive Device-Free Sensing

Pros

3 No dedicated transmitter;

3 No concern of privacy;

3 Through the wall and NLOS sensing;

3 360◦ field of view.

Cons

7 Lower accuracy;

7 Need of dedicated training for each propagation environment.

Objectives

• Develop new models to improve recognition accuracy.

• Develop new models to reduce the need of training.




Physical Principle

Each human activity changes the multipath propagation betweentransmitter and receiver imprinting a specific pattern on the CFR.




Validation of the Approach through Ray-TracingSimulator (1)

Figure: Amplitude of the CIR for the ”empty” and ”1 static person” scenarios.




Validation of the Approach through Ray-TracingSimulator (2)

Figure: CIRs overlapped for 3 consecutive instants of time for ”1 moving person”.




Sources of data

• Signals of opportunity:◦ WiFi signal emitted by an AP already present in the environment;◦ LTE signal emitted by an eNodeB ”close” to the target area.

• Signal measurements:◦ Received Raw Signal Sample (RRSS);◦ Channel State Information (CSI);◦ Reference Signal Received Power (RSRP);◦ Received Signal Strength Indicator (RSSI).

• SDR receivers:◦ IEEE 802.11b Beacon receiver (Matlab/Simulink);◦ IEEE 802.11n Receiver (Intel NIC 5300);◦ LTE receiver (C++).




Activity recognition and people counting based on thepassive use of WiFi signals.




Architecture & Setup

RRSS CSI

# TX/RX 1/1 2/3Carrier [GHz] 2.4 2.4BW [MHz] 20 20Rate [pkt/s] ≈ 10 ≈ 100Window [s] 10 10# Envs 3 3

ActivitiesEmpty, Sit,

Stand, WalkEmpty, Sit,

Stand, WalkCrowd size up to 5 up to 7




Processing

CSI RRSS

ActivityRecognition

Doppler-based approach

(i) Preprocessing(ii) Doppler spectrum estimation

over a sliding window(iii) Shape extraction(iv) Feature Selection(v) Classification

Shape-based approach

(i) Spectral estimation(ii) Shape extraction (MPEG-7)(iii) Statistical analysis over

sliding window(iv) Feature Selection(v) Classification

PeopleCounting

Doppler-based approach

(i) Preprocessing(ii) Doppler spectrum estimation

over a sliding window(iii) Shape extraction(iv) Feature Selection(v) Classification

Distance-based approach

(i) Spectral estimation(ii) Distance between spectra(iii) Statistical analysis

over a sliding window(iv) Feature Selection(v) Classification




Scatter Plots

(a) Activity Recognition - CSI Doppler-based features. (b) Activity Recognition - RRSS shape-based features.

(c) People Counting - CSI Doppler-based features. (d) People Counting - RRSS distance-based features.




Classification Results

CSI RRSS

ActivityRecognition

• Naıve Bayes classifier(50% train, 50% test)• Classes:{empty,sit,stand,walking}• Room A(30m2) avg acc. 77%• Room B(30m2) avg acc. 92%• Room C(45m2) avg acc. 87%

• Naıve Bayes classifier(50% train, 50% test)• Classes:{empty,sit,stand,walking}• Room A(30m2) avg acc. 93%• Room C(45m2) avg acc. 88%• Room E(200m2) avg acc. 78%

PeopleCounting

• Naıve Bayes classifier(50% train, 50% test)• Classes:{0p,1p,2p,(3,4)p,(5,6,7)p}• Room A (30m2) avg acc. 92%• Room C (45m2) avg acc. 89%• Room D (75m2) avg acc. 73%

• Naıve Bayes classifier(50% train, 50% test)• Classes:{0p,1p,2p,(3p,4p,5p)}• Room A (30m2) avg acc. 78%• Room B (45m2) avg acc. 87%




Through-The-Wall presence detection using WiFisignals.





CSI

Carrier [GHz] 2.4BW [MHz] 20# TX/RX 2/3Rate [pkt/s] ≈ 100Window [s] 10# Setups 3

ActivitiesEmpty,Static,

Dynamic




Processing




Scatter Plot

Figure: Scatter plot of the selected features for the Setup C - Double TTW scenario.Dr. Simone Di Domenico 18 / 39



Classification ResultsPrediction

EmptyRoom

Presence(Static)

Presence(Dynamic)

EmptyRoom

1 0 0

Tru

th

Presence(Static)

0 1 0

Presence(Dynamic)

0 0 1

1

Table: Confusion matrix for Setup A.

PredictionEmptyRoom

Presence(Static)

Presence(Dynamic)

EmptyRoom

0.98 0.02 0

Tru

th

Presence(Static)

0 0.97 0.03

Presence(Dynamic)

0 0 1

0.98

Table: Confusion matrix for Setup B.

PredictionEmptyRoom

Presence(Static)

Presence(Dynamic)

EmptyRoom

0.98 0.02 0

Tru

th

Presence(Static)

0.02 0.98 0

Presence(Dynamic)

0 0 1

0.99

Table: Confusion matrix for Setup C.

• Effectiveness of the method inrecognizing the presence of aperson in a TTW scenario.

• Detection of stationary humansis challenging also in not TTWscenarios.

• Presence detection is possibleeven in double TTW scenario.




Algorithms to reduce the need of training for activityrecognition and people counting.




Training options

Three different types of training options:

• Trained : recognition system is trained and tested in the sameenvironment and conditions.

• Trained-once: recognition system is trained in one environmentunder certain conditions, and is then tested in differentenvironments and/or conditions.

• Training-free: recognition system is based on anenvironment-independent model and requires lighter forms oftraining.◦ ”Empirically-modeled”◦ ”Fully-modeled”




Trained-once - Key Idea

Problem: Find features that are more sensitive to the change of thepropagation environment and less sensitive to the characteristics of the

environment in static conditions.

Solution: Each person leads to a temporal variation of the CFR.

The higher the number of moving people/intensity of movement, the higher thenumber of intercepted multipath components and hence, the higher the temporal

variation of the CFR.

(i) CFR variation is correlated to the number of people/human activity.(ii) CFR variation is poorly correlated to the environment.

A trained-once model can be developed by exploiting the CFR variation.

Doppler spectrum is used to measure the CFR variation.




Trained-once - Doppler Spectrum

Figure: Mean Doppler Spectrum for different activities.




Trained-once - Doppler Spectrum

Figure: Mean Doppler Spectrum for different activities.




Trained-once - Doppler approach




Trained-once - Scatter Plots

Figure: Spectral Skewness and Kurtosis for Empty and 1 person moving scenarios fordifferent rooms.




Trained-once - Classification Results

Training Env.Testing Env.

Room A Room B Room C

Room A - 75% 71%Room B 72% - 79%Room C 71% 88% -

Classes = {Empty-Sit, Stand, Walking}

Table: Activity Recognition performances using the Trained-once approach.

Training Env.Testing Env.

Room A Room C Room D

Room A - 72% 70%Room C 81% - 68%Room D 73% 58% -

Classes = {Empty, 1 person, (2,3,4) people, (5,6,7) people}

Table: People Counting performances using the Trained-once approach.




”Empirically-modeled” - Key Idea

Problem: Find ”physical features” that are independent to the environmentand are a monotone function of the degree of motion, making possible the use

of a threshold-based classifier.

Solution: Doppler spectrum tends to broaden as the intensity ofmovements/number of people increases.

Spectral spreading can be used as a metric to build the training-free model.

RMS Doppler Bandwidth and Spectral Spread measure the spectral broadeningand have been selected to build the model.

There are no mathematical models that relate these quantities to the humanactivities/numbers of people, then a calibration to find out the threshold for the

classifier is needed.




”Empirically-modeled”-Scatter Plots

(a) Human activity-Room A. (b) Human activity-Room B. (c) Human activity-Room C.

(d) People counting-Room A.(e) People counting-Room C.(f) People counting-Room D.




”Empirically-modeled”- ResultsEmpty Sit/Stand Walk

Empty 1 0 0Sit/Stand 0.37 0.63 0

Walk 0 0.02 0.980.87

Table: Human activity - Room A.

Empty Sit/Stand WalkEmpty 1 0 0

Sit/Stand 0.45 0.55 0Walk 0 0 1

0.85

Table: Human activity - Room B.

Empty Sit/Stand WalkEmpty 1 0 0

Sit/Stand 0.28 0.71 0.01Walk 0 0 1

0.72

Table: Human activity - Room C.

Empty 1 p >1 pEmpty 1 0 0

1 p 0 0.92 0.08>1 p 0 0 1

0.97

Table: People counting - Room A.

Empty 1 p >1 pEmpty 1 0 0

1 p 0 0.76 0.24>1 p 0 0 1

0.92

Table: People counting, Room C.

Empty 1p >1 pEmpty 1 0 0

1 p 0 1 0>1 p 0 0.20 0.80

0.93

Table: People counting, Room D.Dr. Simone Di Domenico 30 / 39



”Fully-modeled” - Key Idea

Problem: Find a mathematical model that relates one or more physicalquantities to each activity/number of people.

Solution: Such a model does not exist yet, but Doppler energy spectrum describeshow speedily scatterers move and, hence, its shape can reveal the presence of

moving scatterers.

(i) Empty scenario: Doppler energy is concentrated around 0 Hz since there areno movements.

(ii) 1 or more persons: Doppler energy is spread out over a wider range offrequencies.

Spectral Rolloff (SR) is a physical quantity which measures in Hz the Dopplerspreading and hence, can be used to build our simple model without the need of

calibration or training:

Predicted scenario =

{”Empty” if SR = 0

”1 or more persons” if SR! = 0




”Fully-modeled” - Scatter Plot

Figure: Spectral Rolloff for ”empty” and ”1 moving” person scenarios.




People counting through the opportunistic use of LTEsignals.




Introduction & Motivation

WiFi signal has been widely used as source of opportunity fordeveloping RF activity recognition system, but its availability is not

always guaranteed.

LTE signal is an excellent candidate as a signal of opportunity thanksto its wide availability and good penetration in indoor environments.




Key Idea

Amplitude variation of the LTE signal is strongly correlated to what happens inproximity to the transmitter and/or receiver, therefore the LTE received signal can

be used to estimate the number of people nearby the receiver.

LTE uses the Cell specific Reference Signal (CRS) to compute the CSI, which isthen used to compute the RSRP as follows:

RSRP c(n) =1

Nsub

Nsub∑j=1

|hcj(n)|2 (2)

where n is the discrete time index and c stands for the c-th CRS channel.

Since the RSRP is computed on the CSI, its variations depends only on thechanges along the propagation environment.

RSRP variations can be evaluated by dispersion-based sample moments toestimate the number of people inside the room.




RSRP variations

Figure: RSRP histograms for different numbers of people.Dr. Simone Di Domenico 36 / 39




• Very low cost SDR receiver.

• No SIM card needed.

• People were free to move.

• eNodeB with the highest SNR.

• Room size was 5m x 9m.

# Carrier [MHz] 796

# BW [MHz] 1.4

# TX/RX 2/1

Rate [RSRP/s] 2000

Window [s] 15

# Envs 1

# Setups 4

Crowd size Up to 5




Scatter Plot

Figure: Standard deviation and Fano factor of the RSRP for different numbers of people.




Classification Results

PredictionEmpty 1p 2p {3,4,5}p

Empty 1 0 0 0

Tru

th 1p 0 0.38 0.05 0.572p 0 0.26 0.62 0.13

{3,4,5}p 0 0 0.27 0.730.72

Table: Confusion Matrix for Position 1.


Empty 1 0 0 0

Tru

th 1p 0 0.65 0 0.352p 0 0.05 0.91 0.04

{3,4,5}p 0 0 0.08 0.920.87



Empty 1 0 0 0

Tru

th 1p 0 0.87 0.13 02p 0 0.02 0.98 0

{3,4,5}p 0 0 0.05 0.950.95


PredictionEmpty 1p {2,3,4,5}p

Empty 1 0 0

Tru

th 1p 0 0.40 0.60{2,3,4,5}p 0 0.13 0.87

0.75

Table: Confusion Matrix for TTWscenario.

Achieved average accuracy ranges from 72% to 95%, including themore challenging TTW scenario.



Documents

University of Rome Tor Vergata Department of Electronics ... · Dr. Simone Di Domenico 27 / 39 University of Rome "Tor Vergata" Department of Electronics Engineering. ... Spectral