Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Robust Device-Free Indoor Localization
When Machine Learning Meets Wi-Fi
Steve Liu, VP R&D, Chief-Scientist
Samsung AI Center – Montreal &
Professor & William Dawson Scholar
McGill University
Nov. 5th, 2019
Disclaimer: Any views or opinions presented in this talk are personal and do not represent those of people, institutions, organizations that the presenter may or may not be associated with in professional or personal capacity.
Outline
Robust Device-Free Indoor Localization
• Indoor Localization & Background
– Fingerprinting-based Device-Free Localization Approach
• A Major Challenge: Environment Changes
• AutoFi
• Experiments
• More Recent Advances: Toward Robustness in Real World
• Conclusions & Future Work Directions
3
Indoor Localization & Background
Why this Topic?
The next big thing!
• Internet of Things
• AI and Machine Leaning (=> Intelligent)
• Advanced Communications like Wi-Fi 6 & 5G (=> Connected )
5
Indoor Localization
GPS signal not available
6
Applications
What can we do with location information?
Indoor navigation
Smart home automation
Automated industry
7
State-of-the-Art
Device-Oriented
• Pro: High accuracy: 10cm-level (e.g., [1],[2],[3])
• Con: Complicated design– Customized hardware
– Antenna arrays (10+ antenna)
– Usually heavy computational overhead
• Locating devices, NOT users
[1] M. Kotaru, K. R. Joshi, D. Bharadia, and S. Katti, “Spotfi: Decimeter level localization using WiFi”, Sigcomm ‘15. [2] X. Li, D. Zhang, Q. Lv, J. Xiong, S. Li, Y. Zhang, and H. Mei, “Indotrack: Device-free indoor human tracking with commodity WiFi,” IMWUT’ 17[3] K. Qian, C. Wu, Y. Zhang, G. Zhang, Z. Yang, and Y. Liu, “Widar2.0: Passive human tracking with a single WiFi link,” MobiSys ‘18.
8
Why Device-free Localization?
People are not always carrying devices
• Cannot, or Do not want to carry
9
State of the Art
Device-Oriented
Device-Free
• Triangulation methods– Utilize user’s blocking/reflection of the signals
– Calculating Time-of-Fight (ToF), Angle-of-Arrival (AoA), Propagation features, etc.
– Dedicated devices required
10
Disadvantages of Dedicated Devices
• Cost– Equipment cost
– Deployment
– Triangulation: positions of devices must be known in advance
• Scalability
• Range limitation (motion sensors, RFID, …)
• Blocked by walls
• …
Motion Sensors CamerasRFID
Tag & Reader
11
State of the Art
Device-Oriented
Device-Free
• Triangulation methods
• Fingerprinting-based method– Correlating environment features with human locations
– Environment features:
• Wireless signals, e.g., Wi-Fi
• Light or sound
• Magnetic field
• …
12
Benefits of Wi-Fi based DfP Localization
Work through walls!
Available Everywhere!
13
Wi-Fi Based DfP Localization
Fingerprinting: Associate Wi-Fi features such as CSI (Channel State Information) or
RSSI (Received Signal Strength Indication) with users’ locations.
14
Wi-Fi Channel State Information
Channel State Information (CSI)
• Describe the Wi-Fi channel properties
• Sensitive to human locations
• Provides much more information than RSSI
• Readily available from commercial Wi-Fi cards
15
1 RSSI → 1 integer
56 subcarriers with CSI values(complex numbers) → 56 magnitudes + 56 phases
→ 112 floating point numbers
More Background on Wi-Fi CSI
Wi-Fi Multiple-Input Multiple-Output (MIMO)
e.g., Intel 5300 NICe.g., Linksys WRT160N16
More Background on Wi-Fi CSI
MIMO in math
Received signals
Streams (Spatial Channels)CSI
Transmittedsignals
Noise
Source: http://www.sharetechnote.com/html/Communication_ChannelModel.html
𝒚 = 𝑯 𝒙 + 𝒏
17
How does CSI look like?
CSI profiles at h11
• Captures the math representation of channel characteristics from transmitter antenna 1 to receiver antenna 1
• Time/frequency variant
Time (seconds)
Frequency(Subcarriers)
CSIMagnitude
18
CSI Profiles are Location-Dependent
Subcarriers
CSI
Magnitude
19
CSI Profiles to CSI Fingerprints
A CSI fingerprint of location L is a CSI profile associated with this location
• i.e., fingerprint = CSI profile at location L
20
How to use ML for Localization?
Use CSI fingerprints to train ML classifiers
• Locations – labels (output), CSI profiles (input)
• SVM, Random Forest, KNN, Neural Networks, etc.
MLClassifiers
For training
21
Classification for Localization
Trained models to classify the labels of new (online measured) CSI profiles
• Predicted labels = estimated locations
Trained ML
Classifiers
10 locations, (sub-)meter-level resolution - 99.7% accuracy.22
A Major Challenge
- Environment Changes
A Major Challenge
The changes in the indoor environment make the (old) fingerprints inconsistent with
the current situation (new).
24
A Major Challenge
The CSI fingerprints will be “contaminated” by environment changes.
• The recorded fingerprints (old) no longer represent the changed environment (new)
25
A Major Challenge
The CSI fingerprints will be “contaminated” by environment changes.
• The recorded fingerprints (old) no longer represent the changed environment (new)
Before After
26
A Major Challenge
Localization accuracy drops significantly
• A case study:
Record the fingerprints again, and retrain the model?
27
Impractical: Inconvenient & time-consuming
AutoFi
Idea: Can We Reuse the Fingerprints?
Reconstruct vectors measured from the changed domain to the old fingerprint domain, so we can reuse the already trained model
𝑀𝑛𝑒𝑤−𝑜𝑙𝑑
-------------------------------------------------------------------------------------
New vector domain(Contaminated CSI profiles)
Old vector domain(fingerprints)
𝑀𝑛𝑒𝑤−𝑜𝑙𝑑
𝑉𝑒𝑚𝑝𝑡𝑦𝑜𝑙𝑑 𝑉𝑝1
𝑜𝑙𝑑
𝑉𝑒𝑚𝑝𝑡𝑦𝑛𝑒𝑤 𝑉𝑝1
𝑛𝑒𝑤
29
The Mapping Functions
Map CSI from the new domain to the old domain
Contaminated CSI profiles(locations unknown)
Unknown mapping How to determine?
𝑉𝑝1𝑜𝑙𝑑 = 𝑀𝑝1 × 𝑉𝑝1
𝑛𝑒𝑤 + 𝑛𝑝1,
𝑉𝑝2𝑜𝑙𝑑 = 𝑀𝑝2 × 𝑉𝑝2
𝑛𝑒𝑤 + 𝑛𝑝2 ,
⋮
𝑉𝑝𝑁𝑜𝑙𝑑 = 𝑀𝑝𝑁 × 𝑉𝑝𝑁
𝑛𝑒𝑤 + 𝑛𝑝𝑁 .
Fingerprints in Old Domain(associated with locations)
30
The Mapping Functions
Approximation: the mapping functions are the same for different locations
• Domain-to-domain mapping
𝑀𝑝1 = 𝑀𝑝2 = ⋯ = 𝑀𝑝𝑁 = 𝑀.
Contaminated domain (New) Fingerprint domain (Old)
M: New →Old
31
The Mapping Function
Mapping function: M
𝑉𝑝1𝑜𝑙𝑑 = 𝑀 × 𝑉𝑝1
𝑛𝑒𝑤 + 𝑛𝑝1,
𝑉𝑝2𝑜𝑙𝑑 = 𝑀 × 𝑉𝑝2
𝑛𝑒𝑤 + 𝑛𝑝2 ,
⋮
𝑉𝑝𝑁𝑜𝑙𝑑 = 𝑀 × 𝑉𝑝𝑁
𝑛𝑒𝑤 + 𝑛𝑝𝑁 .
32
Fingerprints in Old Domain(associated with locations)
Contaminated CSI profiles(locations unknown)
The Mapping Functions
How to automatically determine the mapping function M?
𝑉?𝑜𝑙𝑑 = 𝑀 × 𝑉?
𝑛𝑒𝑤 + 𝑛?
Both unknown!
• In order to get M, we need to automatically detect a location p as a
reference point such that both 𝑉𝑝𝑛𝑒𝑤𝑎𝑛𝑑 𝑉𝑝
𝑜𝑙𝑑are known
33
Determine M
Observation: The status of an empty area can be detected using a rule-based algorithm
• Reference point (state) = Detecting Empty
• Both before and after the environmental changes
• How? Using variance of CSI magnitude (can be detected automatically)
Associate 𝑉?𝑛𝑒𝑤 with ?=Empty => 𝑉𝑒𝑚𝑝𝑡𝑦
𝑜𝑙𝑑 = 𝑀 × 𝑉𝑒𝑚𝑝𝑡𝑦𝑛𝑒𝑤 + 𝑛
34
Contaminant Removal
𝑀 = 𝑀E𝑚𝑝𝑡𝑦
Contaminated profile (raw CSI measured in the newly changed environment)
𝑀
? ?
De-contaminated (purified) profile• Transformed CSI• In the old domain, i.e. when
there is no environ changes
Trained ML model
Determine Location
35
Contaminant Removal Example
An example of contaminant removal result @ P1
Still some residual errors!
36
Autoencoder
A neural network learns efficient data representation (encoding) of the input data
• Encoder
• Decoder
– To (re)generate data
37
Training the Autoencoder
The autoencoder tries to reconstruct the input
• Training: Inputs and outputs are the same – the fingerprint profiles recorded
• Learns an efficient representation in the recorded domain (fingerprint profiles)
• Use BP to train
Fingerprint profiles
The encoding
Fingerprint profiles
38
Denoising with the Autoencoder (Inference)
• The autoencoder denoises measured new CSI profiles
– Using the (already) purified profiles as input
– Old fingerprint domain features are identified through the coding/decoding
– Features related to the environment changes are diluted/omitted
New (purified) profiles Reconstructed profiles
The encoding
39
Putting All Components Together
The architecture of AutoFi (Training in red, localization in blue).
40
Experiments
Experiment Setup
A Linksys WRT160N router, a laptop with Intel 5300 NIC.
• Wi-Fi traffic: 10 – 20 pings per second.
• Meter-level resolution (minimum distance 80cm)
42
Experiment Setup
Introducing the “contaminants”
43
Experiment Results
Baseline, no mapping was applied, Random Forest (RF).
Opening windows Opening doors
Accuracy Mean Min Mean Min
Baseline 18.8% 0% 41.7% 0%
44
Experiment Results
Using only contaminant removal technique
Opening windows Opening doors
Accuracy Mean Min Mean Min
Baseline 18.8% 0% 41.7% 0%
Contaminant Rmv 69.0% 0% 70.0% 0%
45
Experiment Results
AutoFi (using both contaminant removal and autoencoder)
Opening windows Opening doors
Accuracy Mean Min Mean Min
Baseline 18.8% 0% 41.7% 0%
Contaminant Rmv 69.0% 0% 70.0% 0%
AutoFi 84.9% 47.6% 90.2% 71.3% 46
Brief Summary
• Problem for robust localization
– Small variations in the environment may significantly contaminate the
fingerprints
• Solution - AutoFi
– Reuse the fingerprints and the trained ML model with a contaminant
removal technique
automatically maps the contaminated profiles back to the fingerprint
domain
– Utilize an autoencoder to further denoise the purified profiles
47
More Recent Advances
- Toward Robustness in Real World
More Robust Motion Detection
AutoFi used CSI variance as an indicator of human motion
• CSI variance is not a robust indicator of human motion
• In some empty rooms, CSI variance caused by noise can be even higher than that caused by human motion
• Simply applying white noise filtering does not work
– Because this also removes the variance caused by human motion
Our solution MoFi [to appear in RTSS@Work 2019]
49
More Robust Motion Detection
• Robustness Study on Motion Detection
– Different users
– Different rooms
– Different AP/device placements
– Environment changes, e.g., moved furniture, open/close doors/windows
– Interference from other 2.4GHz devices, e.g., microwave ovens and Bluetooth beacons
50
Addressing User Diversity
Different body shapes yield different CSI profiles
• A system trained on one user may not work for a new user
• Labelling CSI profiles for a new user is mostly impractical
– Requiring user involvement
– Time consuming
• Work under submission and review
51
Wi-Fi Sensing
A new IEEE TIG (Topic Interest Group) on Wi-Fi sensing:
52
Wi-Fi based Motion Detection
Photo credit: Linksys: https://www.linksys.com/us/linksys-aware/ 53
Conclusions & Future Directions
Conclusions and Future Directions
• Wi-Fi based device-free localization is very promising
• Robust ML solutions have been developed for real-life scenarios
– Robust to environment changes: AutoFi
– Robust to random noises: MoFi
– Robust to user diversity: Under submission
• Future: Many more to explore!
– Multi-user localization
– Fast bootstrap/adaptation
– …
55
BACKUP SLIDES
57
Physics Behind This
Signals affected by human body
58
Physics Behind This
Different body locations introduce different effects
59