DLoc:Deep Learning based Wireless Localization for Indoor

Deep Learning based Wireless Localization for Indoor NavigationRoshan Ayyalasomayajula, Aditya Arun, Chenfeng Wu, Sanatan

Sharma, Abhishek Sethi, Deepak Vasisht and Dinesh Bharadia

1

Mobicom 2020

https://wcsng.ucsd.edu/dloc/

https://wcsng.ucsd.edu/wild/

Outdoor Localization

2


2

Navigation

https://www.kvhmobileworld.kvh.com/why-gps-spoofing-is-no-threat-autonomous-

navigation/

Autonomous Driving

https://play.google.com/

Delivery Bots

https://medium.com/@miccowang/delivery-robots-as-last-mile-solution-

aebebe557ad4

Autonomous Drones

https://www.kvhmobileworld.kvh.com/why-gps-spoofing-is-no-threat-autonomous-navigation/


https://medium.com/@miccowang/delivery-robots-as-last-mile-solution-aebebe557ad4


2

Navigation


navigation/

Autonomous Driving


Delivery Bots


aebebe557ad4

Autonomous Drones





2

https://www.nasa.gov/sites/default/files/gps_constellation_0.jp

g

Navigation


navigation/

Autonomous Driving


Delivery Bots


aebebe557ad4

Autonomous Drones

https://www.nasa.gov/sites/default/files/gps_constellation_0.jpg




Indoor Localization

3

Indoor Localization

3

https://www.clipartkey.com/upic/4785/


Indoor Localization

3

https://www.clipartkey.com/upic/4785/ https://medium.com/@brian.moran_91776/top-10-reasons-how-indoor-navigation-helps-shopping-malls-

be6376bf3d2


https://medium.com/@brian.moran_91776/top-10-reasons-how-indoor-navigation-helps-shopping-malls-be6376bf3d2

Indoor Localization

3

https://www.clipartkey.com/upic/4785/ https://medium.com/@brian.moran_91776/top-10-reasons-how-indoor-navigation-helps-shopping-malls-

be6376bf3d2

https://jonnegroni.com/2015/04/15/the-humans-of-wall-e-were-probably-better-off-without-him/


https://medium.com/@brian.moran_91776/top-10-reasons-how-indoor-navigation-helps-shopping-malls-be6376bf3d2

https://jonnegroni.com/2015/04/15/the-humans-of-wall-e-were-probably-better-off-without-him/

Lack of Context

4

Lack of Context

4

19, 13

X-axis (m)

Y-a

xis

(m

)

Lack of Context

4

19, 13

X-axis (m)

Y-a

xis

(m

)

Lack of Context

4

19, 13

X-axis (m)

Y-a

xis

(m

)

Lack of Context

4

19, 13

X-axis (m)

Y-a

xis

(m

)

WiFi based localization

5


5

MonoLocoMobiSys'18

Chronos NSDI'16

ToneTrackMobicom'15

SpotFiSigcomm'15

ArrayTrackNSDI'13


5

MonoLocoMobiSys'18

Chronos NSDI'16

ToneTrackMobicom'15

SpotFiSigcomm'15

ArrayTrackNSDI'13

Median: few decimeters


5

MonoLocoMobiSys'18

Chronos NSDI'16

ToneTrackMobicom'15

SpotFiSigcomm'15

ArrayTrackNSDI'13

Median: few decimeters >10% of cases: few meters

Deep Learning based Wireless Localization for Indoor

NavigationDLoc and MapFind

6

Deep Learning based Wireless Localization

7


Localization: Novel learning based approach to solve for the environment dependent localization.

7



Context: Bot that collects both Visual and WiFi data.

7

Table

TablesDesk




Dataset: Deployed it in 8 different in a Simple and Complex Environment

7

Table

TablesDesk




Dataset: Deployed it in 8 different in a Simple and Complex Environment

Results: Shown a 85% improvement compared to state of the art at 90th

percentile.

7

Table

TablesDesk

Challenge: Multipath, Non-Line of Sight

8


8

r

A

P

Smartphon

e

θ


8

r

A

P

Smartphon

e

Reflecto

r

θ


8

A

P

Smartphon

e

Reflecto

r

Obstacl

e


8

A

P

Smartphon

e

Reflecto

r

Obstacl

e

Need Knowledge of Environment

Requirements to design the neural network

9


9

Input Representation


9


Output/Target Representation


9


NetworkOutput/Target Representation


9



Objective/LossFunction


9



Objective/LossFunctionGradient

Input Representation: Raw CSI data

10


10

Maximillian Arnold et. al., SCC 2019

Michal Nowicki et. al., ICA, 2017

Xuyu Wang, et al., IEEE Access 5, 2017

Xialong Zheng, et al., IEEE/ACM Transactions on Networking, 2017


10





Complex Channel Values and AWG noise


10

Can we represent them as images?





Complex Channel Values and AWG noise

Input Representation: AoA-ToF images

11


11


11

Does not have context of Space and AP locations

Input Representation: XY images

AoA-ToF (Polar) to XY (cartesian)

12

Angle

of A

rriv

al,

𝝷°

Time of

Flight(m)



12

Angle

of A

rriv

al,

𝝷°

X axis

(m)

Time of

Flight(m)

Y a

xis

(m)

Polar to Cartesian



12

Angle

of A

rriv

al,

𝝷°

X axis

(m)

Time of

Flight(m)

Y a

xis

(m)

Polar to Cartesian

Context embedded Input Representation

Location Decoder Output targets

13


13


13


13

Image-to-Image translation problem

Network Architecture

14


14

1. Conv2d

[7,7,1,3]

2. Instance norm

3. ReLU

1. Conv2d

[3,3,2,1]

2. Instance norm

3. ReLU

Resnet Block* 1. ConvTranspose2d

[3,3,2,1]

2. Instance norm

3. ReLU

1. Conv2d

[7,7,1,3]

2. Instance norm

3. Sigmoid

1. Conv2d

[7,7,1,3]

2. Instance norm

3. Tanh

6 Resnet Blocks3 Resnet

Blocks

Location

DecoderEncoder

4 64

128256 256

256

12

8 64 1

Input Images

Output Location


14

1. Conv2d

[7,7,1,3]

2. Instance norm

3. ReLU

1. Conv2d

[3,3,2,1]

2. Instance norm

3. ReLU

Resnet Block* 1. ConvTranspose2d

[3,3,2,1]

2. Instance norm

3. ReLU

1. Conv2d

[7,7,1,3]

2. Instance norm

3. Sigmoid

1. Conv2d

[7,7,1,3]

2. Instance norm

3. Tanh

Llocation


Blocks

Location

DecoderEncoder

4 64

128256 256

256

12

8 64 1

Input Images

Output Location

Location Loss

Closeness in MSE sense

𝐿𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛 = 𝐿2[𝐷𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛𝐸 𝐻 − 𝑇]

15

Location Loss



Penalize multiple peaks

15

Location Loss



Penalize multiple peaks

𝐿𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛 = 𝐿2 𝐷𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛𝐸 𝐻 − 𝑇 + λ 𝐿1[𝐷𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛𝐸 𝐻 ]

15

High 90th percentile errors: Asynchronous Clocks

16


16

r

AP

Smartphon

e


16

r

Δr

AP

Smartphone

ToF offset

17

ToF offset compensation

18

DLoc: Network Architecture

19


Blocks

Location DecoderEncoder

4 64

128256 256

256

12

8 64 1

Input Images

Output Location

Offset Corrected Images


19


Blocks


4 64

128256 256

256

12

8 64 1

Input Images

Output Location

6 Resnet Blocks

Consistency Decoder (𝐷𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦)

256

12

8 64 4


Insight: Single source

20


21


Blocks


4 64

128256 256

256

12

8 64 1

Input Images

Output Location

𝐿𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛+

6 Resnet Blocks


256

12

8 64 4



21


Blocks


4 64

128256 256

256

12

8 64 1

Input Images

Output Location

𝐿𝑙𝑜𝑐𝑎𝑡𝑖𝑜𝑛

𝐿𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦

+

6 Resnet Blocks


256

12

8 64 4


Offset Compensation Loss

Defines consistency across images

𝐿𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 =1

𝑁𝐴𝑃

𝑖=1

𝑁𝐴𝑃

𝐿2[𝐷𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦(𝐸(𝐻)) − 𝑇𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦]𝑖

22

Offset Compensation Loss

Defines consistency across images

𝐿𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 =1

𝑁𝐴𝑃

𝑖=1

𝑁𝐴𝑃

𝐿2[𝐷𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦(𝐸(𝐻)) − 𝑇𝑐𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦]𝑖

22

Context: MapFind

23

Context: MapFind

23

WiFi Client

LiDAR

RGB-D

Server

Kobuki

Context: MapFind

23

Table

TablesDesk

Context: MapFind

23

How much data is needed?

Table

TablesDesk

Path Planning

24

Path Planning

Maximize coverage

Minimize traversal length

24

Path Planning

Maximize coverage

Minimize traversal length

24

Context Enabled Accurate Indoor Localization

Results

25

Setup

26

Setup

26

Complex High-multipath and

NLOS environment (1500 sq.

ft.)

AP

1

AP

3

AP

2

Plasma

Screens

Plasma

Screens

AP4 -

NLOS

Setup

26



ft.)

AP

1

AP

3

AP

2

Blocka

ge

Plasma

Screens

Plasma

Screens

AP4 -

NLOS

Setup

26



ft.)

Simple LOS based

environment (500 sq. ft.)

AP

3 AP

2

AP

1

Reflect

or

AP

1

AP

3

AP

2

Blocka

ge

Plasma

Screens

Plasma

Screens

AP4 -

NLOS

DLoc Result – Complex Environment

27

0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 4 4.4 4.8Localization Error (m)

0.10.20.30.40.50.60.70.80.9

1

CD

F


27


0.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model


27


0.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model


27


0.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model


27


0.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model

DLoc Result – Simple Environment

28

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2Localization Error (m)

00.10.20.30.40.50.60.70.80.9

1

CD

F


28


00.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model


28


00.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model


28


00.10.20.30.40.50.60.70.80.9

1

CD

F

DLoc

SpotFi

Baseline DL Model

Accurate Indoor Localization

Generalization across multiple setups

29


29

Setup-1 Setup-2

Setup-3 Setup-4


29

Setup-1 Setup-2

Setup-3 Setup-4

Trained on

Setup

Tested on

Setup

Median Error (cm)

90th Percentile Error (cm)

DLoc SpotFi DLoc SpotFi

1,3,4 2

1,2,4 3

1,2,3 4


29

Setup-1 Setup-2

Setup-3 Setup-4

Trained on

Setup

Tested on

Setup

Median Error (cm)



1,3,4 2 198 420

1,2,4 3 154 380

1,2,3 4 161 455


29

Setup-1 Setup-2

Setup-3 Setup-4

Trained on

Setup

Tested on

Setup

Median Error (cm)



1,3,4 2 71 198 171 420

1,2,4 3 82 154 252 380

1,2,3 4 105 161 277 455

Open-Sourced Dataset

30


30

• Enabling Baseline comparison for all algorithms


30

• Enabling Baseline comparison for all algorithms• Pushing Indoor Localization to realization


30

• Enabling Baseline comparison for all algorithms• Pushing Indoor Localization to realization• Pushing towards a competition similar to ImageNet program


Labelled WiFi CSI data (WILD-v1)

8 different setups

4 different days

108K datapoints

2 different environments

30





Labelled WiFi CSI data (WILD-v1)

8 different setups

4 different days

108K datapoints

2 different environments

30


WILD-v2 Coming Soon

• 20 different setups

• 10 different days

• 1 million datapoints

• 8 different environments

• 20 different AP locations



Conclusion and Future Work

• Novel Deep Learning based algorithm with 85% incremental performance

compared to state-of-the-art.

• MapFind we have collected over 108k datapoints (and expanding) that is open-

sourced.

• Enabling large scale and autonomous indoor navigation

31

https://wcsng.ucsd.edu/dloc/


Documents

DLoc:Deep Learning based Wireless Localization for Indoor