Indoor Location Sensing: Principals, Potentials, and Practice

J IE L IU

HARBIN INSTITUTE OF TECHNOLOGY ( SHENZHEN)

j ie l iu@hit .edu.cn

"The light of navigation", Dutch

sailing handbook, 1608, showing

compass, hourglass, sea astrolabe,

terrestrial and celestial globes,

divider, Jacob's staff and astrolabe.

‘80 ‘90 ‘00 ‘10June 23, 1977

Next Billion

Devices

Modern Location Sensing Proliferation

Paradigms of Location Sensing

• Geometric-based: estimate locations directly from geometric measurements

• Experience-based: estimate locations from features collected from the past

• Tracking-based: estimate locations from past locations and motion

1. Geometric Based: “It must be here.”

• Distance-Based: Trilateration

d1A B

C

d2

d3

A B

C

α2

α3

α1

• Angle-Based: Triangulation

“Determine mobile device location using distance and/or orientation measurements to multiple anchor nodes with known location”

Measurements

• Questions:◦ What signal to use for measurements?

◦ How to measure distances / orientations?

◦ How to compute location from these measurements

◦ How to determine (configure) anchor node locations?

Anchor node

Distance measurement

Orientation measurement

Distance Measurement Primitives

• Time Of Flight (TOF)

• Time Difference Of Flight (TDOF)

• Time Difference Of Arrival (TDOA)

Tim

e

1t

2t

Tx Rx

Tim

e

1t2t

Tx Rx

3t

Tim

e

Tx(1)

1t

2t

Tx(2) Rx

3t

)( 12 ttvd −=v

1v

2v

)(

)(

21

2123

vv

vvttd

−

−=

)( 23 ttvd −=

Orientation from Difference in Time of Arrival

• When d >> L

◦ Can be calculated from Time Difference Of Arrival (TDOA) of the signals

Tx

dd2

d1

Lq

L

ddCos 12 −q

Rx(1)

Rx(2)

Distance Measurement Techniques • Audio based: audible and ultrasound

◦ Advantages

◦ Slow speed : easy to measure propagation time

◦ Generation and detection using speakers and mics

◦ Disadvantages

◦ Strict line of sight – blocked by most physical materials

◦ Lack of widely deployed infrastructure

◦ Interfere with hearing (audible: humans, ultrasound : pets)

• RF based: all kinds of frequencies ◦ Advantages

◦ Easy to deploy infrastructure

◦ Better penetration than audio

◦ Disadvantages

◦ High propagation speed: difficult to measure time-of-flight

• Active Light based: laser, IR◦ Advantages

◦ Can be very accurate using ToF

◦ Disadvantages

◦ Can be expensive

◦ Needs direct line of sight

• Camera based◦ Advantages

◦ Can measure distances and angles at once

◦ Price is getting cheap

◦ Range is limited due to resolutions

◦ Disadvantages

◦ Need to calibrate

◦ Higher computational cost than 1d approaches

◦ Needs direct line of sight

2. Experience-based: “I have been (near) here!”

• Fundamentally, it is a signature recall approach.

• Build features mapping 𝐹 𝑉 = 𝐿 , that min{𝐷 𝐿, 𝐹 𝑉 }

• Localization: For given measurement 𝑉, find 𝐿 = F( 𝑉)

Indoor Location Dependent Signatures/Features

Bluetooth/Zigbee BeaconsMagnetic Beacons

RFIDModulated Light

Ground VibrationSparse Set of Beacons

Cell NetworkWiFi

SoundFM Radio

Photos

Earth Magnetic FieldTurns and Stairs

Video

Instant Trajectory

(Nearly) Ambient

Deployed Infrastructure

3. Tracking-based: “How I have moved”

• Start with a known location (𝑡1)

• Measure motion over time(𝑡1~𝑡2)

• Estimate new location (𝑡2)

• Repeat𝑡1

𝑡2

4. Sensor Fusion: Combining Two or More Approaches

• Two or more types of complementary signals (modalities): ◦ To measure distances: e.g. Cricket (RF + Ultrasound)◦ To compensate signatures: e.g. FM + WiFi

• Absolute location with motion tracking: E.g.◦ WiFi + step counting ◦ Markers + visual odometry

• Location/movement + side knowledge : E.g. ◦ Loop closure◦ Map constraints

Cricket Indoor Location System

• Cricket: ◦ Anchor nodes → beacons

◦ Mobile node → Listener

• Computes distance by TDOF of RF and US

• Multipath mitigation: first US pulse to measure distance◦ US signals die down after 50mS

RF data

(space name)

Beacon

Listener

Ultrasound

(pulse)RF US

US Reflections

50 ms ( T_US )t

Nissanka Priyantha, Anit Chakraborty and Hari Balakrishnan, "The Cricket Location Support System”, MobiCom 2000

FM+WiFi Based Indoor Localization

FM towers near Seattle

Complementary Roles of WiFi and FM

FM ALL WiFi RSSI FM ALL + WiFi RSSI

Fusing Motion with Maps

• WiFi Signatures• IMU based direction & step counting • Sequential Bayesian updates• Particle filters• Map constraints

Potentials:Indoor location examined

Lots of Publications, Very Little Comparison

System A3m Location Accuracy

System B4m Location Accuracy

ORWhich on is better?

Microsoft Indoor Location Competitions

• Evaluate and compare technologies from academia and industry in the same, unfamiliar space.

• Bring teams working in this area together in a more effective way.

2014: Berlin 2015: Seattle 2016: Vienna 2017: Pittsburgh 2018: Porto

Microsoft Indoor Location Competitions2014

◦ 36 teams signed up, 22 competed

2015

• 48 teams signed up, 23 competed

Room A Room B

8

1

2

34

5

6

7

9

10

11

12

13

14

15

16

17

18

1920

300 m2

Origin Point(0,0)

Y A

xis

X Axis 2000 m2

Basic RulesINFRASTRUCTURE FREE

• Pre-deployed 10 WiFi AP for everyone.

INFRASTRUCTURE ASSISTED

• Allow deployment of up to 10 devices

• 8 hours of setup up time

• Unknown evaluation points

• Individually evaluated to prevent interference.

Craziness

Craziness

More Craziness

2014 Localization Accuracy

0.721.56

1.96 2.03 2.04 2.09 2.222.35 2.58 2.72 2.81

3.19 3.473.71 3.83 3.87

3.96 4.044.86

5.23

8.91

10.22

0

1

2

3

4

5

6

7

8

9

10

11

Loca

tio

n E

rro

r (m

eter

s)

Average Error RMSE Location Error Standard Deviation

2.4Ghz Phase Offset

WiFi + IMUBayesian Inf.

WiFi + IMUNeural Net.

2.4GHz ToFWiFi + LEDsUltrasonic ToF

WiFi + BLE Beacons

Modulated Magnetic Field

Modulated Magnetic Field

Error when the center of the evaluation area is always reported as the true location

2015 Results

0.2 0.31 0.39 0.6 0.73 0.739 0.78 1.37 1.63 1.92.6

3.5 3.65 3.93 4.22 4.31 4.67 4.925.6

7.398.24

11.24

22.99

0

5

10

15

20

25

Loca

lizat

ion

Err

or

(m)

Team

infrastructure-free industry

Lid

ar

ult

raso

nic

UW

B

LED

+Cam

era

Mar

kers

+IM

U

iBea

con

+WiF

i

Atm

el T

oF

Ph

on

e +

pro

filin

g

IMU

2016: 465m2 Evaluation Area, Two Levels

UWB

Sound/Ultrasound

24GHZRadar

LIDAR

Zigbee

IMU Only

PDR + beacons

TDoA SDR

WiFiToF

(init): the system was explicitly initialized to a known ground truth location before evaluation

2017: 600m2 Evaluation Area - Two Floors

2.231 2.415

4.042

5.638

7.2178.07 8.494

0

2

4

6

8

10

McMaster Seoul Nat.Univ.

Iwate univ. NationalTaiwan Univ.

Ariel Univ. Rutgers Univ. FraunhoferPortugal

FraunhoferMulti-sensor

Glodon

Erro

r in

Met

ers

Team

0.018 0.033 0.1680.604

2.579 2.961

6.064 6.281 6.559 6.819

0

2

4

6

8

JRC Kaarta Quantitec ETH KAUST Kiwii Astrobotic McMaster Arin Tech. TrakRay M2Robots

Erro

r in

Met

ers

Team

2D

3D

PDR

Fingerprinting

DoD Beacons

Lidar

Ultrasound

Sound

UWB

2018: Two LevelsLiDAR System under Test

2D Results

2.3 2.43.2

4.3

5.3

9.4

12.5

0

2

4

6

8

10

12

14

16

18

20

*Naviguy Ariel Univ. (Step) *Ali et al. Ariel Univ. (GoIn) Rea et al. Naviguy (Fusion) *Fineway

Mea

n L

oca

lizat

ion

Err

or

(m)

Team

PDR

PDR + Fingerprinting

WiFi ToF

Camera+PDR

Camera+PDR+Finger.

Teams that start with * require initialization.Naviguy and Ariel Univ. were considered a tie! Ariel University is the winner as Naviguy required initialization while Ariel University did not.

Naviguy Ariel Univ. (Step)

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

Teams that start with * require initialization. SND Smart Ltd and Yodel Labs are considered a tie!

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

3D Results

0.27 0.4 0.470.7 0.71 0.73

0.940.99

1.161.21

1.49 1.7

3.79

6.94

16.64

0

5

10

15

20

25 Sound

AtmelPhaseSound+

ARKitUWB+ARKit

UWB

Camera+LED + PDR

2020: Indoor Location Competition 2.0• Completely Virtual, Open

Competition

• 30,000 traces from over 200 buildings

• $10,000 Award

• 1,170 teams participated!

• Best results: Average Location Error < 1.5m

Practice: What’s Hard About Indoor Localization

• Inaccurate measurements◦ Receiving signal phases and amplitudes

(higher frequencies and higher bandwidths are better)◦ Time synchronization ◦ Multi-paths fading and reflections◦ Missing line of sight

• Environments change over time ◦ Setup re-arranged◦ Signal source (beacons) replaced◦ Signal inference◦ Human movements

• Device variations

• Infrastructure cost

Start ?

Detecting the start of Rx is hard due to “gradual signal build up”

There is NO Single Best Solution

• There is a huge variety of signals and techniques for indoor location sensing.

• A pushpin is not the end.◦ Indoor maps

◦ Place semantics

◦ Adoption paths

Infrastructurecomplexity

Device cost

Accuracyrequirements

Find the right one, not the best one

IoT

Close the Loop with Value Proposition

Acknowledgements

BodhiPriyantha

DiWang

DimitriosLymberopoulos

MichelGoraczko

GeraldDeJean

JackyShen

YuanchaoShu

QiangXu