View
220
Download
5
Category
Preview:
Citation preview
March 18, 2010 Deokwoo Jung
Towards Cooperative Localization of Wearable Sensors using
Accelerometers and Cameras
Deokwoo Jung(with Thiago Teixeira and Andreas Savvides)
deokwoo.jung@yale.eduEmbedded Networks & Applications Lab
Yale University
March 18, 2010 Deokwoo Jung
Indoor Localization• Indoor localization is an essential technology for many applications
– Security Application, Assisted Living, Life logging System etc
• Indoor localization comparison
Cricket [Priyantha.et.al ,00,04]
RADAR[Bahl.et.al,00]
Surroundsense[Azizyan.et.al,09]
Our System
PrecisionPhysical Location
(cm)Physical Location
(<5 meter)Logical Location
(>5 meters)
Physical Location
(cm)
Mobile device
Customized Sensor NodeWLAN Card -
LaptopMobile Phone
Mobile Phone/ Wearable Sensor
Infra- structure
Ultrasound Beacon Nodes on Ceiling
WLAN APs + RF fingerprint data
base
GSM Network + Ambient signal
databaseNetworked cameras
Sensing modality
Ultrasound RF signal Ambient signals-
Light, sound, colorHuman walking motions
March 18, 2010 Deokwoo Jung
Cooperative Localization Approach• Our Approach
– Localizing mobile phones by combining their built-in accelerometer (human motion) and infrastructure camera (human centroids)
• Why Human Centroids and Human Motion ?– They are Complementary to each other
Wearable inertial sensors (Human Motion) Camera (Human Centroid)
ID tracking Accurate - node address Difficult – feature extraction
Location Positioning Difficult –walking orientation and distance estimation Accurate –background subtraction
March 18, 2010 Deokwoo Jung
Sensing Modeling and Approach
• Human Walking Model
– ∫ ∫Accel. ≠ Walking distance
– The law of movement of human body by complex kinetics
– Inverted pendulum model of human gait.
– The body center of mass (BCOM) oscillates in the z direction
– as the person moves forward (y direction).
March 18, 2010 Deokwoo Jung
Statistical Analysis of Sensor Data
• Intuition : BCOM follows a sinusoidal pattern– Velocity of Body α Standard Deviation of Vertical Acceleration
• A correlation coefficient for the similarity measure between accelerometer and camera data
• Experiment
-0.5 0 0.5 1 1.5 20
0.1
0.2
0.3
0.4
az ( g)
Distrib
ution
V=0.61 m/sec
-0.5 0 0.5 1 1.5 20
0.1
0.2
0.3
0.4V=0.76 m/sec
-0.5 0 0.5 1 1.5 20
0.1
0.2
0.3
0.4V=0.91 m/sec
-1 0 1 20
0.1
0.2
0.3
0.4V=1.67 m/sec
= 0.0914 = 0.1294
= 0.1943 = 0.4537
0 0.5 1 1.5 20
0.1
0.2
0.3
0.4
0.5
Walking Velocity, m/sec
Stan
dard
devia
tion o
f az
az
= 0.36*v - 0.13
Sample data linear regression
March 18, 2010 Deokwoo Jung
Tracking Algorithm
• A camera extracts only centroid information – Privacy Preserving and Cheap
• A simple tracking algorithm computes a speed of anonymous centroids– associates human centroids in consecutive frames based on their
distances.
• Problem: Many possible ambiguous associations
March 18, 2010 Deokwoo Jung
Path Disambiguation Problem
• Path Ambiguity Problem in Human Centroid Tracking– A tracker associates one object with more than two objects
in two consecutive image frames when two or more objects come close to each other.
March 18, 2010 Deokwoo Jung
Disambiguation Algorithm
• Path Disambiguation as Non-linear Optimization Problem– Find a set of association hypotheses to maximize a matching rate,– The number of correct ID matchings between accelerometers
and centroids
• Develop a search algorithm in a tree structure – A leaf node: a hypothesis of path segmentations– Three stage pruning algorithm
• Sub-tree evaluation, • Classification and Pruning, • Reconstruction
N
iK
Nji T
TK
jiIN
ET 1
1,,1,,
,,|,maxarg11
max1
March 18, 2010 Deokwoo Jung
Clustering and Pruning in Hypothesis Tree
• Hypothesis Quality Metric : – how credible a given path hypothesis is compared to others?– Correlation Coefficient Distance metric
• D(ρ|H) = |E(ρ, e0|H) − E(ρ, e1|H)|
Wrong Hypothesis Correct Hypothesis
Accelerometers
6.04.03.04.0
5.05.04.03.0
5.06.03.02.0
4.04.02.05.0
2H
Cen
troi
d t
race
s 2
8.02.01.01.0
1.09.02.01.0
2.01.03.07.0
1.01.08.01.0
1H
Cen
troi
d t
race
s 1
Accelerometers
-0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.002
0.004
0.006
0.008
0.01
0.012
Correlation Coefficient, x
Dis
trib
utio
n
P( =x| Correct Matching )
P( =x | Incorrect Matching )
-0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.01
0.02
0.03
0.04
0.05
0.06
Correlation Coefficeint , x
Dis
trib
utio
n
Pr(=x| Incorrect Matching)
Pr(=x| Correct Matching)
D(ρ|H1) D(ρ|H2)
March 18, 2010 Deokwoo Jung
Clustering and Pruning in Hypothesis Tree
• Leaf Clustering, Pruning, and Path Reconstruction– Clusters the leaf nodes into groups and prunes the subset of
groups with lower metric values.
– When only one leaf is left, reconstructs the matching sequence
Tree Pruning Algorithm
8.02.05.01.0
1.09.02.01.0
2.01.03.07.0
1.04.08.01.0
1H
Cen
troi
d t
race
s 1
Accelerometers
8.02.05.01.0
1.09.02.01.0
2.01.03.07.0
1.04.08.01.0
1H
Cen
troi
d t
race
s 1
AccelerometersAccelerometersAccelerometers
8.05.01.04.0
7.07.07.03.0
6.08.06.02.0
5.07.02.08.0
2H
Cen
troi
d t
race
s 2
Accelerometers
8.05.01.04.0
7.07.07.03.0
6.08.06.02.0
5.07.02.08.0
2H
Cen
troi
d t
race
s 2
0 10 20 30 40 50 600
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Time (sec)
Cor
rela
tion
Coe
ffic
ient
Dis
tanc
e, D
()
Perfect centroid trace estimation, HGround Truth
No path ambiguity Path ambiguities are accumulating
March 18, 2010 Deokwoo Jung
Performance Evaluation via Experiment & Simulation
• Experiment Setup– A ceiling mounted camera (12ft) with
a Intel iMote2 node • Computes the centroid position of a p
erson, 15 times per second.
– A wearable sensor node with an Analog Devices ADXL330 accelerometer on the person’s waist
• Collecting body acceleration data with 15Hz sampling rate.
• Transmitting its measurements to a computer (fusion center) via a Zigbee wireless link.
– People walk for 1 minute in a 5.4 m2 space.
March 18, 2010 Deokwoo Jung
Experiment Dataset
• Walking trajectory of 12 people collected from camera
0 2 40
1
2
3
Trace1
X ( meter)
Y (
met
er)
0 2 40
1
2
3
Trace2
0 2 40
1
2
3
Trace3
0 2 40
1
2
3
Trace4
0 2 40
1
2
3
Trace5
0 2 40
1
2
3
Trace6
0 2 40
1
2
3
Trace7
0 2 40
1
2
3
Trace8
0 2 40
1
2
3
Trace9
0 2 40
1
2
3
Trace10
0 2 40
1
2
3
Trace11
0 2 40
1
2
3
Trace12
March 18, 2010 Deokwoo Jung
Similarity Metric Performance • 100 % matching rate without path ambiguity
Standard Deviation of z-acceleration and velocity of BCOM over time for tracesBar Graph of Correlation Coefficient Matrix
March 18, 2010 Deokwoo Jung
Disambiguation Algorithm Performance
• The performance depends on the level of crowd in camera field of view. – Evaluate the performance using crowd density metric, the number of
pedestrians per area, m2, [Abishai.et.al, Pedestrian flow and level of service]
• Crowd Density Scenario
Scenario A: Normal flow B: Restricted Flow C: Dense Flow D: Very Dense Flow
Crowd Density
office building in business
hour
crowded shopping mall in weekend
Crowded weekend party
Subway station in Manhattan during the
rush hour
People / m2
<0.5 0.5~0.8 0.81~1.26 1.27~2
• If the crowd density > 2, the pedestrian flow is jammed, i.e. practically people’s movement appears to be static
• Our system is mainly targeting for the scenario A (free flow), i.e. people can walk around without much interaction.
March 18, 2010 Deokwoo Jung
Performance over complexity of scenario
• The number of tracking errors grows with polynomial order with crowd density (left)
• The matching performance of disambiguation algorithm for different crowd densities (right)
• The performance gap is widening as crowd density increases. – The performance becomes twice in the scenario D.
0.18 (1) 0.37 (2) 0.56 (3) 0.75 (4) 0.94 (5) 1.13 (6) 1.32 (7) 1.51 (8) 1.70 (9)
0
5
10
15
20
25
30
35
40
Crowd Density, people / m 2 (# people)
Ave
rag
e nu
mbe
r o
f cul
uma
tive
tra
ckin
g e
rror A. VERY LOW DENSITY
NORMAL FLOWB. LOW DENSITY RESTRICTIED FLOW
C. MODERATE DENSITY DENSE FLOW
D. HIGH DENSITY VERY DENSE FLOW
0.18 (1) 0.37 (2) 0.56 (3) 0.75 (4) 0.94 (5) 1.13 (6) 1.32 (7) 1.54 (8) 1.70 (9)
30
40
50
60
70
80
90
100
Crowd Density, people / m 2 (# people)
Ave
rag
e m
atc
hing
ra
te, %
Tracker OnlyDisambiguation algorithm
A B DC
March 18, 2010 Deokwoo Jung
Performance over disambiguation stages
• Matching rate improvement by disambiguation algorithm
0 10 20 30 40 50 600
20
40
60
80
100
time (sec)
mat
chin
g ra
te,
%
0 10 20 30 40 50 600
20
40
60
80
100
time (sec)
mat
chin
g ra
te,
%
0 10 20 30 40 50 600
20
40
60
80
100
time (sec)
mat
chin
g ra
te,
%
March 18, 2010 Deokwoo Jung
Localization System Demonstration
• Controlled experiments with 10 people walking scenario.
• The performance of disambiguation algorithm (right) is compared to the tracker-only localization (left).
March 18, 2010 Deokwoo Jung
Conclusion
• We presented a hybrid localization system using accelerometers and cameras.
• The proposed disambiguation algorithm operates reliably, degrading gracefully even for crowded scenarios
• The constraint of accelerometer position (waist) can be relaxed using additional inertial measurement sensors.
• Future work is to have a complete system implementation running on a mobile phone + More sensors
March 18, 2010 Deokwoo Jung
QUESTION ?
Thanks for your
interest!For more
information, please visit
http://pantheon.yale.edu/~dj92/
Recommended