23
Multi-Camera Multi-Human Tracking System Copyright © Yu-Sheng Chen [Yosen]

Multi-Camera Multi-Human Tracking System (oral presentation)

Embed Size (px)

Citation preview

Page 1: Multi-Camera Multi-Human Tracking System (oral presentation)

Multi-Camera Multi-Human

Tracking System

Copyright © Yu-Sheng Chen [Yosen]

Page 2: Multi-Camera Multi-Human Tracking System (oral presentation)

Outline

Top view of the entire tracking system

What local trackers do

Identify people depth order

Handle body occlusion

What central server does

Compare observations across cameras

Classify to different people

Recognize people when re-entering

Yosen Chen, Copyright Reserved 2

Page 3: Multi-Camera Multi-Human Tracking System (oral presentation)

Images

Local camera

#A tracker

,

1

A

tz 2

A

tz

2

B

tz1

B

tzLocal camera

#B trackerImages

Central Server

multi-camera correspondence

Person#1 Person#2

Identify how many people

in the observed space

Shared data on Inter-process platform

, 2

A

tz 1

B

tz1

A

tz 2

B

tz

Listen/

Broadcast

Page 4: Multi-Camera Multi-Human Tracking System (oral presentation)

What local trackers do?

Multi-human

tracking &

detection

Central

Server

Operation flow of local trackers

Identify

people depth

order

If detect new,

broadcastGet people labels

& people heights.

Handle body

occlusionStart

Keep updating body info. to server databases

Extract

people body

info.

Local trackerCentral server

Page 5: Multi-Camera Multi-Human Tracking System (oral presentation)

What local trackers do: Identify depth order

A. By appearances

B. By standing locations

Image source

1:Front

2:Rear

Page 6: Multi-Camera Multi-Human Tracking System (oral presentation)

Depth order estimation by head shape completeness is a bad idea!

Yosen Chen, Copyright Reserved 6

Page 7: Multi-Camera Multi-Human Tracking System (oral presentation)

What local trackers do: Identify depth order

A. By appearances

B. By standing locations

Image source

1:Front

2:Rear

Page 8: Multi-Camera Multi-Human Tracking System (oral presentation)

Yosen Chen, Copyright Reserved 8

Tell the depth order by people’s

standing locations

How to know the standing locations?

To be discussed in the central server part!!

Bd Ground plane

Ad

Camera image

Cd

BdAd Cd depth order: A=1, B=2, C=3

A

BC

Page 9: Multi-Camera Multi-Human Tracking System (oral presentation)

Depth order estimation by 3D standing locations is accurate.

Yosen Chen, Copyright Reserved 9

Page 10: Multi-Camera Multi-Human Tracking System (oral presentation)

What local trackers do: Handle body occlusion Occluded parts can be estimated by

3D geometry with body modeling

Page 11: Multi-Camera Multi-Human Tracking System (oral presentation)

Occlusion Estimation by 3D Body

Modeling

Yosen Chen, Copyright Reserved 11

Ground plane

Camera image

B

AAB

Camera visible parts

Camera invisible parts

Page 12: Multi-Camera Multi-Human Tracking System (oral presentation)

Body Occlusion Handling

Yosen Chen, Copyright Reserved 12

Depth order

check

Body occlusion

handling

Target info

storage

Body appearance is available for

color extraction

Body appearance is not available

Page 13: Multi-Camera Multi-Human Tracking System (oral presentation)

What central server does?

Compare

observations

across cameras

Local

trackers

Operation flow of central server

Get observation

correspondence

between camerasIf detect new,

broadcast

Return people labels & people heights.Start

Keep updating body info. to server database

People

database

Compare with

database to recognize

/create people

Local trackerCentral server

Page 14: Multi-Camera Multi-Human Tracking System (oral presentation)

What central server does: Compare observations across

cameras Geometric-based color comparison

Page 15: Multi-Camera Multi-Human Tracking System (oral presentation)

2D position on image plane 3D line in real space

3D standing point in real space 2D feet position on image plane

observation r

Optic center Image plane of

camera j

Ground plane

Optic center

observation q

Image plane of

camerak

3D Geometric Correspondence across

Cameras

Calibrated camera system

Head Point

height

Estimated Standing Location

Yosen Chen, Copyright Reserved 15

Page 16: Multi-Camera Multi-Human Tracking System (oral presentation)

What central server does: Classify to different people Use decisions in higher confidence to

determine lower ones.

Page 17: Multi-Camera Multi-Human Tracking System (oral presentation)

Observation Correspondence

across Cameras

Object Conf = 3 Object Conf = 2

Matching body color (Block-based body color comparison)

Yosen Chen, Copyright Reserved 17

Different colors mean different labels

Page 18: Multi-Camera Multi-Human Tracking System (oral presentation)

Multi-People Correspondence

Yosen Chen, Copyright Reserved 18

1. Multi-people depth order √ 2. body occlusion handling √ 3. Multi-people correspondence √

Page 19: Multi-Camera Multi-Human Tracking System (oral presentation)

What central server does: Identify people if re-enter Construct people database once

they got tracked.

Page 20: Multi-Camera Multi-Human Tracking System (oral presentation)

Yosen Chen, Copyright Reserved 20

Check height

difference

How to recognize people when they re-enterAre they all from the same person?

Body Color

comparison

by block

Label history

consistency

No, if diff > threshold No, if color unmatched

Yes, they are from

the same person!!

Camera

observations

People model

in database

Page 21: Multi-Camera Multi-Human Tracking System (oral presentation)

Future Works…

Extend to more cameras

Challenge of

Communication load & algorithm complexity

Tracking/labeling stability

More challenges:

Outdoors Surveillance (human body detection)

Uncalibrated multi-camera system (Training)

Motive cameras (rotation 3D-2D mapping)

Yosen Chen, Copyright Reserved 21

Page 22: Multi-Camera Multi-Human Tracking System (oral presentation)

Thanks for Pf. Li-Chen Fu

Advanced Control Lab, NTU

Intelligent Robot Lab, NTU

Page 23: Multi-Camera Multi-Human Tracking System (oral presentation)

Copyright © Yu-Sheng Chen [Yosen]