Upload
dolan
View
60
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work. Transitive Re-identification. Technion - I.I.T. Haifa, Israel. This research was supported by the MAGNET program in the Israeli ministry of industry - PowerPoint PPT Presentation
Citation preview
Transitive Re-identification
Technion - I.I.T. Haifa, Israel
BMVC 2013
This research was supported by the MAGNET program in the Israeli ministry of industryand commerce, by the Israeli ministry of science and by the E. and J. Bishop research fund.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
recognizing individuals over different camera views(we focus on ReID based on appearance based cues)
โข upper image from: A. Bialkowski, S. Denman, P. Lucey, S. Sridharan, and C. B. Fookes. โA database for person re-identification in multi-camera surveillance networks.โ (DICTA 2012)
โข lower images: courtesy of Marco Cristani
Camera A, time t Camera B, time t+โt
ReIDentification (ReID)
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentification
โข require a training set
โข classification is with a classifier or a metric learned on the training set
โข do not depend on any training set
โข classification is based on pre-defined distance metric
Appearance-based ReIDentification methods:
perform better for โcamera-specificโ scenarios:ReID benefits from training with corresponding appearance-pairs captured by specific camerapairs, as the background, illumination, resolution and pose are camera dependent.
Learning based methods: Direct methods:
โข always camera-invariantโข camera-invariant or camera-specific, depending on the used training set
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentificationLearning based methods perform better for โcamera-specificโ scenarios:
1 2 3 4 5 6 7 8 9 10 11 1210
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on 3-7 dataSDALF for cameras 3-7
ICT โ learning based
SDALF - direct
โข ICT (Implicit Camera Transfer) by Avraham et al (ECCV Re-Id 2012)โข SDALF (Symmetry-Driven Accumulation of Local Features) by Farenzena et al
(CVPR 2010) and Bazzani et al (CVIU 2013)โข performance tested on 2 cameras from the SAIVT-SoftBio dataset
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
ReIDentificationLearning method: ICT Direct method: SDALF
same
not same
Camera A
Camera B
F
FC
Binary SVM with RBF kernel, usingDecision values
Image A
Image B
(a) localizing meaningful body parts
(a) (b) (c) (d)
* Image from: L. Bazzani, M. Cristani, and V. Murino. Symmetry-driven accumulation of local features for human characterization and re-identification.
(b) weighted HSV histogram(c) MSCR - maximally stable color regions(d) RHSP - recurrent highly structured patches=> matching SDALF descriptors
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Motivation
Given a site with cameras , we may ReID either by:
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
1. applying a direct method
2. applying a learning based method where distinct inter-camera training sets must
be collected and annotated for each camera-pair. This is impractical.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Motivation
We aim at reducing the number of required direct inter-camera training sets
from to by suggesting a transitive algorithm which uses inter-camera training sets
only for camera-pairs , and infers a ReID classifier for any other camera-pair in the
system.
directly trainable pairs non-directly trainable pairs
By recursively applying the transitive algorithm:
By applying the transitive algorithm:
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
๐ช๐ต+๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
When a new camera is added to a set of N previously installed cameras,
with the proposed transitive approach, collecting data for only one pair is
required.
Motivation
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithmโข We focus on a camera triplet case .
๐ช๐
๐ช๐
๐ช๐
AB
C
๐ช๐
๐ช๐
๐ช๐
๐ช๐
๐ช๐ต
๐ช๐
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm AB
C
โข Given two training sets associated with camera-pairs and ,
respectively, we would like to infer a classifier for the camera-pair, for
which is missing.
Training
A B A C
= ?
Test
= ?
= ?
B C
๐บ๐จ๐ฉ
๐บ๐ฉ๐ช
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive AlgorithmExpanding the learning based method โ the Naive solution:training the classifier with
1 2 3 4 5 6 7 8 9 10 11 1210
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on 3-7 dataSDALF for cameras 3-7ICT trained on 3-5 and 5-7 data
ICT
SDALF
Naive ICT
Our goal is to find a more effective use of the available training sets that will decrease this performance gap.
AB
C
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )=?
๐ (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )= โ๐ ๐จ๐ฉโ{๐ ,๐ }
โ๐๐ฉ๐ชโ{๐ ,๐ }
โซ๐๐ฉโ๐น๐
๐ท (๐ ๐จ๐ช ,๐ ๐จ๐ฉ=๐ ๐จ๐ฉ ,๐ ๐ฉ๐ช=๐ ๐ฉ๐ช , ๐๐ฉ|๐ ๐จ , ๐๐ช ยฟโ ๐๐ฉยฟ
๐ (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )= โ๐ ๐จ๐ฉโ{๐ ,๐ }
โ๐๐ฉ๐ชโ{๐ ,๐ } [ โซ
๐๐ฉโ๐น๐
๐ท (๐ ๐จ๐ช ,๐ ๐จ๐ฉ=๐ ๐จ๐ฉ ,๐ ๐ฉ๐ช=๐ ๐ฉ๐ช|๐ ๐จ ,๐๐ฉ ,๐ ๐ชยฟ ๐ ๐๐ฉ(๐๐ฉ)โ ๐๐ฉ]
The Transitive ReID algorithm (TRID) establishes a path between the non-directly trainable camera pair by marginalization over a โconnecting elementโ, the domain of possible appearances in camera .
AB
C
๐บ๐จ๐ฉ
๐บ๐ฉ๐ช
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )=?
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )=โซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐๐จ , ๐๐ฉ)๐ท (๐ ๐ฉ๐ช|๐ ๐ฉ , ๐๐ช ) ๐ ๐ฟ๐ฉ (๐ ๐ฉ)โ ๐๐ฉ
๐โโซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐ ๐จ , ๐๐ฉ )๐ท (๐ ๐ฉ๐ช|๐๐ฉ , ๐๐ช ) ๐ ๐ฟ ๐ฉ (๐๐ฉ )โ ๐ ๐ฉ
AB
C
.
.
.
.
The Transitive ReID algorithm (TRID) establishes a path between the non-directly trainable camera pair by marginalization over a โconnecting elementโ, the domain of possible appearances in camera .
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )=โซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐๐จ , ๐๐ฉ)๐ท (๐ ๐ฉ๐ช|๐ ๐ฉ , ๐๐ช ) ๐ ๐ฟ๐ฉ (๐ ๐ฉ )โ ๐๐ฉ
๐โโซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐ ๐จ , ๐๐ฉ )๐ท (๐ ๐ฉ๐ช|๐๐ฉ , ๐๐ช ) ๐ ๐ฟ ๐ฉ (๐๐ฉ )โ ๐ ๐ฉ
โข - probability for a match in camera-pair (A, B).
โข Any classifier can be used here provided that it can be modified to output probability.
โข TRID uses the ICT algorithm by training two ICT ReID classifiers. The SVM decision values are converted to probability estimates using a sigmoid according to Plattโs method.
AB
CICT [A B] ICT [B C]
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Transitive Algorithm
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )=โซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐๐จ , ๐๐ฉ)๐ท (๐ ๐ฉ๐ช|๐ ๐ฉ , ๐๐ช ) ๐ ๐ฟ๐ฉ (๐ ๐ฉ )โ ๐๐ฉ
๐โโซ๐๐ฉ
๐ท (๐ ๐จ๐ฉ|๐ ๐จ , ๐๐ฉ )๐ท (๐ ๐ฉ๐ช|๐๐ฉ , ๐๐ช ) ๐ ๐ฟ ๐ฉ (๐๐ฉ )โ ๐ ๐ฉ
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช )โ
๐|๐บ๐ฉ|
โ๐๐ฉโ๐บ๐ฉ
๐ท (๐ ๐จ๐ฉ|๐ ๐จ ,๐ ๐ฉ )๐ท (๐ ๐ฉ๐ช|๐๐ฉ ,๐๐ช )
๐โ ๐|๐บ๐ฉ|
โ๐๐ฉโ๐บ๐ฉ
๐ท (๐ ๐จ๐ฉ|๐ ๐จ , ๐๐ฉ ) ๐ท (๐ ๐ฉ๐ช|๐๐ฉ , ๐๐ช )
โข is a multi-dimensional probability density function.
โข Can be estimated by methods for density estimation using all .
โข However, estimating high-dimensional density is hard and the integration is
computationally costly.
โข Thus, in TRID the integral is approximated by a sum relying on the
smoothness of the probability functions.
AB
C
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Synthetic Experiment 1
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50๐ ๐จร๐ ๐ฉ ๐๐ฉร ๐๐ช ๐ ๐จร๐๐ช
๐ต๐๐๐๐ ๐ฐ๐ช๐ป ๐ป๐น๐ฐ๐ซ๐ฐ๐ช๐ป ๐๐๐๐๐๐๐ ๐๐ ๐จ๐ช๐ ๐๐๐
AB
C-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
-40 -20 0 20 40
-20
0
20
-50 0 50
-50
0
50
0 20 40 60 80 1000
50
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRID
-50 0 50
-50
0
50
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช ):
๐ ๐จร๐๐ช ๐ ๐จร๐๐ช ๐ ๐จร๐๐ช
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
Synthetic Experiment 2๐ ๐จร๐ ๐ฉ ๐๐ฉร ๐๐ช ๐ ๐จร๐๐ช
AB
C
๐ท (๐ ๐จ๐ช|๐ ๐จ ,๐๐ช ):
๐ ๐จร๐๐ช ๐ ๐จร๐๐ช ๐ ๐จร๐๐ช
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
-20 0 20
-20
0
20
-50 0 50-50
0
50
-50 0 50-50
0
50
0 10 20 30 40 50 60 70 80 90 1000
50
100
Rec
ogni
tion
%
(d)
rank score
ICT trained on AC dataNaive ICTTRID
๐ต๐๐๐๐ ๐ฐ๐ช๐ป ๐ป๐น๐ฐ๐ซ๐ฐ๐ช๐ป ๐๐๐๐๐๐๐ ๐๐ ๐จ๐ช๐ ๐๐๐
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experimentโข Testing TRID requires an annotated dataset associated with at least 3 stationary
cameras. โข Common ReID benchmark datasets (VIPeR , CAVIAR4REID, iLIDs MCTS, ETHZ) are
unsuitable for our set-up.โข Multi-camera surveillance database named SAIVT-SoftBio presented by Bialkowski et al:
โข image from: A. Bialkowski, S. Denman, P. Lucey, S. Sridharan, and C. B. Fookes. โA database for person re-identification in multi-camera surveillance networks.โ (DICTA 2012)
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 12 14 16 18 2010
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 12 1410
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3 C5 C7C1
C1 C3 C8C5C1 C3
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3
ABC
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
SAIVT-SoftBio Experiment
0 2 4 6 8 10 12 14 16 18 2010
20
30
40
50
60
70
80
90
100
rank score
Rec
ogni
tion
%
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 12 1410
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] =
0 2 4 6 8 10 1210
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C5 C7C3 C5 C7C1
C1 C3 C8C5C1 C3
SAIVT-SoftBio ExperimentIntroduction Motivation The Transitive Alg. Experiments Summary & Future Work
0 5 10 1510
20
30
40
50
60
70
80
90
100
rank score
Reco
gniti
on %
ICT trained on AC dataNaive ICTTRIDSDALF
[A B C] = C1 C3 C8
A BC
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Summaryโข When performing ReID in multi-camera site, we do not have to choose
between the less accurate direct method and the computationally expensive learning based method. We may perform limited learning, such that requires reasonable resources. We have shown that transitively using such learning performs well for the ReID task.
โข A specific algorithm based on marginalization was suggested: the TRID - which presents a new approach of transitivity in ReID.
โข The accuracy of the TRID is superior to that of a state-of-the-art non learning based approach.
โข The TRID algorithm is in fact a general framework that may be combined with different probabilistic classifiers (not necessarily ICT) and of course different features.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
Future Work
๐ช๐
๐ช๐
๐ช๐
๐ช๐
โข Improve the approximation of the transitive integral.
โข Transitively use indirect training sets to strengthen direct learning when the set of direct annotated pairs is small, but not empty.
โข Test the effect of recursively applying the TRID algorithm, and to study the deterioration of the performance as a function of the number of cameras.
โข Generalize the TRID algorithm beyond the scope of ReID to other domain adaptation tasks.
Introduction Motivation The Transitive Alg. Experiments Summary & Future Work
The Matlab source code of TRID as well as of the ICT are available at:http://www.cs.technion.ac.il/~tammya/Reidentification.html .
Thank-You.