Embed Size (px)
DESCRIPTION
Action Classification: An Integration of Randomization and Discrimination in A Dense Feature Representation. Bangpeng Yao, Aditya Khosla , and Li Fei-Fei. Computer Science Department, Stanford University {bangpeng,aditya86,feifeili}@cs.stanford.edu. Outline. - PowerPoint PPT Presentation
Citation preview
1
Action Classification: An Integration of Randomization and Discrimination in A
Dense Feature Representation
Computer Science Department, Stanford University
{bangpeng,aditya86,feifeili}@cs.stanford.edu
Bangpeng Yao, Aditya Khosla, and Li Fei-Fei
2
• Action Classification & Intuition
• Our Method
• Our Results
• Conclusion
Outline
• Action Classification & Intuition
• Our Method
• Our Results
• Conclusion
Outline
3
4
Action Classification
Object classification:
Presence of parts and their spatial configurations.[Lazebnik et al, 2006][Fergus et al, 2003]…
Phoning RidingBike Running
5
Action Classification
• All images contain humans;
Object classification:
Presence of parts and their spatial configurations.[Lazebnik et al, 2006][Fergus et al, 2003]…
6
Action Classification
• All images contain humans;• Objects small or absence;
Object classification:
Presence of parts and their spatial configurations.[Lazebnik et al, 2006][Fergus et al, 2003]…
7
Action Classification
• All images contain humans;• Objects small or absence;• Large pose variation & occlusion;• Background clutter;
Challenging…
Object classification:
Presence of parts and their spatial configurations.[Lazebnik et al, 2006][Fergus et al, 2003]…
8
Our Intuition
Focus on image regions that contain the most discriminative information.
9
Our Intuition
Focus on image regions that contain the most discriminative information.
How to represent the features? Dense feature space
Randomization & DiscriminationHow to explore this feature space?
Outline
10
• Action Classification & Intuition
• Our Method
• Our Results
• Conclusion
11
... ... ...
......
......
Region Height
Region Width
Dense Feature Space
Normalized Image
Size of image region
Center of image region
12
... ... ...
......
......
Region Height
Region Width
Normalized Image
Size of image region
Center of image region
Dense Feature Space
13
... ... ...
......
......
Region Height
Region Width
Normalized Image
Size of image region
Center of image region
Dense Feature Space
14
... ... ...
......
......
Region Height
Region Width
How can we identify the discriminative regions efficiently and effectively?
Normalized Image
Size of image region
Center of image region
Dense Feature Space
Image size: N×NImage regions: O(N6)
15
... ... ...
......
......
Region Height
Region Width
Normalized Image
Size of image region
Center of image region
Apply randomization to sample a subset of image patches
Dense Feature Space
Random Forests (RF)
16
... ... ...
......
......
Region Height
Region Width
This class Other classes
Random Forests (RF)
Normalized Image
Size of image region
Center of image region
Dense Feature Space
17
... ... ...
......
......
Region Height
Region Width
RF with discriminative classifiers
Normalized Image
Size of image region
Center of image region
This class Other classes
Dense Feature Space
18
Generalization Ability of RF
• Generalization error of a RF:
: correlation between decision trees: strength of the decision trees
• Discriminative classifiers
Better generalization
• Dense feature space decreases
increases
19
… … … …
RF with Discriminative Classifiers
20
… … … …
Train a binary SVM
RF with Discriminative Classifiers
1
2
3
4
5
0
1
1
1
0
BoW or SPM of SIFT-LLC features
21
… … … …
Train a binary SVM
RF with Discriminative Classifiers
1
2
3
4
5
0
1
1
1
0Biggest information gain
22
… … … …
Train a binary SVM
RF with Discriminative Classifiers
1
2
3
4
5
0
1
1
1
0
23
… … … …
RF with Discriminative Classifiers
• We stop growing the tree if:- The maximum depth is reached;- There is only one class at the node;- The entropy of the training data at the node is low
24
Classification With RF
… … … …
Number of treesClass Label
• Action Classification & Intuition
• Our Method
• Our Results
• Conclusion
Outline
25
26
Results on VOC 2011 Actions
ActionOthers’
BestOur
Method
Jumping 71.6 66.0
Phoning 50.7 41.0
Playing instrument 77.5 60.0
Reading 37.8 41.5
Riding bike 88.8 90.0
Riding horse 90.2 92.1
Running 87.9 86.6
Taking photo 25.7 28.8
Using computer 58.9 62.0
Walking 59.5 65.9
Our method ranks the first in six out of ten classes.
27
Results on VOC 2011 Actions
ActionOthers’
BestOur
Method
Jumping 71.6 66.0
Phoning 50.7 41.0
Playing instrument 77.5 60.0
Reading 37.8 41.5
Riding bike 88.8 90.0
Riding horse 90.2 92.1
Running 87.9 86.6
Taking photo 25.7 28.8
Using computer 58.9 62.0
Walking 59.5 65.9
28
Results on VOC 2011 Actions
ActionOthers’
BestOur
Method
Jumping 71.6 66.0
Phoning 50.7 41.0
Playing instrument 77.5 60.0
Reading 37.8 41.5
Riding bike 88.8 90.0
Riding horse 90.2 92.1
Running 87.9 86.6
Taking photo 25.7 28.8
Using computer 58.9 62.0
Walking 59.5 65.9
29
Generalization Ability of RF
• Discriminative classifiers
Better generalization
• Dense feature space Tree correlation decreasesTree strength increases
0 100 200 300 4000.2
0.3
0.4
0.5
0.6
0.7
Number of trees
Mea
n A
vera
ge-P
reci
sion
dense feature, weak classifierSPM feature, strong classifierdense feature, strong classifier
dense feature (spatial pyramid)SPM feature
Vs.
strong classifier weak classifierVs.
Train discriminative SVM classifiers
Generate feature weights randomly
(Results on PASCAL VOC 2010)
• Action Classification & Intuition
• Our Method
• Our Results
• Conclusion
Outline
30
31
Conclusion
• Exploring dense image features can benefit action classification;
• Combining randomization and discrimination is an effective way to explore the dense image representation;
• Achieves very good performance based on only one type of image descriptor;
• Code will be available soon.
32
… … … …
Train a binary SVM
1
2
3
4
5
0
1
1
1
0
Acknowledgement
Bangpeng Yao, Aditya Khosla, and Li Fei-Fei. “Combining Randomization and Discrimination for Fine-Grained Image Categorization.” CVPR 2011.
Thanks to Su Hao, Olga Russakovsky, and Carsten Rother.
Reference:
