Upload
mitsuru-nakazawa
View
383
Download
1
Tags:
Embed Size (px)
Citation preview
HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
Presenter: Mitsuru NAKAZAWA@Osaka Univ., JPN
Omar Oreifej† and Zicheng Liu‡†University of Central Florida‡Microsoft Research
CVPR2013 paper introduction
1
• This slide is unofficial one because I am a presenter, NOT an author of HON4D– Presenter: Mitsuru Nakazawa@Osaka univ. JPN– nakazawa[at]am.sanken.osaka-u.ac.jp
• HON4D– http://www.cs.ucf.edu/~oreifej/HON4D.html
• Required knowledge: HOG
2
University of Bonn - Institute of Computer Science III - Computer Vision Group 3
Histogram of 4D Surface Normals
• Surface normals:• Quantization according to “projectors” pi: • Add additional discriminative “projectors”
[ O. Oreifej and L. Zicheng. Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences. CVPR 2013 available at http://www.cs.ucf.edu/~oreifej/HON4D.html ]
[URL] Accessed Sept. 25, 2013
Bins of the histogram
Introduction• Compared with conventional color images, depth
maps provide several advantages– Depth maps reflect pure geometry and shape cues
• It seems natural to employ depth data in many computer vision problems like action recognition
Would conventional RGB-based methods also perform well in depth sequences??
4
Related work 1
It is difficult to apply these methods because– Detectors such as STIP and Dollar are not reliable in depth sequences
– Standard methods for automatically acquiring motion trajectories in color images are also not reliable in depth sequences
STIP (Laptev et al. 2005) [10] Dollar (Dollar et al. 2005) [5]
5
Local interest point-based methods originally developed for color sequences
Related work 2• Holistic approaches – instead of using local points,
a global feature is obtained for the entire sequence
Yang et al. 2012 [26]Vieira et al. 2012 [21]HON4D Yang et al. 2012 [26]
We demonstrate that our method captures the complex and articulated structure and
motion within the sequence using a richer and more discriminative descriptor than [26] and [21]
6
Contributions
1. We propose a novel descriptor for activity recognition from depth sequences, in which we encode the distribution of the surface normal orientation in the 4D space of depth, time, and spatial coordinates.
2. We demonstrate how to quantize the 4D space using the vertices of a polychoron, and then refine the quantization to become more discriminative.
7
HON4D can uses different bins based on the gradient
(1)
4D Surface Normal
4th dimension encodes the magnitude of the gradient
Normalize the normal to an unit length normal
8
HOG vs. HONHOG
HON
9
Similar distribution
Discriminable distribution
Gradient orientation is similar for both surfaces
Normalization using the sum across all projectors
Component of each normal in each direction
Histogram of 4D Normals for each cell
• Projectors obtained from 600cells (One of polychorons)
– 600cells divides the 4D space uniformly with 120 vertices– 4D space is quantized using 120 vertices Projectors
10
600cellsStella: Polyhedron Navigatorhttp://www.software3d.com/Stella.php
Projector refinementWhen two different classes of activities are quite close in thefeature space such that their samples mostly fall in similar bins
11
Is uniform space quantization optimal??
We set the weighting coefficients for projectors by using SVM
Training HON4D descriptorSupport vectorWeight corresponding to the support vector
Experiments using existing databases • MSR Action 3D[12], MSR Gesture 3D[23]
12
MSR Action 3D[12] MSR Gesture 3D[23]
HON4D does not use a skeleton tracker, and yet we outperform the skeleton-based method [24]
New database: 3D Action Pairs Dataset
Although the two actions of each pair are similar in motion and shape, the motion-shape relation is different
13
Skeleton + LOP Skeleton + LOP + Pyramid
HON4D HON4D + Discriminative density
Local HON4D• For the case when actors significantly change their spatial locations, and the
temporal extent of the activities significantly vary
14
Experiment using MSR Daily Activity 3D [24]
• Local HON4D: 80.00% • Local occupancy pattern feature (LOP, Wang et al. 2012): 67.50%
Local HON4D: Histogram of 4D Normals of spatiotemporal patches centered at skeleton joints
HON4D is also superior for significantly non-aligned sequences
Conclusion• We presented a novel, simple, and easily implementable
descriptor for activity recognition from depth sequences. – We initially quantize the 4D space using the vertices of a 600-cell
polychoron, and use that to compute the distribution of the 4D normal orientation for each depth sequence.
– We estimate the discriminative density at each vertex of the polychoron, and induce further vertices accordingly, thus placing more emphasis on the discriminative bins of the histogram.
• We showed by experiments that the proposed method outperforms all previous approaches on all relevant benchmark datasets.
15