Contributions A people dataset of 8035 images. Three layer attribute classification framework using...

Contributions

A people dataset of 8035 images. Three layer attribute classification framework using poselets.

1 2

People Dataset

H3D (Humans in 3D) Pascal VOC 2010 (trn+val)

8035images

TRN: 2003 VAL: 2011 TEST: 4022

9 different attributes in total.

At least 2 attributes per image

Agreement of 4 out of 5

1

Attribute classification using poselets2

What is a What is a Poselet Poselet ??

Poselets capture part of the pose from a given viewpointPoselets capture part of the pose from a given viewpoint[Bourdev & Malik, ICCV09][Bourdev & Malik, ICCV09]

PoseletsPoselets

Examples may differ visually but have common semanticsExamples may differ visually but have common semantics[Bourdev & Malik, ICCV09][Bourdev & Malik, ICCV09]

PoseletsPoselets

But how are we going to create training examples of poselets?But how are we going to create training examples of poselets?

How do we train a poselet for a given How do we train a poselet for a given pose configuration?pose configuration?

Finding correspondences at training timeFinding correspondences at training time

Given part of a human poseGiven part of a human pose How do we find a similar pose How do we find a similar pose configuration in the training set?configuration in the training set?

We use keypoints to annotate the joints, eyes, nose, etc. of peopleWe use keypoints to annotate the joints, eyes, nose, etc. of people

Left Hip

Left Shoulder

Finding correspondences at training timeFinding correspondences at training time

Residual ErrorResidual Error

Finding correspondences at training timeFinding correspondences at training time

Training poselet classifiersTraining poselet classifiers

ResiduResidual al Error:Error:

0.10.155

0.20.200

0.10.100

0.30.355

0.10.155

0.80.855

1.1. Given a seed patchGiven a seed patch2.2. Find the closest patch for every other personFind the closest patch for every other person3.3. Sort them by residual errorSort them by residual error4.4. Threshold themThreshold them

Training poselet classifiersTraining poselet classifiers

1.1. Given a seed patchGiven a seed patch2.2. Find the closest patch for every other personFind the closest patch for every other person3.3. Sort them by residual errorSort them by residual error4.4. Threshold themThreshold them5.5. Use them as positive training examples to Use them as positive training examples to

train a linear SVM with HOG featurestrain a linear SVM with HOG features

Which poselets should we train?Which poselets should we train?

• Choose thousands of random windows, generate poselet candidates, train linear SVMs

• Select a small set of poselets that are:– Individually effective– Complementary

Some Poselets

Attribute classification using poselets2

Features• HOGs at two levels (~2K-4K features)

– 16 x 16– 32 x 32

• Color Histograms in H,S,B (30 features)– 10 bins for H, S and B

• Skin classifier output (3 features)– GMM with 5 components– Fraction of skin pixels– hands-skin, legs-skin, neck skin

Poselet-level Attribute Classifiers

Person-level Attribute Classifiers

Context-level Attribute Classifiers

Results

Results

Results

Results

Gender Classification Results

Thanks