Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
The Truth About Cats And Dogs Omkar M Parkhi1,2 Andea Vedaldi1 C. V. Jawahar2 Andrew Zisserman1
Objective
1Department of Engineering Science, University of Oxford 2 International Institute of Information Technology, Hyderabad, India E-mail : {omkar,vedaldi,az}@robots.ox.ac.uk, [email protected]
This research is funded by UKIERI, ONR MURI N00014-07-1-0182 and ERC grant VisRec no. 228180.
Quantitative Results
Cat detection on PASCAL VOC 2010 Validation data.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Recall
Prec
isio
n
PR Curve: Cat detection results
gc−basic AP: 0.37dpm−ber−rr AP: 0.41dpm−rr AP: 0.46dpm AP: 0.48ub AP: 0.67
Method AP Basic GrabCut 0.37 Adding Global Posteriors 0.41 Adding Berkeley Edges 0.46 Re ranking the detections 0.48
Effect of the system components on validation data
Qualitative Results
Overview
Distinctive Part Detection
Image
Object Detection
Grab-Cut Segmentation
To detect the whole object: 1) Detect the distinctive part (head). 2) Segment the animal based on distinctive part. 3) Obtain the animal bounding box from the segmentation and the distinctive part.
Motivation
Cats and dogs are very varied in their imaged shape
• Key observation: DefPMs are • very poor at detecting the whole cat. • very good at detecting a distinctive part such as the head.
3) Post Processing • Alternate two steps [Rother et al. 04]
• Estimate appearance model • Minimize standard graph-cut energy function to assign a correct label yi to each pixel xi.
Appearance Model • Data term
• from GMM estimated using head region and from global posterior probabilities.
• Pairwise term from edge detector output • Model initialization
• Foreground From head detection. • Background From predicted bounding box. (Boundary region).
• Model update re-estimating GMMs using the output from the previous step.
2-a) Grab-Cut Segmentation
! !,! = !− !!!!!log!(!!! !!! + ! !(!! ,!,!!!!Neighbours
!!|!!)!Energy Data Term Pairwise Term
Seeds for modeling GMMs Segmentation Output: Effect of poor data and edge terms.
Method Cat Dog Felzenswalb et. al 2010 31.8 21.5 Our Method 45.3 36.8
Detection results on PASCAL VOC 2010 Test data
Dataset and Annotations
Object Part Trimap
• PASCAL VOC 2010 annotations: cats and dogs • Bounding boxes • Segmentations (trimaps)
• Additional head annotations • Bounding boxes
2-c) Global Model
• Head seed insufficient to model object foreground. • Posterior probabilities given by histograms learnt from
ground truth segmentations over all training images • Used as data term for the first iteration of segmentation.
Posteriors using head seed only
Posteriors after introduction of global model
Improved segmentation due to global posteriors.
• Clean up: erode, dilate, and select component connected to head.
• Adjust predicted bounding box to be consistent with head detection.
• Rerank detections based on the head size.
Problem Overview Deformable Part Models (DefPMs) such as [Felzenszwalb et al. 09] are not deformable enough for very flexible objects such as cats and dogs. We extend DefPMs by introducing Distinctive Parts Models (DisPMs), which combine DefPMs with segmentation and successfully detect highly deformable animals.
The distinctive part is modeled by a DefPM. • Parts connected by springs • HOG + LBP features • Fast inference with dynamic programming • Discriminatively trained by Latent variable SVM
• DefPMs are much better at detecting the head than the whole animal.
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
Recall
Prec
isio
n
PR Curve: Object Detection and Part Detection
Cat Detection AP: 0.28Cat Head Detection AP: 0.49
1) Part Model
Deformable Parts Model for Cat Head
2-b) Modeling Edges • Berkeley PB edge detector [Martin et. al. 04] • Edge detector response used as pairwise term.
• Cut encouraged at higher edge potential. • Significant improvement over gradients edge terms.
! !! ,!! !) = !!!!!(!!(!)/!) !
Edge Detector Response Improvement segmentation output due to Berkeley edge model
Incorrect Segmentation
Incorrect Segmentation
Correct Segmentation
Correct Segmentation
Foreground Seed
Background Seed