Mul$-‐criteria Energy Minimiza$on with Boundedness, Edge-‐density and Rarity, for Object Saliency in Natural Images
Abstract
Sudeshna Roy and Sukhendu Das Visualiza$on and Percep$on Lab.
Dept. of Computer Science and Engineering, IIT Madras, India
Algorithm Overview
Results and Evalua$ons on PASCAL VOC 2012 Dataset
Image
Edge Map
Superpixels and Edges
Proposed Output Edge Density (edi)
Saliency (si)
Boundedness (bi)
Conclusion
• Mo#vated by saliency and category independent object segmenta#on methods, we predict a single object proposal which captures all the salient objects in an image, within O(number of superpixels)3.
• The CRF parameters are learned using a margin-‐rescaled struct-‐SVM approach with (1-‐ IoU) loss func#on. • Fast but effec#ve objectness criteria, viz. boundedness and edge-‐density. Submodular CRF is minimized using graph-‐cut.
Results are shown on the challenging PASCAL VOC 2012 dataset.
• An efficient method based on saliency in conjunc#on with objectness cues to predict a category-‐independent object segmenta#on, using a CRF based formula#on.
• Can be a preprocessing step for many high-‐level vision task.
References
Method CPMC [1] OP [2] Proposed IoU Score 0.3319 0.3266 0.4097
[1] [5] [4] [3] [2]
1. Carreira and Sminchisescu, “Constrained parametric min-‐cuts for automa#c object segmenta#on,” CVPR, 2010. 2. Endres and Hoiem, “Category independent object proposals,” ECCV, 2010. 3. Roy and Das, “Saliency detec#on in images using graph-‐based rarity, spa#al compactness and background
prior,” VISAPP, 2014. 4. Yang, et. al., “Saliency detec#on via graph-‐based manifold ranking,” CVPR, 2013. 5. Perazzi et. al., Saliency filters: Contrast based filtering for salient region detec#on. CVPR. 2012.
Input
Objectness
Image Plane
i
j
Unary Poten$al (Saliency, Objectness)
Pairwise Poten$al (based on color)
Submodular CRF Inference: Graph-‐cut
* *
* Considering top 10 proposal masks
φi,j(2) (yi, yj, xi, xj|w2)
φi(1)(yi,xi|w1)
Salient Object Likelihood: φi(1)(yi, xi(1)|w1) = ws ( (1-‐ yi) (1-‐si) + yisi ) + wb ( (1-‐ yi) (1-‐bi) + yibi )
+ wed ( (1-‐ yi) edi + yi (1-‐edi) )
φi,j(2) (yi, yj, xi(2), xj(2)) = |yi -‐ yj| exp(-‐ kc||ci – cj||) Edge Cost:
ci denotes color of ith superpixel in lab space and kc indicates the sensi#vity of color similarity.
CRF parameter Learning: • Convex in w, given L • Itera#vely op#mized, by finding yn using graph-‐cut, and thus L • M is the number of training instances. •
wmin
12w 2 +
1M
ξnn=1
M∑
subject to, wTv =1
wTφn −wTφn ≤ L (yn,yn )+ξn
ξn ≥ 0,w2 > 0L = (1-‐ IoU)
E(y, x,w) =
E(y, x,w) = φi(1)
i∈V∑ +λ φi, j
(2)
(i, j)∈E∑Energy Func#on over superpixels:
w = [w1 w2 ]= [ws wb wed λ]T