View
2
Download
0
Category
Preview:
Citation preview
From Contours to Regions: An Empirical Evaluation
presented by: Yingjie Tang
Outline:• Motivation • Contours VS Regions • Contour Detection • Oriented Watershed Transform • Ultrametric Contour map • Evaluation:
Benchmark & Results • Conclusions & Contributions
Motivation of Segmentations: 1. Provide natural domains for computing features used in
recognition 2. Visual tasks can benefit from the complexity reduction
Contours VS Segmentation:
Contours: Not closed Segments: Closed and can provide partition
gPb Detector: Oriented Gradient of Histograms
gPb Detector: Multiscale/ Multi- channel Pb
gPb Detector: weighted combination of multi scale local cues
s: indexes scales i: indexes feature channels(brightness, color a, color b,
texture) measures the histogram difference in channel i between two halves of disc centered at (x,y) at angle θ.
gPb Detector: Globalization
gPb Detector: Globalization
gPb Detector: Globalization
Benefit of Globalization
gPb Detector: Globalization
Benefit of Globalization
Segmentation:
Contour Detector
Oriented Watershed
Transformation
Ultrametric Contour
Map
Hierarchical Segmentation▪ From contours to segmentation ▪ Watershed Transform ▫ Concept
13—Hsin-Min Cheng
Hierarchical Segmentation▪ From contours to segmentation ▪ Watershed Transform ▫ Example
14—Hsin-Min Cheng
Hierarchical Segmentation▪ From contours to segmentation ▪ Watershed Transform
15
Boundary strength ArtifactsWeight each arc
—Hsin-Min Cheng
Hierarchical Segmentation▪ From contours to segmentation ▪ Oriented Watershed Transform
16
WTOWT
—Hsin-Min Cheng
Segmentation:Watershed Transform.
Image Boundary Strength E(x,y)
Partition image Reweight
Disadvantages: Since the contour detector produces a spatially extended response around strong boundaries, simply weighting each arc by mean value of E(x,y) can introduce artifacts.
Segmentation:Watershed Transform
ImproveOriented Watershed Transform
Input boundary signal E(x, y) = maxθ E(x, y, θ).
Watershed arcs computed from E(x,y).
Watershed arcs with an approximating straight line segment subdivision overlaid.
Oriented boundary strength E(x, y, θ) for four orientations θ.
Oriented boundary strength E(x, y, θ) for four orientations θ.
Segmentation:Contour Subdivision
Scale Invariant
Segmentation:Ultrametric Contour Map Original
image
Base level: Over-segmentation
Upper level: Under-segmentation
Moving between levels offers a continuous trade-off between these extremes
Segmentation:Ultrametric Contour Map
1. Construct graph G = (P0, K0, W (K0)) given by OWT 2. Iteratively merge regions by removing min weight boundary. 3. Produces region tree where:
• Root is entire image • Leaves are P0 • Height(R) is boundary threshold at which R first appears • Distance(R1, R2) = min{Height(R) : R1, R2 ⊆ R}
Segmentation:Ultrametric Contour Map
Merging Algorithm Description:
Evaluation: On Berkeley Segmentation Dataset(BSDS)
• Precision-Recall on Boundaries • Variation of Information • Rand Index • Segmentation Covering
https://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/BSDS300/html/dataset/humans/1105-0171-0180.html
Evaluation: Precision-Recall on Boundaries
A B
C D
prediction outcome
actual valueP F
P’
F’
R(recall) = A/(A+C)P(precision) = A/(A+B)
F-measure = 2*PR/(P+R)
• Optimal Dataset Scale(ODS) • Optimal Image Scale(OIS) • Average Precision(AP)
Evaluation: Precision-Recall on Boundaries
Evaluating boundaries on the Berkeley dataset
Contours Hierarchical Regions Segmentations
The proposed approach outperformed the state of art
Evaluation: Precision-Recall on Boundaries
Evaluating boundaries on the Berkeley dataset
Boundary benchmarks on the BSDS
Evaluation: Variation of Information & Rand Index & Segmentation Covering
Rand Index between test and ground- truth segmentations S and G is given by the sum of the number of pairs of pixels that have the same label in S and G and those that have different labels in both segmentations, divided by the total number of pairs of pixels.
V I(C, C′) = H(C) + H(C′) − 2I(C, C′)Measures the distance between two segmentations in terms of their average conditional entropy
Evaluation: Variation of Information & Rand Index & Segmentation Covering
Region benchmarks on the BSDS
Evaluating regions on the BSDS
Conclusions & Contributions:
• Algorithm gPb-owt-ucm, offers the best performance on every dataset and for every benchmark criteria tested
• This algorithm is straight- forward, fast, has no parameters to tune, and supports interactive user refinement
• The generic grouping machinery has found use in optical flow and object recognition applications.
Q & A
Constrained Parametric Min-Cuts for Automatic Object
Segmentation
Outline • Constrained Parametric Min-Cuts(CPMC) • The Grid Geometry • Fast Rejection • Ranking Segments • Experiments
Constrained Parametric Min-Cuts(CPMC)
Min-Cuts:a minimum cut of a graph is a cut (a partition of the vertices of a graph into two disjoint subsets that are joined by at least one edge) whose cut set has the smallest number of edges (unweighted case) or smallest sum of weights possible. —Wikipedia
Maximum flow: maximum flow problems involve finding a feasible flow through a single-source, single-sink flow network that is maximum. —Wikipedia
Constrained Parametric Min-Cuts(CPMC)
object segmentation framework
Constrained Parametric Min-Cuts(CPMC)
Energy function in pixels:
Fast RejectionLarge set of initial segmentations (~5500)
High Energy Low Energy
~2000 segments with the lowest energy
Cluster segments based on spatial overlap (at least 0.95)
Lowest energy member of each cluster (~154 in PASCAL VOC)Credit:
SasiKanth Bendapudi Yogeshwar Nagaraj
Ranking Segments:
3 sets of features: • Graph partition properties (8 features) • Region properties (18 features) • Gestalt properties (8 features)
Experiments:
Three Datasets: • Weizmann’s Segmentation Evaluation Database
(with only one prominent foreground object in each and test F-measure criteria)
• MSRC dataset(nearly 11 objects in each image and is used to evaluate the quality of the pool of segments
generated) • VOC 2009 dataset(most complicated and used
segmentation covering as an accuracy measure)
Experiments: Segment Pool Quality
Average of best segment F-measure scores over the entire dataset.
Average best image covering scores on MSRC and VOC2009 train+validation datasets
Experiments: Segment Pool Quality
Quality of the segments in VOC2009 joint train and validation sets for the segmentation problem
Evaluating the Ranking Object Hypothesis
Average best segment F-measure as the number of retained segments from the ranking is varied.
Evaluating the Ranking Object Hypothesis
Complementing the basic descriptor set with appearance and shape features improves the ranking slightly, but with a more expressive
regressor, random forests, the basic set is still superior.
Conclusion & Contributions
• Presented an algorithm that casts the automatic image segmentation problem as one of finding a set of plausible figure-ground object hypotheses.
• It is shown that the proposed framework is able to generate small sets of segments that represent the objects in an image more accurately than existing state of the art segmentation methods.
Q&A
Recommended