Image SegmentationImage Segmentation

Seminar IIISeminar IIIXiaofeng FanXiaofeng Fan

Today’s PresentationToday’s Presentation

Problem DefinitionProblem Definition ApproachApproach SegmentationSegmentation Methods Methods ApplicationsApplications Question?Question?

DefinitionDefinition

What is Image Segmentation What is Image Segmentation

Image segmentation is a partitioning of an Image segmentation is a partitioning of an image into related sections or regions.image into related sections or regions.

Category:Category:

According the goal:According the goal:• Complete segmentationComplete segmentation

• Partial SegmentationPartial Segmentation

Approaches to Image Approaches to Image Segmentation Segmentation

Most image segmentation approaches can be Most image segmentation approaches can be placed in one of three classes: placed in one of three classes:

Characteristic feature thresholding or clusteriCharacteristic feature thresholding or clustering;ng;

Boundary detection; Boundary detection;

Region based algorithm. Region based algorithm.

Region-based algorithmRegion-based algorithm

Straightforward region growingStraightforward region growing

Hybrid region growing and spectral clustHybrid region growing and spectral clustering approach ering approach

The two ways to segmentation The two ways to segmentation

Recursive Hierarchical Image SegmentatRecursive Hierarchical Image Segmentation ion

SwendsenSwendsen-Wang algorithm-Wang algorithm

HYBRID OF REGION GROWING AND SPECTRAL CLUSTERINGHYBRID OF REGION GROWING AND SPECTRAL CLUSTERING APPROAAPPROA

CHCH

A high-level outline of the new hybrid image segmentation approach is as follows: A high-level outline of the new hybrid image segmentation approach is as follows:

Label each image pixel as a separate region and set the global criterion value, Label each image pixel as a separate region and set the global criterion value, critvalcritval, e, equal to zero. qual to zero.

Calculate the dissimilarity criterion value between each spatially adjacent region. Calculate the dissimilarity criterion value between each spatially adjacent region. Dissimilarity Criterion :Dissimilarity Criterion :

For two regions For two regions ii and and jj, characterized by the mean vectors X, characterized by the mean vectors Xii = (x1 = (x1ii, x2, x2ii,..., x,..., xcici))T and XT and Xjj = (x1 = (x1jj, x2, x2jj,..., x,..., xcjcj)T, the Euclidean Spectral Distance (ESD) is defined as:)T, the Euclidean Spectral Distance (ESD) is defined as:

Find the smallest dissimilarity criterion value, and merge all pairs of spatially adjacent Find the smallest dissimilarity criterion value, and merge all pairs of spatially adjacent regions with this criterion value. regions with this criterion value.

Calculate the dissimilarity criterion value between all pairs of non-spatially adjacent rCalculate the dissimilarity criterion value between all pairs of non-spatially adjacent regions. egions.

Merge all pairs of non-spatially adjacent regions with dissimilarity criterion value less tMerge all pairs of non-spatially adjacent regions with dissimilarity criterion value less than or equal to the criterion value found in step 3. han or equal to the criterion value found in step 3.

2 1/ 2

1

[ ( ) ]l

i ki kjk

ESD x x

HYBRID OF REGION GROWING AND SPECTRAL CLUSTERING APPROACHYBRID OF REGION GROWING AND SPECTRAL CLUSTERING APPROACH (con)H (con)

If the number of regions remaining is less than the preset value If the number of regions remaining is less than the preset value minregionsminregions, go to , go to step 7. Otherwise go to step 2. step 7. Otherwise go to step 2.

Let Let prevcritval = critvalprevcritval = critval. Reset . Reset critvalcritval to be the current global criterion value. If to be the current global criterion value. If preprevcritvalvcritval = zero, go to step 2. Otherwise calculate = zero, go to step 2. Otherwise calculate cvratio = critval/prevcritval.cvratio = critval/prevcritval. If If cvrcvratioatio is greater than the preset threshold is greater than the preset threshold convfactconvfact, save the region label map from , save the region label map from the the previousprevious iteration as a "raw" segmentation result. If the number of regions re iteration as a "raw" segmentation result. If the number of regions remaining is two or less, save the region label map from the maining is two or less, save the region label map from the currentcurrent iteration as the iteration as the coarsest instance of the final hierarchical segmentation result, and go to step 8. Ocoarsest instance of the final hierarchical segmentation result, and go to step 8. Otherwise go to step 2. therwise go to step 2.

Calculate the global criterion value over each region separately. For the region wiCalculate the global criterion value over each region separately. For the region with maximum global criterion value, search backwards from the last "raw" segmenth maximum global criterion value, search backwards from the last "raw" segmentation result (from step 7) to an iteration where this region is split into two or mortation result (from step 7) to an iteration where this region is split into two or more regions. Replace the labeling for this region with the labeling from the found itee regions. Replace the labeling for this region with the labeling from the found iteration and store as the next more detailed level of the final hierarchical segmentaration and store as the next more detailed level of the final hierarchical segmentation. Repeat this step until the preset number of levels in the final hierarchical segtion. Repeat this step until the preset number of levels in the final hierarchical segmentation is obtained.mentation is obtained.

Improvement:Improvement:

Step 4 requires the calculation of the dissimilarity criterion valStep 4 requires the calculation of the dissimilarity criterion value between each region and every other region in the image. Due between each region and every other region in the image. Divide-and-conquer method is devised to reduce the computatiivide-and-conquer method is devised to reduce the computational load.onal load.

The divide-and-and conquer algorithm:The divide-and-and conquer algorithm:1.1. Specify the number of levels of recursion required (Specify the number of levels of recursion required (nblevelsnblevels), p), p

ad the image size to even;ad the image size to even;2.2. If If levellevel > 1, divide the image data into quarters. Otherwise go t > 1, divide the image data into quarters. Otherwise go t

o step 4;o step 4;3.3. Run step 2 to step 1 completely processingRun step 2 to step 1 completely processing , reassemble the i, reassemble the i

mage segmentation results;mage segmentation results;4.4. Execute the image segmentation algorithm with the following Execute the image segmentation algorithm with the following

modification: If modification: If level < nblevels,level < nblevels, terminate the algorithm when terminate the algorithm when the number of regions reaches the number of regions reaches minregionsminregions/2, and do not check /2, and do not check for for critvalcritval or output any "raw" segmentation results. or output any "raw" segmentation results.

SwendsenSwendsen-Wang algorithm-Wang algorithmBasic character:

1. Given an adjacency graph, algorithm compute a local probability at each edge for how likely the two vertices (image elements) belong to the same pattern.

2. At each step, the algorithm splits, merges, or regroups a connected component

3. The algorithm “mixes” rapidly at low components density.

The Swendsen-Wang Cuts AlgorithmThe Swendsen-Wang Cuts Algorithm

1. Initialize a graph partition 2. Repeat, for current state A

n

llGG

1

:

Swendsen-Wang Cuts: SWC-1Input: Go=<V, Eo>, discriminative probabilities qe, e Eo, and generative posterior probability p(W|I).Output: Samples W~p(W|I).

3. Repeat for each subgraph Gl=<Vl, El>, l=1,2,...,n in A 4. For e El turn e=“on” with probability qe.

5. Partition Gl into nl connected components: gli=<Vli, Eli>, i=1,...,nl

6. Collect all the connected components in CP={Vli: l=1,...,n, i=1,...,nl}.

7. Select a connected component V0CP at random8. Propose to reassign V0 to a subgraph Gl’, l' follows a probability q(l'|V0,A,Go)

9. Accept the move with probability α(AB) or α(AC).State A

V 0

CP

V 0

V 1

V 2

x

x

x

x

x

x

The Swendsen-Wang Cuts AlgorithmThe Swendsen-Wang Cuts Algorithm

State A

State C

State B

A

CB

V 0

V 1

V 2

x

x

x

x

x

x

x

V 0

V 1

V 2

x

x

x

xx

xx

xx

x

V 0

V 1

V 2

State A State B

))|,(

)|,(

),,|'(

),,|(

)1(

)1(,1min(

))(

)(

)(

)(,1min()(

'

'

0

0

),(

),(

0

'0

AVVp

BVVp

GAVlq

GBVlq

q

q

Ap

Bp

BAq

ABqBA

ll

ll

o

o

eVVCe

eVVCe

l

l

Swendsen-Wang Cuts: Main TheoremSwendsen-Wang Cuts: Main Theorem

Theorem. Accepting the proposed move with probability

results in an ergodic and reversible Markov Chain.

V 0

V 1

V 2

x

x

x

x

x

x

x

V 0

V 1

V 2

x

x

x

xx

xx

xx

x

The Swendsen-Wang Cuts AlgorithmThe Swendsen-Wang Cuts Algorithm

Advantages:• ergodic and reversible in the space of graph partitions • achieves fast mixing at low temperature

- eliminates the slow simulated annealing procedure• converges in 3-30 seconds in a PC for image segmentation

- about 400 times faster than the classical Gibbs sampler- 20-40 times faster than the DDMCMC algorithm.

• the algorithm can optimize over the number of models • works for general forms of posterior probabilities.

- more general than the existing graph cut approaches.

Example:Example:

Thank youThank you

Question?