Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Dr. Iyad Jafar
Digital Image Analysis and ProcessingCPE 0907544
Image Segmentation – Part I
Chapter 10
Sections : 10.1 – 10.2
Outline
Introduction
Fundamentals
Point, Line, and Edge Detection
Edge Linking and Boundary Detection
The Hough Transform
2
Introduction What is image segmentation?
The process of subdividing an image into its individualcomponents/objects
A central operation in all machine vision applications The level of subdivision depends on the problem in hand It is nontrivial for complex images The accuracy of the segmentation process determines
the correct operation of computerized vision systems
3
Segmentation
Fundamentals Let R represents the entire spatial region occupied by
the image. The segmentation processes subdivides Rinto n subregions R1, R2, …, Rn such that
4
1
(1)
(2) is a connected set, i = 1,2,3,...,n
(3) , for all i and j, i j
(4) Q( ) TRUE, for i = 1,2,3,...,n
(5) Q( ) FALSE, for any adjacent regions and
n
ii
i
i j
i
i j i j
R R
R
R R
R
R R R R
Fundamentals Segmentation can be achieved by
Detecting discontinuities (edge-based segmentation) Assumes that the boundaries between the objects themselves
and the background are sufficiently different
Detecting similarities (region-based segmentation) Assumes that the objects to be segmented are similar according
to some criteria
5
Boundary Detection
Region Filling
Boundary Detection
Standard deviation similarity
Point, Edge, Line Detection Edges
Edge pixels are pixels at which the intensity ofan image function changes abruptly
Edges are sets of connected edge pixels
Lines Are edge segments in which the intensity ofthe background on either side of the line ismuch higher or lower than the intensity of linepixels
Point A line whose length and width is roughly onepixel
6
Line
Isolated Point
Edge
Point, Edge, Line Detection The discontinuities we are interested in can be detected
using derivatives
First Derivative
Second Derivative
Notes First derivative generally produce thicker edges Second order derivative has stronger response to fine details Second order derivative produces double-edge response at ramp and step
transitions The sign of the second derivative determines the type of transition
7
'( ) ( 1) ( )f x f x f x ''( ) ( 1) ( 1) 2 ( )f x f x f x f x
Point Detection We will use second derivative as it has stronger response
to discontinuities
or
8
2 22
2 2
f ff
x y
2
21 1 2
ff ( x , y ) f ( x , y ) f ( x, y )
x
2
21 1 2
ff ( x, y ) f ( x, y ) f ( x, y )
y
2 1 1 1 1 4f f ( x , y ) f ( x , y ) f ( x, y ) f ( x, y ) f ( x, y )
0 1 0
1 -4 1
0 1 0
1 1 1
1 -8 1
1 1 1
Point Detection We saw earlier that isolated points have very high
response for second derivatives
We can detect the points by simply thresholding theabsolute value of the second derivative R(x,y) using
The main concern is in specifying the threshold value
9
1 , |R(x,y)| T g( x, y )
0 , otherwise
Image Second Derivative Image Thresholded Second Derivative Image with T = 0.95 max(R)
Line Detection
Again , we will use second derivative as it hasstronger response to discontinuities and producesthinner edges than first derivative
Keep in mind that the second derivative producesdouble-line effect.
Solution Take the absolute value of the Laplacian image (produces
thicker lines) Take the positive values of the Laplacian image only (in
noisy images we have to use a threshold to reduce theeffect of noise)
10
Line Detection Example 10.1
11
Original Scaled Laplacian Image
Absolute Value of Laplacian Image Positive Values of Laplacian Image Only
Double-line Effect
Thicker Lines
1 1 1
1 -8 1
1 1 1
Laplacian Mask
Line Detection The mask used in the previous is isotropic, i.e. it is
independent of line direction.
It is desirable sometimes to detect lines in specificdirections (0o, +45o, -45o, and 90o). We can use maskswhose coefficients along the desired direction have largerweights
These masks provide strong response for bright lines inthe specified direction and that are one pixel thick
In order to detect a line in a specific direction, we use thecorrect mask then we threshold for positive result only12
Line Detection Example 10.2. Detection of lines at 45o
13
Original Filtered Image
Positive Values of Filtered Image Thresholded Image
Strong Response
2 -1 -1
-1 2 -1
-1 -1 2
Mask
Weak Response
Isolated Points that can be removed easily by point
detection
A line that is 1 pixel thick in the 45o
direction
Edge Detection Edges can be classified according to intensity profiles into
Step Edge (ideal edge) Involves transition between two intensity levels over a distance of one pixel Occur mostly in computer generated images
Ramp Edge The transition between two intensity levels occur over a distance that is greater
than one pixel Appear in real images as a result of noise and focusing limitations of imaging
devices
Roof Edge Essentially, they represent blurred lines that pass through a region
14Ideal Edge Ramp Edge Roof Edge
Edge Detection It is not unusual to find the three types of edges in one image
15
Ideal Edge
Ramp Edge Roof Edge
Edge Detection Detection of edges can be through the use of
first-order or second-order derivatives For the first derivative, the magnitude can be used to detectthe presence of an edge
For the second derivative, the sign of the second derivative isused to detect the presence of the edge
16
Edge Detection Although it sounds straight forward to detect edges using first-order
or second-order derivatives, the presence of noise may affect thisoperation significantly depending on noise level
17 Edge First Derivative Second Derivative
Incr
eas
ing
Noi
se L
eve
l
In general, detecting edges, involves :
1) Smoothing to reduce the sensitivity of derivatives to noise
2) Detection of all possible edge points
3) Edge localization to select from the candidate points those that comprise the edge
Edge Detection Using Gradient The gradient is a powerful tool in finding the strength and
direction of edges.The gradient at pixel (x,y) is defined as
The magnitude of the gradient measures the strength ofthe edge (maximum rate of change)
The direction of the gradient is perpendicular to the edgedirection
18
Edge Detection Using Gradient
Theoretically, all edge pixels have the same gradientmagnitude and direction
19
Edge Detection Using Gradient Gradient Masks to compute gradient at Z5
Discrete 1st Derivative
Not efficient in detecting diagonal edges
Roberts Cross-gradient Operator
2x2 masks are not as good as symmetric masks which captureinformation from opposite sides around the center point20
-1 0
0 1
0 -1
1 0
Horizontal Operator
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9
Pixel z5 and its neighboursVertical Operator
-1
1-1 1
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
-1 0 1
-1 0 1
-1 0 1
Edge Detection Using Gradient Gradient Masks to compute gradient at Z5
Prewitt Operators
Sobel Operators
They have better response than Prewitt masks and have better smoothingwhich is essential to reduce the effect of noise21
Mask to Compute Gx Mask to Compute Gy
-1 -1 -1
0 0 0
1 1 1
Mask to Compute Gx Mask to Compute Gy
Gradient Masks to compute gradient at Z5
Prewitt and Sobel masks shown before give thestrongest response for horizontal and vertical edges. Wecan modify these masks to obtain better response fordiagonal edges Prewitt Operators
-1 -1 0
-1 0 1
0 1 1
Edge Detection Using Gradient
22
0 1 1
-1 0 1
-1 -1 0
Prewitt Diagonal Masks
Sobel Diagonal Masks
-2 -1 0
-1 0 1
0 1 2
0 1 2
-1 0 1
-2 -1 0
Example 10.3
Edge Detection Using Gradient
23
Image Gx computed using Sobel Operator
Gy computed using Sobel Operator
|Gx| + |Gy| Angle of Gradient
Example 10.3. - Continued Note that in the previous slide, the edges of the wall bricks
were successfully detected. However, this might not be desirable if we are interested in the main edges
We can eliminate such small edges (which might considered as noise) by Smoothing the image before computing the gradient Thresholding the gradient image Smoothing followed by thresholding
Edge Detection Using Gradient
24Gradient Image (|Gx| + |Gy|) after the original
image was smoothed by 5x5 maskGradient Image (|Gx| + |Gy|) without
smoothing
Example 10.3. - Continued
Edge Detection Using Gradient
25
Gradient Image (|Gx| + |Gy|) after the original image was smoothed by 5x5 mask
Gradient Image (|Gx| + |Gy|) without smoothing
Thresholded Gradient Image Thresholded Gradient Image (better connectivity for edges)
Example 10.3. - Continued The Sobel masks used in the previous slides were those that
have stronger response for vertical and horizontal directions.How about diagonal directions?
Edge Detection Using Gradient
26
Gx computed using Sobel Operator (Horizental) Gy computed using Sobel Operator (vertical)
Gx computed using Diagonal Sobel Operator (+45) Gx computed using Diagonal Sobel Operator (-45)
The Marr-Hildreth Edge Detector
The method is based on the following arguments Intensity changes are not independent of image size, thus
detection of such changes requires the use of operators ofdifferent sizes
An intensity change is associated with a peak or a trough inthe first derivative and a zero-crossing in the secondderivative
A filter that could achieve such results is defined as theLaplacian of Gaussian (LoG)
27
The Marr-Hildreth Edge Detector
The zero crossings occur when x2 + y2 = 2σ2, i.e. a circle with radius 28 2
The Marr-Hildreth Edge Detector The Marr-Hildreth algorithm consists of convoluting the
LoG filter with an image f(x,y)
and then finding the zero crossings in g(x,y) to determinethe locations of edges in f(x,y)
The above equation can be rewritten as
In summary, the algorithm is performed by Filter the image f(x,y) with an nxn Gaussian mask Compute the Laplacian of the smoothed image Find the zero crossings of the resulting image in the previous
step29
The Marr-Hildreth Edge Detector
How to generate the LoG mask ?
Sample the LoG expression then scale its coefficients suchthat they sum to zero
To define a nxn LoG mask, sample the Gaussian function thenconvolve it with a Laplacian mask
How to specify the size of the LoG mask ?
Recall that 99.7% of the volume under the Gaussian lies in [-3σ,3σ]
As a rule of thumb, n should be selected to be an odd integerthat is equal or greater than 6σ
Using smaller values reults in truncation of the Gaussian whilelarger values make little difference in the result
30
The Marr-Hildreth Edge Detector
How find the zero-crossings?
Use a 3x3 neighborhood at each candidate edge pixel inthe image filtered by the LoG mask
A zero crossing at p implies that the signs of at least twoopposing neighbors pixels must differ
We have four possible combinations: left/right,top/bottom, and the two diagonals
Pixel p is considered an edge pixel if at least two of theopposing neighbors differ in sign
Sometimes it is not sufficient to check sign of the neighbors,but we may need to check their absolute difference against athreshold value
31
The Marr-Hildreth Edge Detector Example 10.4.
32
Original Result of LoG filtering with σ = 4 and n = 25
Zero crossings using a threshold of 0
Zero crossings using a threshold of 4% of the max valus of LoG image
Edge Linking and Boundary Detection
Ideally, edge detection should produce the sets of pixels thatform the edges in the image
Practically, this is not always correct ! Some detected pixels correspond to noise Some of the edges pixels are not detected due to breaks in the edges
Nonuniform illumination Efficiency of the detection method
It is a common practice to follow the detection process bylinking process to fill in the breaks and to remove non-edgepixels
Three common approaches Local - Assumes knowledge about edge points in a local region Regional – Requires that the points on the boundary be known Global – Operates on the entire edge image
33
Local Edge Linking It is based on analyzing a small neighborhood about every
candidate edge pixel in the edge image
All pixels that are similar according to some criteria arelinked to form an edge
Two principle properties used to measure similarity of edgepixels are the magnitude and direction of gradient A pixel (s,t) in a small neighborhood Sxy around pixel (x,y) is
similar to pixel (x,y) by gradient magnitude if
A pixel (s,t) in a small neighborhood Sxy around pixel (x,y) issimilar to pixel (x,y) by gradient direction if
The pixel (s,t) in Sxy is linked to pixel (x,y) if the previous twoconditions are satisfied
34
Local Edge Linking The previous approach is expensive as it requires the
investigation of each edge pixel
Alternatively, we simplify it by the following steps1) Compute the gradient magnitude and angle arrays, M(x,y) and α(x,y)
of the edge image f(x,y)
2) Form the binary image g(x,y) by using
3) Scan the rows of g(x,y) and fill (set to 1) all gaps (sets of 0s) in eachrow that don’t exceed a specified length K. A gap is bounded by 1sfrom both ends. Processing is done on each row individually
4) To detect gaps in any other direction θ, rotate g(x,y) by θ andrepeat the horizontal scanning in step 3. Once done, rotate theresult by -θ
35
Local Edge Linking
Example 10.5.
TM = 30% of maximum gradient value
A = 90o TA = 45o
K = 25 pixels
36
Original Gradient Image
Horizontally Connected Edge
Pixels
Vertically Connected Edge
Pixels
Logical OR
Regional Edge Linking
Assumes that the locations of regions of interest in theimage are known or can be determined.
In other words, the regional membership of pixels in thecorresponding edge image is known
A popular approach in this category is functionalapproximation by curve-fitting the known points (assumethat the desired edge/boundary is approximated by somemathematical expression)
Another approach is polygonal approximations. Theycapture the essential shape features of a region whilekeeping the representation of the boundary simple
37
Regional Edge Linking Polygonal Approximation
Given a set of points that represent an open curve with end pointsA and B which are assumed to be vertices of a the polygon
38
Regional Edge Linking
Algorithm: for a set of ordered distinct points P in a binaryimage, finding a polygonal fit can be done by
1) Specify two starting points, A and B2) Specify a threshold T and two empty stacks, OPEN and CLOSED3) If the points in P correspond to a closed curve, then put A into OPEN
and B into OPEN and CLOSED. If the points in P correspond to opencurve, put A into OPEN and B into CLOSED
4) Compute the parameters of the line passing through the last vertex inClosed and the last vertex in OPEN
5) Compute the distances from the line in Step 4 to points in P whosesequence places them between the vertices in Step 4. Select the pointVmax that has maximum distance Dmax.
6) If Dmax > T, then place Vmax at the end of the OPEN stack as a newvertex. Go to Step 4
7) If Dmax <= T, then remove the last vertex from OPEN and insert it asthe last vertex in CLOSED
8) If OPEN is not empty, go to Step 4, else exit. The vertices in CLOSEDare the vertices of the polygonal fit to points in P
39
Regional Edge Linking Example 10.6.
Processing is performed in the clockwise direction and the points represent a closed curve
40
1 2 3 4
5 6 7 8
Global Edge Linking and Hough Transform
The input is the edge image with all edge pixels beingcandidates for linking
Acceptance or elimination of pixels is based on some globalproperties, usually a specified shape such as a line or circle
Given n edge points, suppose we want to find subsets ofthese points that lie on straight lines.
A simple approach is find all possible lines between every pairof points then pick those lines that are close to particularlines
However, this is computationally expensive !! It requiresfinding n(n-1)/2 lines and performing n2(n-1)/2 comparisons !
41
Hough Transform Consider a point (xi,yi) in the xy-plane and the general equation of a
straight line yi = a xi + b
Infinite number of lines pass through this point with different values of aand b
If we rewrite the line equation such that b = - a xi + yi and consider theab-plane (the parameter space), then we have the line equation for afixed pair (xi,yi)
If we consider another point (xi,yi) that lie on the same line as (xi,yi), thenin the ab-space the two lines corresponding to these two point willintersect at (a’,b’) and so do all the lines for the points on the line in thexy-plane
42
In order to find the principle lines in the image, we can compute all theparameter-space lines and then select those points that have large number ofintersections
A major difficulty in this approach is that the slope of the line approaches infinityfor vertical and near vertical lines
One way around is to use the normal representation of the line in the xy-plane
In this representation, a vertical line is represented in the parameter-space byθ = 90 and p being the y-intercept.
Points that lie on the same line have their normal representation intersects at(θ’,p’). As the number of points that lie on the same line increases, we willhave more curve crossing in the parameter space at the point (θ’,p’)
Hough Transform
43
Representation of the two
points in the parameter
spaceTwo points that lie on the
same line
Intersection of the curves that represent the
two points
Procedure for finding principle lines in animage
1. Subdivide the pθ parameter space into the so-calledaccumulator cells and initialize them to zero. Therange of each of the parameters is specified bya. -90≤ θ ≤ 90b. -D ≤ p ≤D with D being the maximum diagonal
length in the image
2. For every edge/line candidate pixel (xi,yi) in the edgeimage, we substitute every possible value of θ in p =xi cos θ + yisin θ. Round the values of p to theallowed subdivision values on the p-axis
3. If θm results in pn, then increment the count in cell(m,n) by one, i.e.A(m,n) = A(m,n)+1
4. Select accumulator cells with counts greater thancertain threshold and find the corresponding θ andp for these cells
5. Draw the lines in the xy-plane for each pair of pairof θ and p found in Step 4
Hough Transform
44
Subdividing the parameter space to generate accumulator cells
Example 10.7.Hough Transform
45
Edge PixelsAll Possible Lines
Horizontal lines
Vertical lines
-45o line
45o line
Pick p and θ with crossing count = 3 to select principle
lines in the image
y
x
Example 10.8. We want to segment the two edges of the principlerunway
Hough Transform
46
Image Edge Image Parameter Space Representation of Edge Points
The Two Principle Lines Detected after Thresholding the Parameter Space
The Two Principle Lines Superimposed on Original Image
Related Matlab Functions
fspecial
Hough
edge
47