DD2423 Image Processing and Computer Vision

EDGE DETECTIONMarten Bjorkman

Computater Vision and Active Perception

School of Computer Science and Communication

November 8, 2012

1

General

Under rather general assumptions about the image formation the world consistsof smooth regular surfaces with different reflectance properties.

One can assume that a discontinuity in image brightness corresponds to somediscontinuity in (and their combinations):

• depth,

• surface orientation,

• reflectance, or

• illumination.

Thus, find discontinuities in image brightness (edges) and characterize these withrespect to the physical phenomena that gave rise to them.

2

Example: Line drawing

Luminance and second order derivative.

3

Example: Line drawing

Noisy luminance and second order derivative.

4

Why edges?

• Edge features are perhaps the most important features to humans.

• Independent of illumination.

• Easy to detect computationally.

• Used to form higher level features (lines, curves, corners, etc).

5

How do edges look in practice?

Ideal models:

RidgeLineRamp edgeStep edge

In practice, edges are blurred and noisy.

Problem: Concept of discontinuity doesn’t exist for discrete data!

6

Fundamental problem

Differentiation is ill posed - an arbitrary small perturbation in the input can leadto arbitrarily large perturbation in the output.

Ex: f (x) = arctan(x) f ′(x) = 11+x2

f (x) = arctan(x)+ εsinωx f ′(x) = 11+x2 + εωcosωx .

• The difference εωcosωx can be arbitrarily large if ω >> 1/ε.7

Noise reduction: Smoothing

• Basic idea: Precede differentiation by smoothing.

• Trade-off problem:

- increasing amount of smoothing: stronger suppression of noise, higherdistortion of “true” structures.

- decreasing amount of smoothing: more accurate feature detection, highernumber of “false positives”.

8

Basic methods for edge detection

• Linear:

- Differentiation (derivatives)

- High-pass filtering

- Matching with model patterns

• Non-linear:

- Fitting of parameterized edge models

- Non-linear diffusion

• Common approach:

1. Detect edge points

2. Link these to polygons

3. Abstraction: Fit to model (straight lines, splines, ellipses )

9

Edge attributes and problems

• Attributes:

- Position

- Orientation

- Strength

- Diffuseness (width)

• Problems:

- Image noise

- Interference (nearby structures at different scales)

- Physical interpretation

10

Method: gradients and derivatives

• Most methods are gradient based.

• Detection of maxima of gradient.

• Detection of zero crossings of second derivative.

• Edge pixels need to be connected together to get to a sequenceof edge points (edge linking).

11

Image gradient

12

Image gradient

13

Image gradient

14

Image gradient

• So the gradient is a vector with magnitude in the x and y directionsequal to the respective partial derivatives.

• Q: How do we compute partial derivatives of a discrete function?

• A: Taylor series approximation!

15

Image gradient

16

Image gradient

17

Image gradient

18

Basic edge detection

19

Magnitude

20

Edge detection in practice

21

Edge detection in practice

22

Sobel operator

Common in many edge detection schemes.

Sv =

1 0 −12 0 −21 0 −1

=

121

∗ (1 0 −1)

Sh =

1 2 10 0 0−1 −2 −1

=

10−1

∗ (1 2 1)

• Smoothing in one direction, derivative in the other.

• Usually, even more smoothing is needed.

23

Sobel operator (cont)

24

Different filter types

25

Example: Roberts vs. Prewitt filter

The major edges are better with Prewitt than Roberts, but Sobel is best.

26

Zero crossings - Laplacian operator

Instead of searching the maxima of the gradient, we may search zero crossingsof the second order derivative.

27

LoG and DoG

LoG (Laplacian of Gaussian): Gaussian smoothing with variance σ2 followed by aLaplacian operator:

DoG (Difference of Gaussian): An efficient approximation of a LoG based on thedifference between two Gaussian smoothing operator with different variances:

28

Difference of Gaussians

29

The Laplacian operator - Properties

• Advantages:

- Closer to mechanisms of visual perception.

- One parameter only (size of the filter).

- No threshold is required.

- Produces closed contours.

• Disadvantages:

- Is more sensitive to noise (usage of second derivative).

- No information on the orientation of the contour.

• Combination of gradient and contour:

- Search for zero-crossings of the Laplacian in the neighborhood oflocal maxima of the gradient.

30

Laplacian operator

31

Zero crossings

32

Canny Edge Detector

1. Convolve image with derivatives of a Gaussian.

2. Compute gradient magnitude.

3. Perform non-maximum suppression.

4. Perform Hysteresis Thresholding.

Based on a Master thesis. Most popular edge detection method since 1986.

33

Canny Edge Detector

• Typical problem: If you detect edges by gradient thresholding, the resultingedges can be severel pixels wide.

• The Canny Edge Detector adds an additional refinement to address this:

- Estimate edge direction Ed.

- Estimate edge strength Es (usually gradient magnitude).

- Remove non-maximum edge pixels in the Ed direction.

34

1. Gaussian smoothing

35

2. Gradient magnitude

36

Gradient orientation

37

3. Non-maximum suppression

Implementation by discrete local search

1. Interpolate

- gradient magnitude Es =| ∇I |=√

δIδx

2+ δI

δy2, using the

- gradient direction Ed = tan−1(δIδy/

δIδx)

2. Perform local search at each point

⇒ gives edge points that can be linked into polygons.

38

Gradient orientation

39

Non-maximum suppression (faster)

40

Non-maximum suppression (example)

41

And after thresholding...

Raw thresholding on gradient magnitude may result in highly fragmented edges.

42

4. Hysteresis thresholding

• Problem: Thresholding on gradient magnitude may lead to fragmented edges.

43

Hysteresis thresholding

44

Edge linking

Link neighboring edge pixels to connected contours.Each point has strength | ∇L | and orientation Θ.

Algorithm: ∀ pixels

if (| ∇L |> threshold) [ and is locally maximum ]

∀ neighbor’s

if (| ∇L | of neighbor > threshold) [ and is locally maximum ]

and ( |Θthis−Θneighbor |< threshold)

link these pixels to be connected

Combine with efficient traversal procedure and mechanism for closing gaps.

45

Differential geometric edge definition

Non-maximum suppression:

Edge point = point where the gradient magnitude assumes a maximumin the gradient direction.

This can be expressed in terms of

Gradient: ∇L = (Lx,Ly)T , and

Gradient magnitude:√

L2x +L2

y.

Normalized gradient direction:

n =∇L| ∇L |

=(Lx,Ly)

T√L2

x +L2y

46

Differential geometric edge definition

Directional derivative in direction v = (cosα,sinα):

∂v = cosα∂x + sinα∂y

Directional derivative in gradient direction:

∂n =Lx√

L2x +L2

y

∂x +Ly√

L2x +L2

y

∂y

Denote:∂nL = Ln =

Lx√L2

x +L2y

Lx +Ly√

L2x +L2

y

Ly

47

Gradient magnitude

Requirements for gradient magnitude to be maximal in gradient direction:

Lnn = 0 Lnnn ≤ 0

• In terms of coordinates:

Lnn = (cos∂x + sin∂y)2L = cos2

αLxx +2cosαsinαLxy + sin2αLyy

=L2

x√L2

x +L2y

Lxx +2LxLy√L2

x +L2y

Lxy +L2

y√L2

x +L2y

Lyy

=(L2

xLxx +2LxLyLxy +L2yLyy)√

L2x +L2

y

= 0

• Since denominator is irrelevant, edges are given by

L2xLxx +2LxLyLxy +L2

yLyy = 0

L3xLxxx +3L2

xLyLxxy +3LxL2yLxyy +L3

yLyyy ≤ 0

48

Summary: gradient based edge detection

• Edges correspond to abrupt changes in image intensity.

• Edges can be detected by

– Smooth out image noise.– Estimate the gradient of the image at every point in the image.– Threshold the gradient image.

AND / OR– Find points where the gradient is maximum in the gradient direction.– Connect edge pixels into edge segments.

49

Detecting Edges by Matching

Idea: Match local image patches to a set of edge models withdifferent orientation.

Direction = orientation of best match (strongest response)

Edge strength = strongest response

Example: (−1 1−1 1

),(

0√

2−√

2 0

),(

1 1−1 −1

),(√

2 00 −

√2

)Natural measure: Normalized cross correlation

C f g =C f g√

C f f√

Cgg

50

Summary of good questions

• Why is edge detection important for image understanding?

• Why is edge detection difficult in practice?

• What families of methods exist for edge detection?

• What information do image gradients provide?

• Why can Laplacians be useful for edge detection?

• Why are differences of Gaussians interesting?

• How does the Canny edge detector work?

• What is hysteresis thresholding?

• What should the image derivatives be equal to on edge points?

51

Readings

• Related sections of Chapter 4, 5 and 10

52