Prof. Sinisa Todorovic

sinisa@eecs.oregonstate.edu

ECE 468 / CS 519: Digital Image Processing

Gradients, Harris Corners

1

Image Gradient along X-axis

I

x

(x, y) = I(x + 1, y)� I(x, y)

=

2

40 0 00 �1 10 0 0

3

5

| {z }D

x

(x,y)

⇤I(x, y)

2

Image Gradient along Y-axis

=

2

40 0 00 �1 00 1 0

3

5

| {z }Dy(x,y)

⇤I(x, y)

Iy(x, y) = I(x, y + 1)� I(x, y)

3

Filtering Image Gradient

convolution is associative

w(x, y;�) ⇤ I

x

(x, y) = w(x, y;�) ⇤ D

x

(x, y) ⇤ I(x, y)

= [w(x, y;�) ⇤ D

x

(x, y)] ⇤ I(x, y)

4

= [Dx

(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)

Filtering Image Gradient

w(x, y;�) ⇤ I

x

(x, y) = w(x, y;�) ⇤ D

x

(x, y) ⇤ I(x, y)

= [w(x, y;�) ⇤ D

x

(x, y)] ⇤ I(x, y)

convolution is commutative

5

= [Dx

(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)

Filtering Image Gradient

w(x, y;�) ⇤ I

x

(x, y) = w(x, y;�) ⇤ D

x

(x, y) ⇤ I(x, y)

= [w(x, y;�) ⇤ D

x

(x, y)] ⇤ I(x, y)

= w

x

(x, y;�) ⇤ I(x, y)

derivative of the filter6

Weighted Image Gradient

Image is discrete ⇒ Gradient is approximate

We always find the gradient of the filter !

w(x, y;�) ⇤ I

x

(x, y) = w

x

(x, y;�) ⇤ I(x, y)

w(x, y;�) ⇤ Iy(x, y) = wy(x, y;�) ⇤ I(x, y)

7

Interest Points

• Harris corners

8

Properties of Interest Points

• Locality -- robust to occlusion, noise

• Saliency -- rich visual cue

• Stable under affine transforms

• Distinctiveness -- differ across distinct objects

• Efficiency -- easy to compute

9

Example of Detecting Harris Corners

10

Harris Corner Detector

homogeneous region⇓

no change in all directions

edge⇓

no change along the edge

corner⇓

change in all directions

Source: Frolova, Simakov, Weizmann Institute

scanning window

11

Harris Corner Detector

2D convolutionSource: Frolova, Simakov, Weizmann Institute

E(x, y) = w(x, y) ⇤ [I(x + u, y + v)� I(x, y)]2

12

Harris Corner Detector

change weighting window shifted image original image

Source: Frolova, Simakov, Weizmann Institute

E(x, y) =X

m,n

w(m, n)[I(x + u + m, y + v + n)� I(x + m, y + n)]2

13

Harris Detector Example

input image

E(x, y)I(x, y)

2D map of changes

14

Harris Corner Detector

image derivatives along x and y axes

Taylor series expansion

For small shiftsu � 0v � 0

I(x + u, y + v) ⇡ I(x, y) +@I

@x

u +@I

@y

v

I(x + u, y + v) ⇡ I(x, y) + [Ix

I

y

]

u

v

�

15

Harris Corner Detector

E(x, y) = w(x, y) ⇤✓

I(x, y) + [Ix

I

y

]

u

v

�� I(x, y)

◆2

= w(x, y) ⇤✓

[Ix

I

y

]

u

v

�◆2

= w(x, y) ⇤✓

[Ix

I

y

]

u

v

�◆T ✓[I

x

I

y

]

u

v

�◆

16

Harris Corner Detector

E(x, y) = w(x, y) ⇤ [u v]

I

x

I

y

�[I

x

I

y

]

u

v

�

= [u v]✓

w(x, y) ⇤

I

2x

(x, y) I

x

(x, y)Iy

(x, y)I

x

(x, y)Iy

(x, y) I

2y

(x, y)

�◆

| {z }M(x,y)

u

v

�

17

= [Dx

(x, y) ⇤ w(x, y;�)] ⇤ I(x, y)

Filtering Image Gradient

w(x, y;�) ⇤ I

x

(x, y) = w(x, y;�) ⇤ D

x

(x, y) ⇤ I(x, y)

= [w(x, y;�) ⇤ D

x

(x, y)] ⇤ I(x, y)

= w

x

(x, y;�) ⇤ I(x, y)

derivative of the filter18

Harris Corner Detector

E(x, y) = [u v]M(x, y)

u

v

�

M(x, y) =

(wx

⇤ I)2 (wx

⇤ I)(wy

⇤ I)(w

x

⇤ I)(wy

⇤ I) (wy

⇤ I)2

�

(x,y)

19

Eigenvalues and Eigenvectors of M

• Eigenvectors of M -- directions of the largest change of E(x,y)

• Eigenvalues of M -- the amount of change along eigenvectors

��1

��2

E(x, y) = [u v]M(x, y)

u

v

�

20

Detection of Harris Corners using the Eigenvalues of M

�2

�1

21

Harris Detector

f(�) =�1(�)�2(�)

�1(�) + �2(�)=

det(M(�))trace(M(�))

objective function

22

Harris Detector Example

input image

23

Harris Detector Example

f values color codedred = high valuesblue = low values

24

Harris Detector Example

f values thresholded

25

Harris Detector Example

Harris features = Spatial maxima of f values

26

Example of Detecting Harris Corners

27

Properties of Harris Corners

• Invariance to variations of imaging parameters:

• Illumination?

• Camera distance, i.e., scale ?

• Camera viewpoint, i.e., affine transformation?

28

Harris/Hessian Detector is NOT Scale Invariant

Source: L. Fei-Fei

“edge”

“edge”“edge”

“corner”

29

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

30

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

31

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

32

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

33

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

34

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

Source: Tuytelaars

35

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

36

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

37

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

38

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

39

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

40

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σxIfmii "

�

Source: Tuytelaars

41

Automatic scale selectionAutomatic scale selection

)),((1

σxIfmii�

Function responses for increasing scale Scale trace (signature)

)),((1

σ ""xIfmii� Source: Tuytelaars

42