Ter Haar Romeny, MICCAI 2008 Regularization and scale-space

ter Haar Romeny, MICCAI 2008

Regularization and scale-space


Regularization is the technique to make data behave well when an operator is applied to them. A small variation of the input data should lead to small change in the output data.

Differentiation is a notorious function with 'bad behaviour'.

2 4 6 8 10 12

4

3

2

1

1

2

3

10 20 30

1

0.5

0.5

2 1 1 2

0.2

0.4

0.6

0.8

2 1 1 2 3 4 5

0.2

0.4

0.6

0.8

1

Some functions that can not be differentiated.


In mathematical terms it is said that the operation of differentiation is ill-posed, the opposite of well-posed. Jacques Hadamard (1865–1963) stated the conditions for well-posedness:

• The solution must exist;• The solution must be uniquely determined;• The solution must depend continuously on the initial or boundary data.

In other words, the solution should be stable.


• smoothing the data, convolution with some extended kernel, like a 'running average filter' or the Gaussian;• interpolation, by a polynomial (multidimensional) function;• energy minimization, of a cost function under constraints• fitting a function to the data (e.g. splines). The cubic splines are named so because they fit to third order;• graduated convexity [Blake1987];• deformable templates ('snakes') [McInerney1996];• thin plates splines [Bookstein1989];• Tikhonov regularization.


The formal mathematical method to solve the problems of ill-posed differentiation was given by Laurent Schwartz:

A regular tempered distribution associated with an image is defined by the action of a smooth test function on the image. TL

Lxx x

i1...inTL 1n

Lxi1...inx xThe derivative is:


Fields Medal 1950 for his work on the theory of distributions. Schwartz has received a long list of prizes, medals and honours in addition to the Fields Medal. He received prizes from the Paris Academy of Sciences in 1955, 1964 and 1972. In 1972 he was elected a member of the Academy. He has been awarded honorary doctorates from many universities including Humboldt (1960), Brussels (1962), Lund (1981), Tel-Aviv (1981), Montreal (1985) and Athens (1993).

Laurent Schwartz (1915 - 2002)


Mathematics Smooth test function

Computer vision Kernel, filter

Biological vision Receptive field


Relation regularization - Gaussian scale-space

An essential result in scale-space theory was shown by Mads Nielsen (Copenhagen University). He proved that Tikhonov regularization is essentially equivalent to convolution with a Gaussian kernel.

Eg

f g2 x minimize this function for g, given the constraint that the derivative behaves well.

Eg

f g2 1gx2 xEuler-Lagrange:


Eg

f g2 1gx2

f g2 1gx2

f g2 12g2

In the Fourier domain the expressions are easier:

d Egd g

2f g2 212g 0

f g 12g 0 g 1112 f


3 2 1 1 2 3

0.2

0.4

0.6

0.8

1.2

1

1 x21

1

In the spatial domain:

Filter proposed by Castan, 1990


Eg

f g2 1gx2 2g

x x

f g2 12g2 24g2

Including the second order derivative:

d Egd g

2f g2 212g 24g 0

g1

1 12 24f

4 2 2 4

0.2

0.3

0.4

0.5

0.6

Function proposed by Deriche (1987)

Etc.


12 2 2

2 1

2 2

2

4 4

8

6 6

48

8 8

384

10 10

3840 O11

Taylor expansion of the Gaussian in the Fourier domain:

By recursion:

Tikhonov regularization is equivalent to Gaussian blurring

4 2 2 4

0.2

0.4

0.6

0.8

1


121

000

121

horSobel

101

202

101

vertSobel

1 0 1

1

0

1

vert

hor

Neighbor

Neighbor

Older implementations are approximations of the Gaussian kernel:

1 1

1

1

vert

hor

Neighbor

Neighbor

Documents

Ter Haar Romeny, MICCAI 2008 Regularization and scale-space