Minimum Likelihood Image Minimum Likelihood Image Feature and Scale DetectionFeature and Scale Detection

Kim Steenstrup PedersenKim Steenstrup Pedersen

Collaborators:Collaborators:

Pieter van Dorst, TUe, The NetherlandsPieter van Dorst, TUe, The Netherlands

Marco Loog, ITU, DenmarkMarco Loog, ITU, Denmark

Gaussian Processes in Practice

2

What is an image feature?What is an image feature?

• Marr’s (1982) primal sketch (edges, Marr’s (1982) primal sketch (edges, bars, corners, blobs)bars, corners, blobs)

• Geometrical features, Marr’s features Geometrical features, Marr’s features defined by differential geometry: defined by differential geometry: Canny (1986), Lindeberg (1998) Canny (1986), Lindeberg (1998)

• Iconic features: Koenderink (1993), Iconic features: Koenderink (1993), Griffin & Lillholm (2005) Griffin & Lillholm (2005)

Observation: Features are usually points Observation: Features are usually points and curves, i.e. sparsely distributed in and curves, i.e. sparsely distributed in space (unlikely events).space (unlikely events).Features have an intrinsic scale / size. Features have an intrinsic scale / size. How blurred is the edge?How blurred is the edge?What is the size if a bar?What is the size if a bar?

Gaussian Processes in Practice

3

A probabilistic primal sketchA probabilistic primal sketch

Our definition: Features are points that are unlikely to occure Our definition: Features are points that are unlikely to occure under an image model. under an image model. Similarly the scale of the feature is defined as the most Similarly the scale of the feature is defined as the most unlikely scale.unlikely scale.

• We use fractional Brownian images as a generic model of We use fractional Brownian images as a generic model of the intensity correlation found in natural images. Captures the intensity correlation found in natural images. Captures second order statistics of generic image points (non-feature second order statistics of generic image points (non-feature points).points).

• The model includes feature scale naturally.The model includes feature scale naturally.

• This leads to a probabilistic feature and scale detection.This leads to a probabilistic feature and scale detection.

Possible applications: Possible applications: Feature detection, interest points for object recognition, Feature detection, interest points for object recognition, correspondance in stereo, tracking, etc.correspondance in stereo, tracking, etc.

Gaussian Processes in Practice

4

Probabilistic feature detectionProbabilistic feature detection

Feature detection:Feature detection:

• Konishi et al. (1999, 2002, 2003)Konishi et al. (1999, 2002, 2003)

• Lillholm & Pedersen (2004)Lillholm & Pedersen (2004)

Scale selection:Scale selection:

• Pedersen & Nielsen (1999)Pedersen & Nielsen (1999)

• Loog et al. (2005)Loog et al. (2005)

Gaussian Processes in Practice

5

Linear scale-space derivativesLinear scale-space derivatives

Scale-space derivatives:Scale-space derivatives:

Gaussian Processes in Practice

6

Scale Space k-Jet RepresentationScale Space k-Jet Representation

We use the k-jet as representationWe use the k-jet as representationof the local geometry:of the local geometry:

(The coefficients of the truncated(The coefficients of the truncatedTaylor expansion of the blurredTaylor expansion of the blurredimage.)image.)

Biologically plausibleBiologically plausiblerepresentation representation (Koenderink et al., 1987)(Koenderink et al., 1987)

Gaussian Processes in Practice

7

Probabilistic image modelsProbabilistic image models

Key results on natural image statistics:Key results on natural image statistics:

• Scale invariance / Self-similarity: Scale invariance / Self-similarity: Power spectrum, : Field (1987), Power spectrum, : Field (1987), Ruderman & Bialek (1994) Ruderman & Bialek (1994)

• In general non-Gaussian filter responses!In general non-Gaussian filter responses!

Fractional Brownian images as model of natural Fractional Brownian images as model of natural images:images:

• Mumford & Gidas (2001), Pedersen (2003), Mumford & Gidas (2001), Pedersen (2003), Markussen et al. (2005)Markussen et al. (2005)

• Jet covariance of natural images resembles that of Jet covariance of natural images resembles that of fractional Brownian images: Pedersen (2003)fractional Brownian images: Pedersen (2003)

Gaussian Processes in Practice

8

Fractional Brownian imagesFractional Brownian images

Gaussian Processes in Practice

9

FBm in Jet spaceFBm in Jet space

(Result from Pedersen (2003))(Result from Pedersen (2003))

Gaussian Processes in Practice

10

Detecting Features and ScalesDetecting Features and Scales

Detecting points in scale-space that are Detecting points in scale-space that are locally unlikely (minima):locally unlikely (minima):

(We could also have maximised .)(We could also have maximised .)

Gaussian Processes in Practice

11

Why minimum likeli scales?Why minimum likeli scales?

Lindeberg (1998) maximises polynomials of Lindeberg (1998) maximises polynomials of derivatives in order to detect features and derivatives in order to detect features and scales.scales.

Similarly, we maximise Similarly, we maximise in order to detect features and scales.in order to detect features and scales.

The difference lies in the choice of The difference lies in the choice of polynomial! polynomial! We use an image model and Lindeberg uses We use an image model and Lindeberg uses a feature model.a feature model.

Gaussian Processes in Practice

12

Synthetic examples: Double blobsSynthetic examples: Double blobs

Gaussian Processes in Practice

13

Synthetic examples: Blurred step Synthetic examples: Blurred step edgeedge

0 10 20 30 40 50 600

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Measurement scale (in )

Pro

ba

bili

ty

= 1 = 1.5 = 2 = 2.5 = 3

Gaussian Processes in Practice

14

Real Example: SunflowersReal Example: Sunflowers

Gaussian Processes in Practice

15

Sunflowers: Multi-scale Sunflowers: Multi-scale

Gaussian Processes in Practice

16

Sunflowers: Fixed scaleSunflowers: Fixed scale

Gaussian Processes in Practice

17

SummarySummary

• Minimising the likelihood of an image point Minimising the likelihood of an image point under the fractional Brownian image model under the fractional Brownian image model detects feature points and their intrinsic scale.detects feature points and their intrinsic scale.

• There is a relationship between feature types There is a relationship between feature types and the and the parameter. parameter.

• Why over estimation of the scale?Why over estimation of the scale?

• Preliminary results look promising, a Preliminary results look promising, a performance evaluation is needed (task performance evaluation is needed (task based?).based?).

• The method is pointwise. How to handle curve The method is pointwise. How to handle curve features (edges, bars, ridges)?features (edges, bars, ridges)?