11

Contour Enhancement and Completion via Left-Invariant Second Order

Stochastic Evolution Equations on the 2D-Euclidean Motion Group

Erik Franken, Remco Duits,Markus van Almsick

Eindhoven University of Technology

Department of Biomedical Engineering

EURANDOM workshop

“Image Analysis and Inverse Problems”

December 13th 2006, Eindhoven, NL

22

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions

33

1. The retina contains receptive fields of varying sizes multi-scale sampling device

2. Primary visual cortex is multi-orientation

Biological Inspiration

• Cells in the primary visual cortex are orientation-specific

• Strong connectivity between cells that respond to (nearly) the same orientation

Measurement in Primary Visual Cortex

Bosking et al., J. Neuroscience 17:2112-2127, 1997

44

image

Orientation Scores

From 2D image f(x,y) to orientation score Uf(x,y,θ) with position (x,y) and orientation θ

x

y

xorientation score

y

55

Orientation Score is a Function on SE(2)

Properties of SE(2)• Group element

translation rotation• Group product

• Group inverseg¡ 1 = (¡ R ¡ 1

µ b; ¡ µ)

SE (2) = R2o SO(2) = Euclidean Motion GroupWe consider orientation score U 2 L2(SE (2))

66

Approach: Image Processing /analysis via Orientation Scores

“Enhancement” operation

Initial image

“Enhanced” image

Orientation score transformation

Inverse orientation score transformation

Segmented structures

Segment structures of interest

77

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions

88

Invertible Orientation Score Transformation

i.e. “fill up the entire Fourier spectrum”.

Image to orientation score

Orientation score to image:

Stable reconstruction requires

Oriented wavelet

99

Invertible Orientation Score TransformationDesign considerations: reconstruction,

directional, spatial localization, quadrature, discrete number of orientations

re(Ã) im(Ã) F [Ã]P Nµ

j =0 F [R j sµ Ã]dµ

re(WÃf )(¢;µ)example image f j(WÃf )(¢;µ)j

1010

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions

1111

Represents the “net” operator. It is Euclidean-invariant iff is left-invariant, i.e.

Left Invariant Operators

L g©U = ©L gU; i 2 f »;´;µg

(L gU)(h) = U(g¡ 1h)Where is the left-regular

representation L g

1212

Left-invariant Derivative Operators

areleft-invariant derivatives onEuclidean motion group, i.e.

@»

@́

@µ

x

µ

y

@»@́

@µ@»; @́; and @µ

L g@iU = @iL gU; i 2 f »;´;µg

Not all left-invariant derivatives on SE(2)do commute![@µ;@»] = @µ@» ¡ @»@µ = @́; [@µ; @́] = ¡ @»

=Tangent to line structures

= orthogonal to line structures

1313

Convection-diffusion PDEs on SE(2)

convection diffusion

Time process

Resolvent process

1414

Linear & Left-invariant Operators are G-convolutionsNormal 3D convolution – versus G-convolution on SE(2)

“G-Kernel”

1515

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour

completion• Nonlinear diffusion for Contour enhancement• Conclusions

1616

Direction Process on SE(2)Resolvent of linear PDE

Random walker interpretation

1717

Stochastic Completion Fields

Collision probability of 2 random walkers on SE(2):

Forward Backward

The mode line (in red) is the most likely connection curvebetween the two points

1818

An example

Noisy input image

Greens functions:

“Simple” enhancement viaOrientation score

Stochastic completion field

1919

Exact Solution by Duits and Van Almsick

Explicit PDE problem, case (Mumford) :Analytic Solution of Greens function?

2020

Practical approximations

2121

Exact Green’s Function versus Approximation

The smaller the better approximation

2222

How? •G-convolution with exact/approximate Green’s function•Finite element implementation in Fourier domain (August / Duits)•Explicit numerical schemes (Zweck and Williams)

Application of the Direction process

Non-linear enhancement step

Initial image

Image with completed contours

Orientation score transformation

Inverse orientation score transformation

Compute Stochastic Completion Field

What?

•Orientation-score gray-scale transformation (i.e. taking a power)•Angular/spatial thinning

2323

Automatic Contour completion by SE(2)-convolution

2424

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour

enhancement• Conclusions

2525

The Diffusion Equation on Imagesf = image

u = scale space of image

D = diffusion tensor

Linear diffusion Perona&Malik Coherence-enhancing diff.

2626

Diffusion equation in orientation scorescurvature

Diffusion orthogonal tooriented structures

Diffusion tangent tooriented structures

Diffusion inorientation

Evolvingorientationscore

Rotating tangent spacecoordinate basis

Left-invariantderivatives

are left-invariant derivatives onEuclidean motion group, i.e.

@»

@́

@µ

x

µ

y

@»@́

@µ

@»; @́; and @µ

L g@iU = @iL gU; i 2 f »;´;µg

2727

Example diffusion kernels

2828

• Oriented regions: D’11 and D33 small, D22 large and κ according to estimate

• Non-oriented regions: D’11 large, D22=D33 large, κ = 0

@»

@́

@µ

x

µ

y

@»@́

@µ

How to Choose Conductivity Coefficients

2929

Measure for Orientation Strength

• Hessian in Orientation Score

Note: non-symmetric due to non-commuting operators!

• Gaussian Derivatives can be used, if one ensures to first take orientational derivatives and then spatial derivatives.

• Measure for orientation strength:

3030

Curvature estimation

• If a vector points tangent to a structure in the orientation score, the curvature in that point is:

Ideally zero

• Estimation of v:1. Determine eigenvectors and eigenvalues of

2. Select the 2 eigenvectors closest to the ξ,θ-plane

3. Take eigenvector corresponding to the largest eigenvalue

3131

@»

@́

@µ

x

µ

y

@»@́

@µ

Chosen Conductivity Coefficients

3232

• Numerical scheme: explicit, left-invariant finite differences.

• Using B-spline interpolation cf. Unser et al.

Implementation

3333

Diffusion in orientation score Coherence enhancing diffusion Results

Size: 128 x 128 x 64

3434

Collagen image

Diffusion in orientation score Coherence enhancing diffusion

Size: 200 x 200 x 64

3535

Results – with/without curvature estimation

3636

Outline

• Introduction to Orientation Scores• Invertible Orientation Scores• Operations in Orientation Scores• The Direction Process for Contour completion• Nonlinear diffusion for Contour enhancement• Conclusions

3737

Conclusions

• We developed a framework for image processing via Orientation scores. Important notion: An orientation score is a function on SE(2) use group theory.

• Useful for noisy medical images with (crossing) elongated structures

• Found Analytic Solution of Greens functions• Stochastic Completion Fields of images using

G-convolutions• Non-linear diffusion on orientation scores to

enhance crossing line structures

3838

Current/Future work

• Improving adaptive/nonlinear evolutions on SE(2)– Numerical methods– Nonlinearities

• Applying in medical applications– 2-photon microscopy images of Collagen fibers– High Angular Resolution Diffusion Imaging

• Apply same mathematics in other groups, e.g. SE(3) and similitude groups.

3939

4040

Line enhancement in 3D via invertible orientation scores

Application: Enhancement Adam-Kiewitz vessel

4141

Enhancement Kidney-Boundaries in Ultra-sound images

Via Orientation Scores:

4242

Medical Application: Cardiac ArrhythmiasHeart rythm disorder by extra conductive path/spot

Catheters in hart provide intracardiogram and can burn focal spots/lines

Navigation by X-ray

Detection cathethers in X-ray navigation

automatic 3D-cardiac mapping from bi-plane.

4343

Efficient calculation of G-convolutions

• Using steerable filters in the orientation score + inspired by Fourier transform on SE(2)

Algorithmic complexity can be reduced from

to