Biology Imaging

PT-BIOP

Microscopy course IIFebruary 15-16-17, 2010

Deconvolution

Biology Imaging

CONVOLUTION

DECONVOLUTION

imageobject

Diffraction-limited systemsA microscope is a diffraction-limited system

Diffraction : spreading out of waves past small opening

Optical blur is a consequence of light diffraction through the optical system (and particularly through the objective), resulting in the limitation of optical resolution.

Point Spread Function (PSF) :system response to a punctual light source

Biology Imaging

PSF and OTF

))(()( xhH ℑ=ω

)()( ωω HMTF =

The Optical Transfer Function

(OTF) is the

Fourier transform of the PSF and describes how the optical system behaves in the frequency domain.

The module of the OTF is called Modulation Transfer Function (MTF) and describes how the amplitudes of different frequency components are modified throughout the optical system.

From the frequency domain point of view, a microscope behaves as a low-pass system.

with )(ωH : OTF

)(xh : PSF

Biology Imaging

PSF and OTF

optical system)(xf )(xg

)(xh)()()( xhxfxg ∗=

ℑ)()()( ωωω HFG ⋅=

The convolution operation corresponds to a simple multiplication in the frequency domain.

Many deconvolution algorithms are implemented in the frequency domain.

Simple deconvolution methodInverse Filtering

(the PSF/OTF can be estimated theoretically from the microscope parameters)

)()()(

ωωω

HGF = ))(()( 1 ωFxf −ℑ=

(without considering noise!)

)(ωH

)(/1 ωH

Biology Imaging

Deconvolution techniques1)

Linear methods (filtering)

2)

Iterative algorithms3)

(Deblurring methods)4)

Blind deconvolution

Linear methods

inverse filtering

Wiener filtering

Wiener filtering

During division in Fourier space, noise variations are significantly amplified.

Noise amplification can be reduced by making some assumptions about your image. For instance we can assume that the object was relatively smooth and

impose some constraint on the estimated solution (deconvolved image). This approach is called regularization.

)()()(

)()(2

Wiener

ωωω

ωω

g

nH

HF

ΦΦ

+=

∗

)(ωnΦ

)(ωgΦwith power spectrum of the signal

power spectrum of the noise

regularization term

Biology Imaging

Deconvolution techniques1)

Linear methods (filtering)

2)

Iterative algorithms3)

(Deblurring methods)4)

Blind deconvolution

Linear methods

+ fast

- limited by noise amplification

- possible ringing

Original image

Deconvolved image

Ringing artefactOriginal image Blur (noise free)

Blur + additive noise

Inverse filtering

Inverse filtering

Biology Imaging

Deconvolution techniques1)

Linear methods (filtering)

2)

Iterative algorithms3)

(Deblurring methods)4)

Blind deconvolution

Iterative algorithmsIdea: an estimate of the object is made (typically the raw image). The first estimate of the object is convolved by the PSF to produce a new estimate of the result. The two estimates are compared and an error criterion is established. The error criterion is used to modify the last estimate and produce a new estimate, and so on, iteratively, up to convergence. Mathematical formulation: iterative minimization of a defined cost function

Least squares estimation

2

~2

~~min ~min fHggg

ff−=−

Minimization of the error between the acquired image and the convolution of the estimated object by the PSF.

ITERATIVE IMPLEMENTATIONLandweber

(an example between others)

Imaging

system

f g0 =H ⋅ f

noise

g = H ⋅ f + n

˜ f ˜ g = H ⋅ ˜ f

LS algorithm˜ f

)( )()()1( kLS

TkLS

kLS fHgHff ⋅−+=+ γ

previous step solution correction through

the error criterion

Biology Imaging

Deconvolution techniques1)

Linear methods (filtering)

2)

Iterative algorithms3)

(Deblurring methods)4)

Blind deconvolution

Maximum likelihood estimationDifferent formulation of the cost function to be minimized:

)))~|(log((min ~ ff

gp−

Iterative formulations are made by doing some assumptions about the kind of noise which affects the acquired image.

Iterative algorithms

+ give better results than filtering methods

+ noise amplification well controlled (setting of a regularization parameter)

-> the iteration goes on up the cost function is minimized or a certain number of iterations is reached - computationally expensive

Biology Imaging

Deconvolution techniques1)

Linear methods (filtering)2)

Iterative algorithms

3)

(Deblurring methods)4)

Blind deconvolution

Deblurring methods

Blind deconvolution

Unsharp

masking: a blurred version of the 2D image is subtracted by the image itself.

The blurred image can also be obtained from the upper and lower planes respect to the considered one (nearest-neighbour).

This technique is not really a deconvolution as it does not use any information about the acquisition system. However it can be useful in case of thin stacks/single plane images.

Blind deconvolution is a more recent method.

In this approach, an estimate of the object is made. This estimate is convolved with a theoretical PSF. The resulting blurred estimate is compared with

the raw image, then a correction is computed and used to generate a new estimation. This same correction is also applied to the PSF, generating a PSF estimate. The process is iterative.

+ good results also on noisy or spherically aberrated

images

Biology Imaging

Deconvolution techniques

A comparison

Confocal images of fixed epithelial cell labeled

with concanavalin

A and FITC

Biology Imaging

PSF wideningThe PSF is the basic brick of deconvolution process

How the PSF and the optical resolution are linked?PSF and microscope optical resolution are inevitably linked.

The PSF width (FWHM) in the radial and axial directions gives the optical system resolution, and vice versa.

Rayleigh criterion

Two Airy disks (points) are resolved if they are farther apart than the distance at which the maximum of one Airy disk coincides with the first minimum of the second Airy disk.

As deconvolution is performed at the PSF scale, it is necessary that the image is acquired with a correct voxel size, that is, with a voxel size coherent with the optical resolution of the microscope.

According to the Nyquist

theorem: resolutionopticalstepsampling _21_ ⋅=

Note

anyway that the effective optical resolution of the real microscope is worst than the theoretical one (lens imperfections, misalignment).

Biology Imaging

PSF wideningWhich elements influence the PSF?

the objective NA

NA = 0.8 NA = 1.4

Biology Imaging

PSF wideningWhich elements influence the PSF?

the wavelength

the pinhole aperture

Biology Imaging

PSF wideningWhich elements influence the PSF?

the refractive indexes of the objective medium and of the sample mounting medium

other factors

Optical problem Vibrations

PSF with and without refractive index mismatch, at a depth of 10 μm

Biology Imaging

Theoretical vs measured PSFPractically, the effective PSF can significantly differ from the

theoretical one

Theoretical PSF

Measured PSF (λ=488 nm)Confocal microscope, Zeiss LSM700

Measured PSF (λ=561 nm)Confocal microscope, Zeiss LSM700

Biology Imaging

Theoretical vs measured PSFConcerning deconvolution, a possibility is to use an

experimental PSF instead of a theoretical PSF

How to obtain an experimental PSF:

1.

record 3D stacks of fluorescent beads (suggestion: use beads with a diameter of 100-200 nm), with the same microscope parameter and in the same condition you acquired the images you want to deconvolve

2.

register and average the beads images (this step is useful to reduce noise)

3.

‘distill’

the PSF using the deconvolution

principle

Biology Imaging

Deconvolution in practiceDeconvolution effects

deconvolution improves image resolution, particularly in the axial dimension

improves image contrast

improve image SNR

For these reasons deconvolve your images is a good

practice, particularly when you want to segment objects

or do colocalization analysis, besides getting nicer and cleaner images.

Deconvolution applicability

widefield microscopy

confocal microscopy

spinning disk microscopy

electron microscopy

Radial intensity profiles extracted from original and deconvolved

(various deconvolution software) widefield images of a green

fluorescent bead, diameter 2.5 μm

Widefield images of C. elegans embryo. Transversal sections from the acquired image (black profile),

and from deconvolution results obtained with different software.

Biology Imaging

Deconvolution tips

Tipsacquire your images respecting the Nyquist

criterion

do not saturate your images

if possible, acquire some black slices up and down the object of interest to avoid border effects

repeat the deconvolution with different settings of the algorithm parameters (particularly the regularization parameter) and compare your results

Deconvolution is a delicate, sensitive image processing technique.

Deconvolution must be applied only on images correctly acquired and the result needs always to be verified.

Widefield z-stack of C. elegans embryo. Comparison between the original and the deconvolved images.

Biology Imaging

Around deconvolution

Note

background correction (pre-processing)

bleaching correction (pre-processing)

spherical aberration correction (dynamic PSF)

Deconvolution packages usually implement a series of pre-processing tools which improve deconvolution results:

Widefield z-stack of melanocytes. Comparison between the original and the deconvolved images.

Biology Imaging

Deconvolution at the BIOpSoftware

-DeconvolutionLab:

A free-ware ImageJ

plugin which implements different iterative algorithms and inverse filtering solutions

-SVI Huygens and HRM (Huygens Remote Manager)A commercial software which implements various deconvolution algorithms (particularly, maximum likelihood estimation) and pre-processing steps.

You cans ask for an HRM account at:

http://svitsrv7.epfl.ch/hrm/

http://bigwww.epfl.ch/algorithms/deconvolutionlab/

Biology Imaging

Some references

-

Sibarita

J.-B., Deconvolution Microscopy. Advances In Biochemical Engineering/Biotechnology, Springer Berlin, 201-243 (2005)

-

Biggs D.S.C., Clearing Up Deconvolution. Biophotonics International 11, 32-36 (2004)

-

Sarder

P., et al., Deconvolution Methods

for 3-D Fluorescence Microscopy

Images. IEEE Signal Processing

Magazine, 23, 32-45 (2006)

- http://www.olympusmicro.com/

- http://support.svi.nl/wiki/

Contacts:

http://biop.epfl.ch/

Alessandra Griffa, alessandra.griffa@epfl.ch