Embed Size (px)
Citation preview
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
DECONVOLUTION
PT BIOP COURSE 2012
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
• Image formation - diffraction-limited systems - noise
• Simple deconvolution approaches - inverse filter - Wiener filter
• Iterative deconvolution - least square estimation - regularized least square estimation - maximum likelihood estimation
• Other methods - deblurring methods - blind deconvolution
• PSF widening
• Deconvolution tips
OUTLINE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
object object’s image
Sources of degradation observed during image formation:
• optical blur (light diffraction through the optical system)
• noise (statistical distribution of photons and electronic
devices)
IMAGE FORMATION
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Light diffraction in a microscope
constructive
interference
destructive
interference
DIFFRACTION-LIMITED SYSTEMS
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Light diffraction in a microscope
Diffraction pattern of an ideal, aberration-free objective, in one, two and three dimensions.
JB Sibarita - Microscopy Techniques, 2005 - Springer
DIFFRACTION-LIMITED SYSTEMS
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
object object’s image
Sources of degradation observed during image formation:
• optical blur (light diffraction through the optical system)
• noise (statistical distribution of photons and electronic devices)
IMAGE FORMATION
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Sources of noise:
1. shot noise
2. detection device noise
the quantum nature of light
results in the detected photons
following a Poisson distri-
bution
• dark current
• read noise
Gaussian distribution
NOISE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
CONVOLUTION
DECONVOLUTION
image object
Deconvolution is a mathematical process whose objective is to restore an image
unavoidably distorted by optical blur and noise during the acquisition process.
Point Spread Function (PSF) :
system response
to a punctual light source
Deconvolution is based on prior knowledge of the system point spread function and
noise statistics.
SIMPLE DECONVOLUTION
APPROACHES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
image object
CONVOLUTION
DECONVOLUTION
optical system )(xf )(xg
)(xh
Schematic representation of the
image formation model
(without considering noise!)
Recall the definition of convolution:
xhxfxg *
dudvdwwzvyuxhwvuf ,,,,
The convolution operation corresponds to a simple multiplication in the frequency domain!
The majority of the deconvolution algorithms are implemented in the frequency domain.
)()()( xhxfxg )()()( HFG
SIMPLE DECONVOLUTION
APPROACHES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
))(()( xhH
)()( HMTF
The Optical Transfer Function (OTF) is the Fourier
transform of the PSF and describes how the
optical system behaves in the frequency domain.
The module of the OTF is called Modulation
Transfer Function (MTF) and describes how the
amplitudes of different frequency components are
modified through the optical system.
From the frequency domain point of view, a
microscope behaves as a low-pass system.
with )(H : OTF
)(xh : PSF
PSF AND OTF
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
)()()( HFG
Simple deconvolution method Inverse Filtering
)(
)()(
~
H
GFinv ))(()( 1 Fxf
)(H
)(/1 H
Limitations:
• noise amplification
• inverse filter may be unstable stabilized inverse filter
0
0,1
)(
H
HRstab
otherwise
)()()(~
GRFinv
INVERSE FILTERING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
optical system )(xf
)(0 xg)(xh)(xg
noise
)()(
1)(
~
G
HF
)()()(
10
NG
H
)(
)()(
~
H
NFinv
The image formation process is also affected by noise:
Inverse Filtering in presence of non
negligible noise
noise amplification
INVERSE FILTERING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Wiener filtering
)(
)()(
)(
)(
1
)(
)()(
)()(
2
2
2
g
n
g
n H
H
HH
HRWiener
)(n
)(gwith power spectrum of the signal
power spectrum of the noise
)()()(~
GRF
The Wiener filter requires us to know the spectral content of a typical image, and also that of the noise (estimated).
WIENER FILTER
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
1) Linear methods (filtering) 2) Iterative algorithms
3) (Deblurring methods)
4) Blind deconvolution
Linear methods
+ fast
- limited by noise amplification
- possible ringing
Original image Blur (noise free)
Blur + additive noise
Inverse filtering (regularized version)
Original image
Deconvolved image
Ringing artefact
DECONVOLUTION TECHNIQUES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
An alternative image formation representation: matrix formulation
Imaging
system
f
g0 H f
noise
g H f n
1,1(
)1,0(
)0,1(
)0,1(
)0,0()0,1()0,0(
1,1(
)1,0(
)0,1(
)0,1(
)0,0(
LKf
f
Kf
f
fhh
NMg
g
Mg
g
g
Measurement vector g Vector of unknown f System matrix H
MATRIX FORMULATION
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
1) Linear methods (filtering)
2) Iterative algorithms 3) (Deblurring methods)
4) Blind deconvolution
Iterative algorithms
Iterative methods solve iteratively the minimization of a defined cost function.
Least squares
2~,
~LSJ
In practice minimization problem is difficult to solve analytically.
An iterative solution is necessary.
TT
LS
1~
cost function
analytical solution
ITERATIVE ALGORITHMS
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Iterative algorithms
In the case of the least square criterion:
)~
(),~
(Correction kTkfHgHgf
So:
)~
(~~ 1 kTkk
fHgHff
),~
(Correction~~ 1
gfffkkk
relaxation parameter (it influences the convergence speed)
previous step solution
correction through the error criterion
(Landweber algorithm)
Iterative update strategy:
Correction factor, steepest descent minimization algorithm:
g)fJ(gf ,~
),~
(Correction kk
cost function gradient
LEAST SQUARE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP Iterative algorithms
Least square deconvolution suffers from noise amplification.
However the advantage of the iterative implementation is that we can stop the algorithm before the noise amplification dominates.
Regularized least square minimization
22 ~~,
~fLff LSRJNew cost function:
regularization parameter
REGULARIZATION TERM
high-pass filter
Iterative implementation: kTTTkkfLLHHgHff~~~ 1
(Tikhonov-Miller algorithm)
LEAST SQUARE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
The regularization parameter λ control the amount of
correction for noise amplification and must be carefully
tuned accordingly to the quality of the acquired image.
22 ~~,
~fLff LSRJ
regularization parameter
Degraded image:
Gaussian blur + additive noise
not enough: =0.02 not enough: =0.2
too much: =20 Optimal regularization: =2 too much: =200
LEAST SQUARE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Maximum likelihood
Different formulation of the cost function to be minimized:
))~
|(log(,~
fggf pJML
In the case of Poisson noise hypothesis, the Maximum Likelihood cost function can be written as:
The idea is to maximize the probability of observing the image g if the initial image is f. By minimizing JML, this probability will be maximized.
To express the cost function, some assumption about the signal and the noise distribution need to be done.
fHgfH1gf~
ln~
,~ TT
MLJ
which leads to the following iterative solution:
k
k
Tkf
fH
gHf
~~
~ 1
(Richardson-Lucy algorithm)
MAXIMUM LIKELIHOOD
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
1) Linear methods (filtering)
2) Iterative algorithms 3) (Deblurring methods)
4) Blind deconvolution
Iterative algorithms
+ give better results than filtering methods
+ noise amplification is well controlled (setting of a regularization parameter)
+ it is possible to insert positivity constraints
(least square, regularized least square, maximum likelihood…)
-> the number of iterations and
the regularization parameter
need to be set carefully!
DECONVOLUTION TECHNIQUES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
1) Linear methods (filtering)
2) Iterative algorithms
3) (Deblurring methods) 4) Blind deconvolution
Deblurring methods
Unsharp masking: a blurred version of the 2D image is subtracted by the image itself.
The blurred image can also be obtained from the upper and lower plane respect to the considered one (nearest-neighbor).
lower SNR respect to the restored (iterative constrained algorithm) image
DECONVOLUTION TECHNIQUES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
1) Linear methods (filtering)
2) Iterative algorithms
3) (Deblurring methods)
4) Blind deconvolution
Blind deconvolution Blind deconvolution is a more recent method.
In this approach, an estimate of the object is made. This estimate is convolved with a theoretical PSF. The resulting blurred estimate is compared with the raw image, a correction is computed and used to generate a new estimation. This same correction is also applied to the PSF, generating a PSF estimate. The process is iterative.
original image
Wiener filter
nearest neighbor
blind deconvolution
DECONVOLUTION TECHNIQUES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
A comparison
Confocal images of fixed epithelial cell labeled with
concanavalin A and FITC
DECONVOLUTION TECHNIQUES
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
The PSF is the basic brick of deconvolution processing
How the PSF and the optical resolution are linked?
PSF and microscope optical resolution are inevitably linked.
The PSF width (FWHM) in the radial and axial directions gives the optical system resolution and vice versa.
Rayleigh criterion
Two Airy disks (points) are resolved if they are farther apart than the distance at which the maximum of one Airy disk coincides with the first minimum of the second Airy disk.
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Which elements influence the PSF?
the objective NA
NA = 0.8 NA = 1.4
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Which elements influence the PSF?
the wavelength
the pinhole aperture
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
As deconvolution is performed at the PSF scale, it is necessary that the image is acquired with a correct voxel size, that is, with a voxel size coherent with the optical resolution of the microscope.
According to the Nyquist theorem: resolutionopticalstepsampling _2
1_
An example: wide field microscopy, 1.4 oil objective, 520 nm emission wavelength
nmNA
rlateral 22661.0
nmr MAXmateral 113,
Undersampling should be avoided
Oversampling can increase photobleaching and phototoxicity problems, besides increasing acquisition time and storage needs
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Which elements influence the PSF?
the refractive indexes of the objective medium and the sample mounting medium
other factors
Optical problem Vibrations
PSF with and without refractive index mismatch, at a depth of 10 μm
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Which elements influence the PSF?
When spherical aberration occurs, the PSF is not anymore spatial-invariant, but depends on the focus depth into the sample. An example: confocal microscopy, 1.3 oil objective with watery medium, 520 nm emission wavelength
acquisition depths: 0 μm 5 μm 10 μm 15 μm 20 μm 25 μm
Spherical aberration can be partially corrected on the deconvolution side by using a dynamic PSF.
PSF WIDENING
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Practically, the effective PSF can significantly differs from the theoretical one
Theoretical PSF
Measured PSF (λ=488 nm) Confocal microscope, Zeiss LSM700
Measured PSF (λ=561 nm) Confocal microscope, Zeiss LSM700
THEORETICAL VS MEASURED PSF
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
How to obtain an experimental PSF?
1. record 3D stacks of sub-resolution fluorescent beads, with the same microscopy parameters and in the same conditions you acquired the images you want to deconvolve
2. register and average the beads images to improve their SNR
3. derive the experimental PSF using the deconvolution principle
Theoretical PSF Experimental PSF
• noise-free
• does not require additional microscopy acquisitions
• does not take into account specific optical system peculiarities
• takes into account specific optical system peculiarities
• partially corrects for spherical aberration
• needs to be extracted from beads images
THEORETICAL VS MEASURED PSF
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Deconvolution effects
deconvolution improves image resolution, particularly in the axial direction
improves image contrast
improves image SNR
For these reasons it is a good practice to deconvolve your images, particularly when you want for example to segment objects or do colocalization analysis, besides getting nicer and cleaner images.
Deconvolution applicability
widefield microscopy
confocal microscopy
spinning disk microscopy
2P microscopy
DECONVOLUTION IN PRACTICE
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Tips
acquire your images respecting the Nyquist criterion
do not saturate your images
if possible, acquire some black slices up and down the object of interest to avoid border effect
repeat the deconvolution with different settings of the algorithm parameters (particularly the regularization parameter) and compare your results
Deconvolution is a delicate, sensitive image processing technique.
It must be applied only on images correctly acquired and the result needs always to be verified.
DECONVOLUTION TIPS
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP Note
background correction (pre-processing)
bleaching correction (pre-processing)
Deconvolution packages usually implement a series of pre-processing tools which improve deconvolution results:
DECONVOLUTION TIPS
BIOIMAGING AND
OPTICS PLATFORM
EPFL–SV–PTBIOP
Software for deconvolution
Specialized deconvolution packages
As a part of imaging software
Free-ware software (ImageJ plugins)
• AutoQuant AutoDeblur
• SVI Huygens (and HRM)
• Leica, Zeiss, Olympus, Nikon
• ‘Parallel Iterative Deconvolution’
• ‘Iterative Deconvolve 3D’
• ‘DeconvolutionLab’
DECONVOLUTION SOFTWARE
