High resolution transmission imaging without lenses

Journal of Physics Conference Series

OPEN ACCESS

High resolution transmission imaging withoutlensesTo cite this article J M Rodenburg et al 2010 J Phys Conf Ser 241 012003

View the article online for updates and enhancements

You may also likePARTIAL ADAPTIVE COMPENSATIONAND PASSIVE INTERFEROMETRYWITH LARGE GROUND-BASEDTELESCOPESTadashi Nakajima and Christopher AHaniff

-

Selected papers from Photon 02Julian Jones

-

Guest Editors IntroductionDr J Baruchel

-

This content was downloaded from IP address 93721439 on 31012022 at 2027

High resolution transmission imaging without lenses

J M Rodenburg12

A C Hurst1 and A Maiden

12

1Department of Electronic and Electrical Engineering University of Sheffield Sir

Frederick Mappin Building Mappin Street Sheffield S1 3JD UK 2Phase Focus Ltd The Kroto Innovation Centre University of Sheffield North

Campus Broad Lane Sheffield S3 7HQ UK

Email JMRodenburgshefacuk

Abstract The whole history of transmission imaging has been dominated by the lens whether

used in visible-light optics electron optics or X-ray optics Lenses can be thought of as a very

efficient method of processing a wave front scattered from an object into an image of that

object An alternative approach is to undertake this image-formation process using a

computational technique The crudest scattering experiment is to simply record the intensity of

a diffraction pattern Recent progress in so-called diffractive imaging has shown that it is

possible to recover the phase of a scattered wavefield from its diffraction pattern alone as long

as the object (or the illumination on the object) is of finite extent In this paper we present

results from a very efficient phase retrieval method which can image infinitely large fields of

view It may have important applications in improving resolution in electron microscopy or at

least allowing low specification microscopes to achieve resolution comparable to state-of-the-

art machines

1 Ptychographic iterative phase retrieval

Consider the optical arrangement shown in Figure 1a A coherent source of radiation illuminates

an aperture Some distance downstream of the aperture there is an object of interest which can be

shifted laterally with respect to the illumination A long way downstream of this is a detector that can

measure the intensity of the Fraunhofer diffraction pattern We have demonstrated that it is possible to

obtain a complex representation of the object (that is an object that can be regarded as a transmission

function which introduces a phase change and an absorption term into the incident wave-front) even

with such a simple experimental set-up in principle only limited in resolution by the wavelength of

the radiation used [1] The only requirement on the dimensions of the experiment is that the diameter

of the illuminating wavefield is sufficiently small so that the Nyquist sampling condition is satisfied in

the diffraction plane If the object was completely disordered then this would imply that the speckle

pattern so produced must be sampled at an angular resolution of half the speckle size

To form an image from diffraction data alone we are faced with what may at first appear to be an

intractable problem To use only this intensity data we must have some means of recording or

computing the phase of the scattered radiation Only then will we be able to back propagate the

detected wave to the plane of the object thus creating an image of it The great strength of a lens is

that it preserves phase retarding the progress of the wave-fronts (their phase) in a pattern that reverses

the curvature of the wave-front so that any point in the image is brought to a focus in the image plane

The history of the so-called phase problem (solving for phase from intensity alone) is long Suffice it

Electron Microscopy and Analysis Group Conference 2009 (EMAG 2009) IOP PublishingJournal of Physics Conference Series 241 (2010) 012003 doi1010881742-65962411012003

ccopy 2010 Published under licence by IOP Publishing Ltd 1

to say that it has been known since about the early 1980s that if we have a prior knowledge that our

object conforms to certain constraints such as having finite size then it is possible to construct an

iterative procedure to converge upon the phase of every pixel of a diffraction pattern A combination

of the principle of ptychography (wherein multiple diffraction patterns are recorded for many relative

positions of the object and illumination [2]) and iterative phase retrieval leads to an algorithm (which

we call PIE) that can solve for phase with incredible efficiency Indeed it turns out if only these

relative displacements are known accurately both the object and the illumination function can be

solved for [3] a variation of PIE (ePIE) can do this in real time as data is collected [4]

Why should we undertake microscopy in such a round-about way In the context of short (atomic-

sized) wavelength radiation (X-rays or high energy electrons) it is difficult to make lenses of large

numerical aperture For example in electron microscopy the diffraction pattern lying in the back

focal plane of the objective lens (ie the selected area diffraction pattern) displays significant intensity

up to very high angles of scatter In contrast the diffractogram is much narrower The interferometer

in diffraction is the atoms within the object itself whereas conventional imaging relies on the

interference of beams that have propagated through macroscopic distances (millimeterscentimeters)

Figure 1 Three configurations for ptychography and lens-assisted pytchography with coherent

radiation incident from the left a) A simple aperture a specimen downstream of this and a detector

in the far-field b) A convergent probe of high numerical aperture defocused at the specimen plane to

create a patch of illumination c) A narrow-beam convergent probe with the specimen substantially

out of the Gaussian focal plane to create a large patch of illumination this is the closest analogy to (a)

making the most use of high-angle diffracted data

2 Visible light ptychography

We start by considering visible light ptychography Unlike X-ray and electron sources laser light is to

a very good approximation totally coherent The scale of the required accuracy of the relative

displacement between the specimen and the illuminating beam is also relatively large (of the order of

Type I

Type II

Type III

a

b

c


2

05m ) and can be easily achieved by commercially-available optical microscope specimen stages

Furthermore the use of a simple aperture arrangement (as in Figure 1a) is practicable If the detector

is a CCD camera with 1kx1k pixels then to achieve sub-micron resolution while still satisfying the

Nyquist sampling condition in the diffraction plane the aperture size can be as large as 05mm This

means that scanning over a large field of view can be achieved in a relatively short time Just like a

conventional lens the resolution of the final reconstruction is determined by the effective numerical

aperture of the scattered wave presented to the detector whether it is a lens or a CCD camera In

practice a flat detector limits the total solid angle of the scattered radiation that can be processed

especially if we require it to be far enough downstream of the object so that it lies in the Fraunhofer

diffraction plane There is no requirement in the PIE or ePIE algorithms on the nature of the

propagator between the object and the detector plane To obtain high resolution with a flat detector we

can place it in the Fresnel regime a short distance downstream of the object thus increasing the solid

angle it subtends at the specimen plane This angle can also be expanded by employing a large poor

quality lens downstream of the object which can bring the Fraunhofer diffraction pattern to a focus in

its back focal plane Aberrations in such a lens manifest themselves only as geometrical distortions in

the diffraction pattern and so can be easily removed computationally

Figure 2 shows a typical image that we can obtain using the ePIE algorithm Its quality is as good

as what we would expect from a lens although of course we have the added benefit of obtaining both

phase and modulus of the specimen transmission function Resolution in this micrograph is limited by

the solid angle of the detector as described above at about 2 m

Figure 2 Visible light ptychograph of lily eggs reconstructed using ePIE The scale bar

is 100 m Note that the dynamic range of the phase has been considerably expanded

Some principal benefits of visible-light ptychography are

1) Very large working distance Because there is no need to have an objective lens close up against

the specimen reasonable resolution can be obtained with the detector mounted centimeters away from

the object This is particularly useful in the context of imaging through enclosed containers such as

Petri dishes say for the purpose of imaging the growth cycle of live cells

2) Post collection focus swim-through because the entire wave-field is reconstructed in modulus

and phase it is possible to produce images at any defocus required after the data collection This is

particularly useful in the context of focusing in on biological features after undertaking a fast

throughput experiment where it may not be practicable to focus accurately on the object of interest in

real-time


3

3) Absolute phase measurement Unlike Zernike contrast which distorts the image of the edges of

objects the absolute phase can be used to quantitatively measure the optical thickness of objects over

a wide field of view This in turn leads to the ability to perform accurate partitioning of the image say

for the purpose of automatically counting cells or similar features The technique can also measure

very large phase changes (many multiples of 2 ) say for characterization of for example contact

lenses

3 Hard X-ray ptychography

The first demonstration of iterative ptychography at short wavelengths was at the Swiss Light Source

using 8keV X-rays to image a zone plate [5] (see Figure 3) The type I configuration (Figure 1a) was

used with an aperture size of 5 m The first reconstructions obtained (Figure 3 top ndash from [5]) used

an estimate of the illumination function calculated by propagating a wave-field coming from the

aperture to the object The distance of propagation could only be measured approximately and was

about 11mm Because the quality of a PIE reconstruction hinges on how well the illumination

function is known different propagation distances were investigated in order to minimize a least-

squares error metric between the calculated and measured diffraction pattern intensities If we use

ePIE (ie solving for both the object function and the illumination function simultaneously) ndash see

centre row of Figure 3 ndash artefacts in the reconstruction are significantly reduced especially in the

modulus reconstruction The phase reconstruction is also smoother although at the cost of slightly

reduced resolution The illumination function calculated this way is also somewhat smoother than that

estimated assuming the aperture is illuminated by perfectly coherent waves In trying to find a general

solution for both functions simultaneously it would seem that the ePIE algorithm settles on a

smoothed average that has least inconsistency

4 Electron ptychography

Although iterative ptychography works spectacularly well for photons the implementation of

electron ptychography at least at the time of writing has proved to be much more difficult we have

so far not obtained any high-resolution publishable results The simplest optical geometry (shown in

Figure 1a) is impractical for electrons Manufacturing an aperture which is both small enough at its

centre and opaque enough around its perimeter is extraordinarily challenging To obtain sub-

Aringngstrom resolution using a 1kx1k detector the aperture diameter must be of the order of

100Aringngstoms To avoid transmission of substantial intensity outside the aperture the thickness of the

aperture blade would have to be of the order of microns leading to an aspect ratio of about 1000 It

may be possible to use a much thinner blade if we were to filter out electrons which have passed

outside the aperture and consequently lost energy but this would still be a very difficult structure to

make

The only practicable optical set up is to use a focusing lens to form the illumination function on the

specimen In other words we still require a lens albeit one that can have relatively poor intrinsic

resolution If this lens is strong and produces a beam with a high angle of convergence we classify it

as the type II configuration (Figure 1b) On the contrary the type III configuration (Figure 1c) uses a

small angle of convergence It should be emphasized the PIE or ePIE algorithms are indifferent to

form of illumination ndash it can be soft-edged consist of curved wavefronts or have any range of solid

angle of incident k-vectors However the form of the illumination can be optimized with respect to

counting statistics the coherence of the source and other experimental constraints such as drift rates

and specimen damage Moreover the best arrangement will depend on the existing specification of

the microscope upon which we intend to implement the method


4

Some relative advantagesdisadvantages of these different configurations are as follows

Type II Counting statistics within the central disc (which we call the lsquobright-fieldrsquo disc) are much

higher provided the specimen is relatively weakly scattering any diffracted amplitude that gets

scattered to within the central disc interferes with a pseudo-reference beam (the unscattered wave) If

the object consists of large areas of empty space this configuration is essentially equivalent to Gabor

holography If we write the amplitude of the unscattered wave as Ar and the amplitude of the scattered

Figure 3 Top modulus (left) and

phase (right) of PIE reconstruction

using 8keV X-rays of a zone plate

surrounded by gold particles ndash for

this we model the illumination

function by propagating the wave

field from the aperture (Fig 1a) to

object Middle modulus (left) and

phase (right) of ePIE ndash we solve for

both the object function and the

illumination function

Bottom high resolution SEM

image of object

10 m


5

wave as As then the intensity measured is22

ssrsrr AAAAAA where the cross terms are

much larger than the intensity of the scattered term Related to this is the fact is the further advantage

that if we only aim to process the bright-field disc the intensities we need to measure all have a

similar dynamic range To record diffracted data outside the central disc (the dark-field data) requires

a detector with a high dynamic range or at least a detector with two regions (inside and outside the

disc) which have different gains Ptychographical information (that is relative phase information

between scattered diffraction orders) is also very strongly expressed because each point in the

diffraction pattern of the object is convolved with a function which is the diameter of the (large)

central disc The most obvious disadvantage is that if it is our intention to improve the resolution of

the microscope via ptychography this configuration requires us to start with a relatively good (or at

least stable) lens The demands on the coherence of the beam are also high because the back-focal

plane of the illumination-forming lens must be smaller than the coherence width at that plane

However if we can handle the dynamic range issue of the dark-field intensity the gains in resolution

could be several factors better than is currently achievable even on aberration-corrected machines

Type III On the contrary the type III configuration can use a poor lens If we process the data

scattered outside the central disc up to large scattering angles then we really can greatly improve upon

the intrinsic resolution of the lens by many factors In the limit of near-parallel illumination we can

place a stop over the (very small) central disc losing only very low resolution components in the

image and increase the gain of the detector so that all the dark-field data can be collected within a

similar dynamic range This configuration promises to provide a method of upgrading a microscope

with rather modest resolution to one that has very high resolution

So what are obstacles to high-resolution electron ptychography At the time of writing the most

powerful algorithm we have developed to data (ePIE) can solve for both the object function and the

illumination but only if the positions of their relative shifts are known accurately Even on

microscopes that can generate very precise scans for STEM images we have found that when large

movements (step sizes) are made the position of the beam is not accurately reproducible This is

exacerbated by specimen drift although we are developing strategies to retrospectively calculate the

probe illumination positions (see Hurst et al this volume) Furthermore ePIE relies on the

multiplicative approximation ndash that is that the exit wave is the product of the illumination function and

the specimen transmission function The influence of thickness and dynamical effects are not serious if

the illumination function is known [6] but they do limit the usefulness of ePIE However the most

challenging problem we face at present is that when we examine in detail the dark-field diffraction

(where the high-resolution data resides) we find that the intensity distribution is often not stable This

could be due to a number of factors instability in the probe instability or damage in the specimen

instability in the source or the column itself or the migration of contamination over the specimen

surface These factors do not seriously impact on conventional imaging but must be addressed

carefully before electron ptychography will deliver images similar in quality to those we can now

routinely obtain with photons

References

[1] Rodenburg J M Hurst A C and Cullis A G 2007 Ultramicroscopy 107 227

[2] Rodenburg J M Advances in Imaging and Electron Physics 2008 150 87

[3] Thibault P Dierolf M Menzel A Bunk O David C Pfeiffer F 2008 Science 321 379

[4] Maiden A M and Rodenburg J M 2009 Ultramicroscopy in press

[5] Rodenburg J M Hurst A C Cullis A G Dobson B R Pfeiffer F Bunk O David C Jefimovs K

and Johnson I 2007 PRL 98 art no 034801

[6] Cheng Lui Walther T and Rodenburg J M 2009 Ultramicroscopy in press


6

High resolution transmission imaging without lenses

J M Rodenburg12

A C Hurst1 and A Maiden

12

1Department of Electronic and Electrical Engineering University of Sheffield Sir

Frederick Mappin Building Mappin Street Sheffield S1 3JD UK 2Phase Focus Ltd The Kroto Innovation Centre University of Sheffield North

Campus Broad Lane Sheffield S3 7HQ UK

Email JMRodenburgshefacuk

Abstract The whole history of transmission imaging has been dominated by the lens whether

used in visible-light optics electron optics or X-ray optics Lenses can be thought of as a very

efficient method of processing a wave front scattered from an object into an image of that

object An alternative approach is to undertake this image-formation process using a

computational technique The crudest scattering experiment is to simply record the intensity of

a diffraction pattern Recent progress in so-called diffractive imaging has shown that it is

possible to recover the phase of a scattered wavefield from its diffraction pattern alone as long

as the object (or the illumination on the object) is of finite extent In this paper we present

results from a very efficient phase retrieval method which can image infinitely large fields of

view It may have important applications in improving resolution in electron microscopy or at

least allowing low specification microscopes to achieve resolution comparable to state-of-the-

art machines

1 Ptychographic iterative phase retrieval

Consider the optical arrangement shown in Figure 1a A coherent source of radiation illuminates

an aperture Some distance downstream of the aperture there is an object of interest which can be

shifted laterally with respect to the illumination A long way downstream of this is a detector that can

measure the intensity of the Fraunhofer diffraction pattern We have demonstrated that it is possible to

obtain a complex representation of the object (that is an object that can be regarded as a transmission

function which introduces a phase change and an absorption term into the incident wave-front) even

with such a simple experimental set-up in principle only limited in resolution by the wavelength of

the radiation used [1] The only requirement on the dimensions of the experiment is that the diameter

of the illuminating wavefield is sufficiently small so that the Nyquist sampling condition is satisfied in

the diffraction plane If the object was completely disordered then this would imply that the speckle

pattern so produced must be sampled at an angular resolution of half the speckle size

To form an image from diffraction data alone we are faced with what may at first appear to be an

intractable problem To use only this intensity data we must have some means of recording or

computing the phase of the scattered radiation Only then will we be able to back propagate the

detected wave to the plane of the object thus creating an image of it The great strength of a lens is

that it preserves phase retarding the progress of the wave-fronts (their phase) in a pattern that reverses

the curvature of the wave-front so that any point in the image is brought to a focus in the image plane

The history of the so-called phase problem (solving for phase from intensity alone) is long Suffice it


ccopy 2010 Published under licence by IOP Publishing Ltd 1


























Type I

Type II

Type III

a

b

c


2
































real-time


3






lenses




























make












4



















image of object

10 m


5








































References









6


























Type I

Type II

Type III

a

b

c


2
































real-time


3






lenses




























make












4



















image of object

10 m


5








































References









6
































real-time


3






lenses




























make












4



















image of object

10 m


5








































References









6






lenses




























make












4



















image of object

10 m


5








































References









6



















image of object

10 m


5








































References









6








































References









6

Documents

High resolution transmission imaging without lenses