12
Spatial light modulators in fluorescence microscopy Pavel Křížek 1,2 and Guy M. Hagen 1,2 1 Charles University in Prague, First Faculty of Medicine, Institute of Cellular Biology and Pathology, Prague, Czech Republic 2 Academy of Sciences of the Czech Republic v.v.i., Institute of Physiology, Department of Cell Biology, Prague, Czech Republic Spatial light modulators (SLMs) are becoming increasingly important in optical microscopy. These reflective or transmissive devices, based on microdisplays using liquid crystal or micromirror array technology, are used to achieve a variety of useful effects and imaging modes. When placed in an image plane, SLMs can be used to create arbitrary, computer controlled masks. These masks can then be placed in the microscope's illumination pathway, detection pathway, or both. The result is a programmable array microscope (PAM): an optical sectioning microscope with improved speed, sensitivity, and flexibility compared to conventional confocal laser scanning microscopes (CLSMs). Another possibility is structured illumination microscopy (SIM), in which SLMs are used only for illumination. Both high speed optical sectioning and live cell, 3D superresolution can also be achieved via SIM. When placed in an aperture plane, SLMs can manipulate the reflected wavefront. This allows for applications such as adaptive optics. Keywords programmable array microscope; adaptive optics; structured illumination microscopy 1. Introduction Since their introduction, spatial light modulators (SLMs) have quickly been adopted as versatile optical devices with many uses. Here we cover only a few applications in biological fluorescence microscopy, where spatial light modulators have been used to manipulate light in both the microscope's image (i.e., field, object) plane and aperture (i.e., pupil, back focal) planes. We note the extensive use of SLMs in other areas of microscopy, such as profilometry [1] and optical tweezers [2, 3], but we do not cover these applications here. Figure 1 reviews part of the optical system used in modern, infinity corrected microscopes. The critical concept for the current discussion is that the combination of the microscope objective and tube lens forms an image of the sample at a location outside the microscope body. Usually, a CCD camera is placed in this location for image acquisition. Placement of a lens L1 with focal length F, at a distance 1F away from this image will result in an image of the objective's aperture being formed at distance 1F on the other side of the lens. This is an image of the objective's back focal plane, i.e., the hole in the bottom of the objective. This is also called the aperture or pupil plane of the microscope. Placement of an additional lens L2 with focal length F' at a distance 1 F' away from the reimaged aperture plane will result in reformation of an image of the sample at a distance F' away, with magnification given by F / F'. Reconstruction of the aperture and/or the image planes of the microscope is necessary when building an optical system employing SLMs (see Figure 3 for an example). Readers are encouraged to consult the classic text by Innoué and Spring for an authoritative and complete treatment of Köhler illumination and imaging in optical microscopes [4]. Another excellent, and, for students, perhaps a more accessible review of the topic can be found in the text by Keller and Goldman [5]. Image Plane (primary) Aperture Plane (reconstructed) Aperture Plane (primary) Image Plane (reconstructed) FFF F F TL F TL F TL Tube Lens (inside microscope) Lens L1 Lens L2 objective Fig 1. Reconstruction of aperture and image planes in infinity corrected microscopes (not to scale). Microscopes from Nikon and Leica use tube lenses with a focal length of 200 mm, Zeiss 160 mm, and Olympus 180 mm. Lenses L1 and L2 should be achromatic doublets, and should have the same focal length as the tube lens for optimal performance. Vertical arrows indicate the locations of field and aperture stops (diaphragms). Spatial light modulators can be placed in the microscope's primary image plane, or in any of the reconstructed optical planes to manipulate light, at high speed and under computer control. There are many possible applications, including Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.) 1366 ©FORMATEX 2010 ______________________________________________

Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

Embed Size (px)

Citation preview

Page 1: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

Spatial light modulators in fluorescence microscopy

Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First Faculty of Medicine, Institute of Cellular Biology and Pathology, Prague, Czech

Republic 2 Academy of Sciences of the Czech Republic v.v.i., Institute of Physiology, Department of Cell Biology, Prague, Czech

Republic

Spatial light modulators (SLMs) are becoming increasingly important in optical microscopy. These reflective or transmissive devices, based on microdisplays using liquid crystal or micromirror array technology, are used to achieve a variety of useful effects and imaging modes. When placed in an image plane, SLMs can be used to create arbitrary, computer controlled masks. These masks can then be placed in the microscope's illumination pathway, detection pathway, or both. The result is a programmable array microscope (PAM): an optical sectioning microscope with improved speed, sensitivity, and flexibility compared to conventional confocal laser scanning microscopes (CLSMs). Another possibility is structured illumination microscopy (SIM), in which SLMs are used only for illumination. Both high speed optical sectioning and live cell, 3D superresolution can also be achieved via SIM. When placed in an aperture plane, SLMs can manipulate the reflected wavefront. This allows for applications such as adaptive optics.

Keywords programmable array microscope; adaptive optics; structured illumination microscopy

1. Introduction

Since their introduction, spatial light modulators (SLMs) have quickly been adopted as versatile optical devices with many uses. Here we cover only a few applications in biological fluorescence microscopy, where spatial light modulators have been used to manipulate light in both the microscope's image (i.e., field, object) plane and aperture (i.e., pupil, back focal) planes. We note the extensive use of SLMs in other areas of microscopy, such as profilometry [1] and optical tweezers [2, 3], but we do not cover these applications here. Figure 1 reviews part of the optical system used in modern, infinity corrected microscopes. The critical concept for the current discussion is that the combination of the microscope objective and tube lens forms an image of the sample at a location outside the microscope body. Usually, a CCD camera is placed in this location for image acquisition. Placement of a lens L1 with focal length F, at a distance 1F away from this image will result in an image of the objective's aperture being formed at distance 1F on the other side of the lens. This is an image of the objective's back focal plane, i.e., the hole in the bottom of the objective. This is also called the aperture or pupil plane of the microscope. Placement of an additional lens L2 with focal length F' at a distance 1 F' away from the reimaged aperture plane will result in reformation of an image of the sample at a distance F' away, with magnification given by F / F'. Reconstruction of the aperture and/or the image planes of the microscope is necessary when building an optical system employing SLMs (see Figure 3 for an example). Readers are encouraged to consult the classic text by Innoué and Spring for an authoritative and complete treatment of Köhler illumination and imaging in optical microscopes [4]. Another excellent, and, for students, perhaps a more accessible review of the topic can be found in the text by Keller and Goldman [5].

Image Plane(primary)

Aperture Plane(reconstructed)

Aperture Plane(primary)

Image Plane(reconstructed)

F’ F’ F F FTLFTL FTL

Tube Lens(inside microscope)

Lens L1Lens L2

objective

Fig 1. Reconstruction of aperture and image planes in infinity corrected microscopes (not to scale). Microscopes from Nikon and Leica use tube lenses with a focal length of 200 mm, Zeiss 160 mm, and Olympus 180 mm. Lenses L1 and L2 should be achromatic doublets, and should have the same focal length as the tube lens for optimal performance. Vertical arrows indicate the locations of field and aperture stops (diaphragms). Spatial light modulators can be placed in the microscope's primary image plane, or in any of the reconstructed optical planes to manipulate light, at high speed and under computer control. There are many possible applications, including

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1366 ©FORMATEX 2010

______________________________________________

Page 2: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

optical sectioning microscopes, superresolution microscopes, and adaptive optics for enhancing microscope image quality.

2. Programmable array microscopy

2.1 Tandem scanning microscopy

Although confocal laser scanning microscopes (CLSMs) have excellent optical sectioning capabilities, they typically acquire images too slowly to be of use in live cell imaging, where dynamic cellular processes may happen very quickly. More recently, this situation has been alleviated somewhat with the introduction of resonant galvanometer scanners by major manufactures of CLSMs. These typically allow imaging rates up to ~ 10 images/sec (for an image size of 1000 x 1000 pixels). However, scanning with multiple points rather than a single point offers a much higher imaging rate. The tandem scanning microscope, invented in 1967 by Petráň and coworkers [6, 7], uses a spinning Nipkow disk to scan the sample with ~ 1000 points simultaneously. The modern implementation of this approach is embodied in the spinning disk units available from Yokagawa. When combined with EMCCD detection [8] and multiple lasers for excitation of multiple dyes or fluorescent proteins, these systems can offer an extremely high level of performance. The limitation of the Yokagawa system is that the disk itself can not be changed. As the size and spacing of the many scanning apertures are fixed, only a narrow range of objective lenses can be used without seriously compromising optical sectioning performance, for example only high magnification (100× or 60×) objectives with an NA ≥ 1.2. This makes these microscopes difficult to use when examining larger samples, such as embryos of model organisms (Drosophila, etc.) There are spinning disk systems which offer interchangeable disks (Olympus DSU), but these disks can not be changed during imaging. The Olympus DSU is a slit scanning system, and has lower optical sectioning ability (albeit with much lower cost) than the Yokagawa design. Additionally, fluorescence recovery after photobleaching (FRAP) protocols can not be realized without additional equipment. Typically, a galvanometer scanner is placed between the microscope and spinning disk unit. Laser light is diverted from the spinning disk into the FRAP module, photobleaching or photoconversion is performed, and confocal scanning then resumes. However, there is a system available (Mosaic, Photonic Instruments, Saint Charles, Illinois) which uses a digital micromirror device (DMD, discussed below) for FRAP and can be combined with spinning disk microscopes. Finally, spinning disk systems suffer from very low light throughput. Although microlenses (and, in the newest models, beam shaping optics) are used to focus the illumination light through the pattern, the duty cycle of the pattern (ratio of open pinhole area to total image area) is very low (about 4%), and the microlenses are not used in the fluorescence emission pathway. The price for confocality in these systems is thus the loss of ~ 96% of the total emitted light. Readers are encouraged to consult the Handbook of Biological Confocal Microscopy, the standard reference in the field [9]. The optical systems of both CLSMs [10] and of tandem scanning (spinning disk) microscopes [11] have been reviewed in this text, while other reports document comparisons between spinning disk systems and CLSMs [12], and also slit scanning microscopes [13].

2.2 DMD-based PAMs

A different approach to accomplish tandem scanning is to use a SLM placed in an image plane to create the scanning patterns and apertures. These are the programmable array microscopes (PAMs). The first functional PAMs used digital micromirror devices (DMDs) from Texas Instruments [14-17]. This type of SLM uses an array (640 x 480 in the first DMDs) of aluminum mirrors mounted on flexible hinges. Each mirror has only two possible positions, tilting ± 10°. Much higher performance DMDs are available today, with larger formats (up to 1920 x 1080 pixels), larger micromirror tilt angles (± 12°) and better optical surface quality. In a PAM, the two positions are designated "on" or "off," with "on" pixels directing light toward the sample, and "off" pixels directing light to a dump. Pixels which are "on" also serve as the confocal pinholes, and direct fluorescence emission to a CCD camera. If the pattern is sparse (low duty cycle), this will be an optically sectioned image. We call this the conjugate (in focus) image. A few other optical sectioning PAMs have been reported which also used DMDs and conjugate detection [18-21]. Further PAMs based on DMDs have included devices operating with reflected light (not fluorescence) [22, 23]. Realizing that "off" pixels were directing out of focus fluorescence emission to the dump, the Jovin group improved their earlier design by adding second imaging system and second CCD camera for collecting this "non-conjugate" image [24, 25]. In this configuration, all of the fluorescence emission which is collected by the objective is imaged and used, except for light lost to diffraction or other sources of optical loss. After registration of the two images, their addition yields a conventional widefield image, whereas subtraction of the non-conjugate image as:

Ios = Ic - Inc dc/(1- dc) , (1)

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1367

______________________________________________

Page 3: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

where Ios is an optically sectioned (confocal) image, Ic is the conjugate (in focus) image, Inc is the non-conjugate (out of focus) image, and dc is the duty cycle of the pattern, results in a confocal image. Both the conjugate and non-conjugate images are formed by integrating at least one complete scan with the CCD camera or cameras. Use of the non-conjugate image allows operation of a PAM with much higher duty cycle scanning patterns while maintaining confocality equivalent to or better than Yokagawa spinning disk systems. Pseudo-random patterns can also be used, increasing the duty cycle up to 50%. One source of pseudo-random patterns are the Sylvester sequences [26]. Scanning with pseudo-random patterns relies on the concept of aperture correlation [27-29]. Here, the idea is to reduce to a minimum the amount of correlated crosstalk between the many open pinholes. The theoretical and practical details of how this is accomplished in PAMs have been thoroughly discussed [14, 17, 24, 30]. The main advantage of using high duty cycle patterns is that it dramatically increases the sensitivity of the microscope. Deconvolution schemes have also been proposed for calculation of the optically sectioned images acquired with a PAM. Although more difficult to implement, such methods have better performance, especially in terms of the signal to noise ratio [24]. There are also more advanced methods for scaled subtraction of non-conjugate images [31]. In addition to routine imaging, PAMs and PAM like devices have been used for a variety of spectroscopic measurements [15, 32-37], selective photoreactions (FRAP, uncaging, etc.) [21, 25, 30], and fluorescence lifetime imaging (FLIM) [38-40]. There are also PAM like devices that operate without detector masks [41, 42], using the CCD camera to make the virtual apertures. Multifocal, multiphoton excitation microscopes have also been designed using SLMs for "scanless" imaging and photostimulation [43]. There have also been systems reported which speed up imaging rates by spatial-temporal multiplexing [44]. One advantage of the PAM is that expansion of an existing system into these areas is usually straightforward. For example, frequency domain FLIM can be accomplished by simply adding an appropriate modulated light source and detector.

2.3 PAMs based on liquid crystal microdisplays

PAMs have also been designed using SLMs based on liquid crystal-on-silicon (LCOS) microdisplays. These microdisplays work by manipulating the polarization of light. Similar to DMD-based designs, the LCOS is placed in an image plane of the microscope, and creates "on" pixels by rotating the plane of polarization of linearly polarized incident light by (ideally) 90°. The LCOS acts as a pixilated, addressable quarter-wave plate with a reflective backing. Typically, the LCOS microdisplay is placed behind a polarizing beamsplitter (PBS) cube, or between crossed linear polarizers. When used in this way, LCOS microdisplays operate as amplitude modulators [45]. Some of the earliest reported PAMs used either one [46, 47] or two [48] LCOS microdisplays, though these devices were limited to reflected light. Recently, a dual sided PAM (i.e., capable of collecting both conjugate and non-conjugate images) based on a LCOS microdisplay was constructed and used for high speed confocal imaging of a variety of biological samples [30, 38, 49]. This PAM used the SXGA-R2D LCOS microdisplay from Fourth Dimension Displays (Dalgety Bay, Fife, Scotland). This display has a 1280 x 1024 pixel format, and 93% fill factor. The displays come with drive sequences allowing easy implementation of binary bitplane weighting (see Section 2.4, PAM design considerations, below). A purely binary microdisplay with relatively easy to use synchronization signals is also available form this company (model SXGA-3DM), but this display does not allow real-time updating of the scanning pattern. Rapid updates of the pattern (based on incoming image information) can be used for a variety of advanced imaging modes, including controlled light exposure microscopy [50-52]. In this method, the dose of incident light is controlled on a pixel-by-pixel basis to achieve a desired signal to noise ratio, while reducing photobleaching and phototoxicity. The LCOS-based PAM has been used for imaging a variety of imaging tasks, including tracking of single quantum dots (QDs) bound to receptor proteins on the surfaces of live cells [49], FLIM [38], and multi-spot FRAP [30]. FRAP can be accomplished with no additional hardware by simply introducing a bleaching pattern accompanied by increasing the light intensity, then returning to confocal imaging. The time required to switch from a bleaching pattern to an imaging pattern can be as brief as ~ 1ms. This PAM is also sufficiently sensitive to use a single LED as the light source for confocal microscopy. LED illumination has numerous advantages including low cost, and reduced background compared to conventional mercury arc lamps. At present, no commercially available confocal system is marketed with LEDs for illumination. Figure 2 shows a maximum intensity projection of optical sections acquired with the LCOS-based PAM [30, 38, 49] using LED illumination. The LED was a 3 W Luxeon Star emitting at 470 nm. When pulsed (~ 50% duty cycle as used in the LCOS-based PAM) and properly cooled, these LEDs can tolerate over 4 A of current - LEDs with lower

performance were inadequate for imaging. The image shows live A431 cells (an epidermoid carcinoma cell line of

Fig. 2 Maximum intensity projection of optical sections acquired with the LCOS-based PAM [30, 38, 49] using LED illumination. Live A431 cells expressing eGFP-EGFR (green) were treated with 200 pM QD655-EGF (red) as described in [53], allowed to internalize for 30 minutes at room temperature, then imaged with an exposure time of 400 ms using an EMCCD camera. Conjugate and non-conjugate images were acquired and processed as described in [30, 38].

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1368 ©FORMATEX 2010

______________________________________________

Page 4: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

human origin [54]) expressing eGFP-EGFR (the epidermal growth factor receptor, ErbB1, in a fusion protein construct with enhanced green fluorescent protein) in green, and internalized QD655-EGF (the epidermal growth factor ligand coupled to QD655 using biotin-streptavadin chemistry) in red, approximately 30 minutes after introduction of 200 pM QD655-EGF. This previously unpublished image was obtained as part of a larger and ongoing study using a variety of approaches to understand the dynamics and distribution of EGFR molecules (and other members of the erbB family) on the cell surface [30, 49, 55-58]. Sample preparation protocols detailing the use of QD probes for live cell imaging have also been published [53].

2.4 PAM design considerations

One possible configuration for a DMD-based, confocal, fluorescence PAM for imaging of biological samples with simultaneous conjugate and non-conjugate detection is shown schematically in Figure 3. The optical system presented here is not ideal. For example, some of the excitation light will be scattered off the surface of the display. A better design would specifically control stray light throughout the entire microscope system. There are large amounts of literature on DMD illumination systems for projection in which this and other issues are addressed; [59] is but one example. Although the suggested achromatic doublet lenses have good performance, higher performance imaging lenses are certainly possible. Optical systems employing mirror optics were used in some of the original PAM designs [14, 24]. Focusing mirrors have the advantage of zero chromatic aberration, a very important consideration in fluorescence microscopy, where commonly used dyes range from the UV to NIR regions of the spectrum.

image plane(primary)

with field stop

apertureplane

(primary)

CCD 1(conjugate image)

DMD

CCD 2(non-conjugate image)

image plane

(reconstructed)

12°12°

sample

microscopeobjective

light source

(laser, lamp, LED)

emissionfilter

filtercube

apertureplane

(reconstructed)with stop

tube lens

Fig. 3 A possible configuration for a dual sided, DMD-based PAM with simultaneous conjugate and non-conjugate detection using two CCD cameras (not to scale). As in Figure 1, lenses should be achromatic doublets, and should ideally have the same focal length as the tube lens. However, when choosing system magnification, one must be sure to consider sampling of the image with the CCD camera or cameras; see reference [60]. Arrow pairs indicate suggested locations of field or aperture stops. Solid lines indicate field images, dotted lines indicate aperture images. Registration and processing of the conjugate and non-conjugate images as in [24, 30] can be accomplished in real time using modern computer hardware, especially if graphics processing cards can be employed [38, 61].Use of achromatic doublet lenses will limit the useful wavelength range, typically to 400 nm to 700 nm, depending on the anti-reflection coatings available. Further enhancements to the system shown in Figure 3 needed to make a complete imaging system would include multiple lasers for excitation of a variety of fluorescent dyes, filter wheels for the emission filters, a piezo-scanning Z stage, and back-illuminated EMCCD cameras, all under computer control.

A difficulty in using DMDs (and other types of SLMs) for PAMs is in circumventing the pulse-width modulation drive scheme used by these displays for projection purposes. These drive schemes create intensity variations in an image by controlling the on time for each pixel at high speed, e.g. 1.44 kHz; the resulting image is integrated by the eye. To accomplish this, each image to be displayed is broken into e.g. 24 "bitplanes", 8 each for red, green, and blue. Bright parts of the image will then be displayed for all 8 bitplanes, while dim parts of the image will be displayed for fewer bitplanes by toggling the needed pixels off. Further, each bitplane has a different duration, usually incremented by factors of 2. This gives at least 256 possible intensity levels for each color. In PAMs, to achieve the highest imaging

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1369

______________________________________________

Page 5: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

rates possible, one must use binary bitplane weighting, in which each bitplane is displayed for the same length of time. This option must be available in the drive electronics and software provided with a given microdisplay. A further non-trivial challenge lies in synchronization of the scanning patterns, relayed to the display as video images, with the CCD camera. Real-time computer operating systems alleviate some of the difficulties. Failure of synchronization leads to banding artefacts in the acquired images, as the camera exposure time must be an integral number of complete scans. A similar problem was encountered in early Yokagawa spinning disk systems, in which the speed of the disk could not be changed [8]. One chief advantage of LCOS microdisplays over DMDs for use in PAMs is that the diffraction efficiency of these devices is much lower. The regular array of tilted mirrors used in DMD microdisplays function as an efficient diffraction grating, and as such much of the incident light will be lost to diffraction of the light into multiple orders. This is especially noticeable when using coherent light sources (i.e., lasers). For the excitation light, this need not be detrimental, as one might be able to simply increase the intensity of the light source. The situation is much worse for fluorescence emission, as any light loss leads to a decrease in sensitivity of the microscope. Figure 4a shows the principle of operation of an LCOS microdisplay operated as an amplitude modulator for projection or for use in PAMs, while Figure 4b shows a possible configuration for an LCOS-based PAM.

vertically polarized light (S)

polarizing beam splitter (PBS) cube

horizontally polarized light (P)

“on “ pixels

quarter-wave plate

mirror

LCOS

uniform illumination

image or pattern

(optional)clean uppolarizer

LCO

S

vertical polarization “S”

If ON, horizontal polarization “P”

lightsource

If OFF, vertical polarization “S”

concave focusingmirror

a b

Fig. 4 a) Principle of operation of an LCOS microdisplay. The display acts as a quarter wave plate (see chapter 8 in [62]) with a reflective backing. For amplitude modulation, LCOS microdisplays are operated between crossed polarizers or behind PBS cubes. In this configuration, light reflected by "off" pixels will reflect off the polarizing surface in the PBS cube and return back to the source. b) One (partial) possible configuration for an LCOS-based PAM. Only the excitation pathway is shown. Schemes using "polarization recycling" can be devised so that both S and P polarized light can be used for both excitation and emission [30]. If this is combined with non-conjugate detection, no light is lost (neglecting optical losses, i.e., imperfections in antireflection coatings, etc).

Although the LCOS-based PAM has a high level of performance, there are several disadvantages to using LCOS microdisplays for PAMs. One main difficulty is that the LCOS devices (and other SLMs) are not optically flat. This can potentially degrade image quality quite badly. One possible solution for this is introduction of a second SLM in an aperture plane to correct the optical aberrations. This is an example of SLM-based adaptive optics, discussed below. Also, LCOS displays are limited to visible light and can be damaged by exposure to UV. The device used in the LCOS-based PAM described here is practically limited to ~ 450 nm to ~ 650 nm. Beyond these limits, the LCOS does not effectively change the polarization state of light, and thus image contrast begins to suffer. Another difficulty with LCOS displays is that they are fluorescent, as are typical PBS cubes. In the LCOS-based PAM using a single microdisplay, this fluorescence degrades image contrast. Imperfect polarization properties of both the LCOS display (incomplete, wavelength and angle dependent rotation of the plane of polarization) and of the PBS cubes (typically about 5% of P polarized light is incorrectly reflected) also degrade image contrast. Newer designs using two LCOS displays are under construction and will eliminate these problems.

3. Structured illumination microscopy using spatial light modulators

3.1 Optical sectioning microscopy using structured illumination

Structured illumination microscopy (SIM) is defined by Heintzmann [63] as "…the acquisition of a set of individual images at a given focus plane, each made with a different position of an illumination mask, but made with no mask in

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1370 ©FORMATEX 2010

______________________________________________

Page 6: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

the detection path, that is, widefield detection." In this context, the PAM would not be considered part of the structured illumination microscopy family, as it relies on a detection pathway mask to create the confocal effect. SIM has been used rather extensively to create optical sectioning microscopes. This technique was introduced by Neil, Juškaitis, and Wilson [64]. The method works by projecting a sinusoidal grid illumination pattern (lines or stripes of illumination, or interference fringes) onto the sample, and acquiring an image with widefield detection. One then acquires two more images with the position of the grid pattern shifted by relative phase angles π/3 and 2π/3. An optically sectioned image can then be computed as:

Ios = [(I1 - I2)2 + (I1 - I3)2 + (I2 - I3)2]1/2,(2) where Ios is an optically sectioned image, and I1, I2, and I3 are the three images acquired with different grid positions. The optically sectioned image can also be calculated by a variety of other methods [31, 63, 65, 66]. A widefield image can be computed as Iw = I1 + I2 + I3. One main difficulty in processing the images is elimination of banding artifacts in the final image. This is usually attributed to photobleaching of the pattern into the sample, or to imprecise shifting of the pattern (the phase of the pattern must be shifted by exactly π/3). There are methods to deal with this problem based on optimization of a goal function designed to minimize banding, or on frequency domain approaches [67-69]. This form of microscopy, which we prefer to call "SIM for sectioning," to differentiate it from the high resolution approaches discussed below, has been expanded upon in many ways. Notably, there are two commercial systems (Zeiss Apotome and Qioptiq Optigrid) available based on shifting grid patterns. There is also a commercial system (Biomedica Mangoni ViCo) which is based on illumination using a 2-D array of shifting points [31, 63]. Although these systems have good performance, they usually acquire images too slowly to be useful for live cell imaging, as they use mechanically shifting diffraction gratings or masks to create the pattern. There are many examples of applications of the SIM for sectioning method in many fields of study. One example which we wish to highlight is the work of Walter, et al. [70]. Here, the authors compare CLSM, SIM for sectioning using the ViCo device, and widefield microscopy used together with deconvolution for their studies of many-color 3D fluorescence in-situ hybridization (M-FISH). Other works have systematically studied the performance of SIM for sectioning methods [71, 72]. It has also been found that, when the frequency of the projected grid is optimized, the method can even have better optical sectioning performance than CLSM [73]. In addition to the shifting gratings or masks that are typically used, it is also possible to construct a SIM for sectioning system using LED line arrays [74]. These devices consist of stripes of emissive LED material which are each individually addressable. There are also 2-D LED array devices [75], but, at the time of writing, their use has not yet been demonstrated in SIM for sectioning applications. Investigators working in this field have turned to SLMs in the quest for higher imaging rates and increased flexibility. It is even possible to construct a SIM for sectioning system with a standard digital projector based on a DMD [76]. This approach has the notable advantages of very low cost, and multiwavelength excitation under computer control. For example, one simply needs to display a fringe pattern with the projector using the desired color. In some cases, no additional excitation filters will be necessary. Other systems have used DMDs [77, 78], or transmissive liquid crystal SLMs [65]. Other areas where SLM-based SIM for sectioning has been used are in endoscopy [68, 79], and in fluorescence lifetime imaging (FLIM) [69], both are methods that benefit greatly from optical sectioning. Figure 5a shows an example of optical sectioning via SIM acquired using a DMD SLM, a (filtered) conventional white light source, and a conventional CCD camera. The optically sectioned images were computed according to Equation (2). Figure 5b shows the widefield result, obtained by summing the images of the three phases on a plane-by plane basis as described above. A flat-field correction was applied by imaging a uniformly fluorescent sample with the DMD illumination system. The right-hand panels in the figure show maximum intensity projections of the acquired focal planes. The sample is an autofluorescent pollen grain approximately 15 μm in diameter.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1371

______________________________________________

Page 7: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

Fig. 5 Example of SIM for sectioning using a DMD SLM. a) SIM as described in section 3.1. b) Widefield image computed by summing the three phases. At right, maximum intensity projections. Unpublished data from the authors' laboratory. The diameter of the pollen grain is about 15 μm.

3.2 Superresolution microscopy using structured illumination

The resolution of optical microscopes in the lateral dimension is limited to rairy = 0.61λ /NA, where NA is the numerical aperture of the microscope objective, and λ is the wavelength of light. This means that structures smaller than ~ 250 nm can not be resolved. As many biological structures within cells are much smaller than this, increasing resolution is of prime importance. Recently, a great deal of progress has been made in extending the resolution of fluorescence microscopes. Together, these methods are called "superresolution" microscopies. The two main approaches involve (i) photophysical (photochemical) manipulation of fluorescent probes, or (ii) enlarging the region of support of the optical transfer function (OTF) of the microscope. The first approach includes methods such as stimulated emission depletion microscopy (STED [80]; the STED beam can be produced using a SLM [81]), and several approaches based on localization of single molecules, such as stochastic optical reconstruction microscopy (STORM) [82] and photoactivated localization microscopy (PALM) [83]. The second approach includes methods such as 4Pi microscopy [84] and structured illumination [85]. Although similar, this method is distinct from the SIM for sectioning approach discussed above. "Superresolution SIM" also uses a fringe pattern, but with a higher spatial frequency than the patterns used for optical sectioning. Using the Moiré effect, high resolution information which can not be resolved by the microscope is sampled, or mixed, into lower spatial frequencies. A superresolution image with about twice the lateral resolution of the original can then be recovered using Fourier domain approaches [85-87]. Figure 6 shows the principle of superresolution SIM. Projection of a fringe pattern onto the sample results in shifting of high resolution (i.e., high frequency) information not resolvable by the microscope into the observable region, as exemplified by the Moiré effect seen in Figure 6a. Figure 6b shows the Fourier domain representation of the result of such an illumination pattern. Rotating the orientation of the illumination pattern allows nearly isotropic sampling of the high resolution information. Figure 6c shows the expanded region of support of the microscope's OTF that results when using three pattern orientations. SIM can be combined with total internal reflection fluorescence (TIRF) microscopy, taking advantage of the very small penetration depth of the evanescent field (~100 nm). TIRF offers improved axial resolution, but limits imaging to structures near the coverslip surface [88]. There are variations of the SIM method which improve axial resolution only, using a standing wave formed by the interference of laser light launched by opposing microscope objectives [89]. This approach has been combined with a method similar to STORM [90]. It is also possible to exploit nonlinear optical effects to gain even higher (theoretically unlimited) resolution [91, 92]. There is a recent application of SIM which offers superresolution in all 3 dimensions [93-96]. This method uses interference of 3 laser beams to form a 3D modulation pattern. Of the current methods for achieving superresolution, 3D SIM is perhaps the most promising, as any fluorescent dye, QD, or GFP variant can be used, and it can be accomplished with relatively straightforward hardware. It gives a fully 3D image with nearly isotropic resolution. 3D SIM has been used for multicolor imaging of the nuclear pore complex, where it is able to provide near-molecular level detail about the structure of this macromolecular complex [95]. As multiple images (typically 9 for the 2D case, or 15 for the 3D case) must be acquired to compute a single superresolution image, the desire to collect the images as quickly as possible has led to the use of SLMs [88, 97-100]. These microscopes typically use phase-only LCOS SLMs to generate the patterns, not amplitude modulators as discussed above. The fringe pattern is thus formed by interference of two (or three, in the 3D case) laser beams, leading to higher contrast (modulation) in the pattern than is possible when directly imaging an amplitude modulating microdisplay.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1372 ©FORMATEX 2010

______________________________________________

Page 8: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

cba

Fig. 6 Using structured illumination for resolution enhancement. a) Moiré fringes, where high frequencies in the image are encoded into low frequencies (vertical stripes in the overlap region). b) The central circular region defines a diffraction limited frequency band in reciprocal space of a conventional microscope. A sinusoidally striped illumination pattern shifts part of the high frequency information, encoding it within the diffraction limited area. This high frequency information can be recovered and shifted back to its original position. The result is an image with increased resolution in the direction defined by the illumination pattern. c) Reconstruction of a wider frequency band from a sequence of images with different orientations of the illumination pattern.

4. Adaptive optics

Adaptive optics (AO) in microscopy is a method borrowed from astronomy (reviewed in [101]). The general idea is to use a deformable mirror or other SLM to adjust the wavefront of a beam of light so as to correct aberrations in the image. In astronomy, the aberrations are caused by atmospheric turbulence, causing the familiar twinkling of stars. In microscopy, the aberrations are caused by refractive index mismatches between the objective lens, mounting medium, and sample, as well as by imperfections in the objective lens. Progress in AO microscopy has recently been reviewed [102, 103]. The problem of microscope image deterioration is particularly severe when attempting to image deeply within samples. This is typically accomplished with multiphoton excitation microscopy, which achieves much greater penetration due to the use of near infrared light. Very recently, a multiphoton system was devised using a phase-only SLM for AO. This was a conceptually simple, wavefront-sensorless, high speed system which works by pupil segmentation [104]. The gain in image quality and increase in the ultimate possible imaging depth in this system is especially noteworthy. One interesting system combines SIM for sectioning with adaptive optics [105]. This idea addresses one of the issues that arises when using SLMs - poor surface flatness (discussed briefly above). It should be possible to devise a system capable of high speed optical sectioning with all the flexibility of the PAM, but with the enhanced image quality available from AO. This will be especially important when imaging deeply within thick samples. There are several commercial offerings for AO components (deformable mirrors and wavefront sensors), but one recent development is especially designed for microscopy [106]. This will allow non-specialists to get involved in this emerging field with a "plug and play" optical system. Figure 7a shows the general idea of adaptive optics, in which a wavefront sensor controls a deformable mirror to correct wavefront aberrations in the signal beam. An alternative, sensorless configuration is shown in Figure 7b, where an SLM is used to control the illumination so as to achieve the best possible image.

a bdistorted

beam

objectivemicroscope

planeaperature

mirror

beamsplitterimage

plane camera

SLM

sourcelight

camerasensorwavefront

feed

back

mirrordeformable microscope

light from

splitter

Fig. 7 a) General scheme for adaptive optics using a deformable mirror and wavefront sensor. b) Sensorless adaptive optics setup using a phase-only microdisplay to adjust the illumination wavefront. Solid lines: field images, dotted lines: aperture images.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1373

______________________________________________

Page 9: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

5. Conclusions

SLMs have emerged as important tools in fluorescence microscopy, finding uses in optical sectioning, superresolution, and adaptive optics. We predict increased interest in SLM microscopy as a wider variety of devices become available, and as new, innovative applications are found. Of particular importance is the system introduced by Debarre and coworkers [105], which combines two SLMs for optical sectioning and adaptive optics. Such combinations are expected to become more commonplace as technolgy allows, and will help maximize the performance, flexibility, and image quality of fluorescence microscopes.

Acknowledgements Support by the Grant Agency of the Czech Republic (GAČR 304/09/1047 to GMH) is gratefully acknowledged. This work was also supported by grants MSM0021620806 and LC535 from the Ministry of Education, Youth and Sports of the Czech Republic, and by grant AV0Z50110509 from The Grant Agency of the Academy of Sciences of the Czech Republic.

References [1] Gorthi SS and Rastogi P, Fringe Projection Techniques: Whither we are? Optics and Lasers in Engineering, 2010. 48(2): p.

133-140. [2] Schonbrun E, et al., 3D interferometric optical tweezers using a single spatial light modulator. Optics Express, 2005. 13(10):

p. 3777-3786. [3] Grier DG, A revolution in optical manipulation. Nature Photonics, 2003. 424(6950): p. 810-816. [4] Inoue S and Spring KR, Video Microscopy: The Fundamentals. 2nd ed. 1997, New York: Plenum Press. [5] Keller E and Goldman RD, Light Microscopy, in Basic Methods in Microscopy, Spector DL and Goldman RD, Editors. 2006,

Cold Spring Harbor Laboratory Press: New York. p. 1-42. [6] Egger MD and Petráň M, New reflected-light microscope for viewing unstained brain and ganglion cells Science, 1967.

157(3786): p. 305-307. [7] Petráň M, et al., Tandem-scanning reflected-light microscope. Journal of the Optical Society of America, 1968. 58: p. 661-

664. [8] Coates CG, et al., Optimizing low-light microscopy with back-illuminated electron multiplying charge-coupled devices:

enhanced sensitivity, speed, and resolution. Journal of Biomedical Optics, 2004. 9(6): p. 1244-1252. [9] Pawley JB, Handbook of Biological Confocal Microscopy. 3rd ed. 2006, New York: Springer Science + Business Media LLC. [10] Stelzer EHK, The intermediate optical system of laser-scanning confocal microscopes, in Handbook of Biological Confocal

Microscopy, Pawley JB, Editor. 2006, Springer Science + Business Media LLC: New York. p. 207-220. [11] Toomre D and Pawley JB, Disk-scanning confocal microscopy, in Handbook of Biological Confocal Microscopy, Pawley JB,

Editor. 2006, Springer Science+Business Media, LLC: New York. p. 221-238. [12] Wang E, Babbey CM, and Dunn KW, Performance comparison between the high-speed Yokogawa spinning disc confocal

system and single-point scanning confocal systems. Journal of Microscopy, 2005. 218: p. 148-159. [13] Wolleschensky R, Zimmermann B, and Kempe M, High-speed confocal fluorescence imaging with a novel line scanning

microscope. Journal of Biomedical Optics, 2006. 11(6): p. 064011. [14] Hanley QS, et al., An optical sectioning programmable array microscope implemented with a digital micromirror device.

Journal of Microscopy, 1999. 196(3): p. 317-331. [15] Hanley QS, Verveer PJ, and Jovin TM, Optical sectioning fluorescence spectroscopy in a programmable array microscope.

Applied Spectroscopy, 1998. 52: p. 783. [16] Liang M, Stehr RL, and Krause AW, Confocal pattern period in multiple-aperture confocal imaging systems with coherent

illumination. Optics Letters, 1997. 22: p. 751-753. [17] Verveer PJ, et al., Theory of confocal fluorescence imaging in the programmable array microscope (PAM). Journal of

Microscopy, 1998. 189(3): p. 192-198. [18] Botvinick EL, et al. In-vivo confocal microscopy based on the Texas Instruments digital micromirror device. in Optical

Diagnostics of Living Cells III. 2000: SPIE. [19] Fainman Y, et al. 3D quantitative imaging of the microvasculature with the Texas Instruments Digital Micromirror Device in

Spatial Light Modulators: Technology and Applications. 2001: SPIE. [20] Dlugan ALP, MacAulay CE, and Lane PM. Improvements to quantitative microscopy through the use of digital micromirror

devices. in Optical Diagnostics of Living Cells III 2000: SPIE. [21] Fukano T, Hama H, and Miyawaki A, Similar diffusibility of membrane proteins across the axon-soma and dendrite-soma

boundaries revealed by a novel FRAP technique. Journal of Structural Biology, 2004. 147(1): p. 12-18. [22] Cha S, et al. 3D profilometry using a dynamically configurable confocal microscope. in Three-Dimensional Image Capture

and Applications II. 1999: SPIE. [23] Cha S, et al., Nontranslational three-dimensional profilometry by chromatic confocal microscopy with dynamically

configurable micromirror scanning. Applied Optics, 2000. 39(16): p. 2605-2613. [24] Heintzmann R, et al., A dual path programmable array microscope (PAM): simultaneous acquisition of conjugate and non-

conjugate images. Journal of Microscopy, 2001. 204(2): p. 119-135. [25] Fulwyler M, et al., Selective photoreactions in a programmable array microscope (PAM): Photoinitiated polymerization,

photodecaging, and photochromic conversion. Cytometry A, 2005. 67A(2): p. 68-75. [26] Harwit M and Sloane NJA, Hadamard Transform Optics. 1979, New York: Academic Press. [27] Juskaitis R, et al., Efficient real-time confocal microscopy with white light sources. Nature, 1996. 383(6603): p. 804-806.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1374 ©FORMATEX 2010

______________________________________________

Page 10: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

[28] Wilson T, et al., Confocal microscopy by aperture correlation. Optics Letters, 1996. 23: p. 1879-1881. [29] Levoy M, et al., Synthetic aperture confocal imaging. ACM Transactions on Graphics, 2004. 23(3): p. 825-834. [30] Hagen GM, et al., FRAP and photoconversion in multiple arbitrary regions of interest using a programmable array microscope

(PAM). Microscopy Research and Technique, 2009. 72: p. 431-440. [31] Heintzmann R and Benedetti PA, High-resolution image reconstruction in fluorescence microscopy with patterned excitation.

Applied Optics, 2006. 45(20): p. 5037-5045. [32] Hanley QS, Masking, photobleaching, and spreading effects in hadamard transform imaging and spectroscopy systems.

Applied Spectroscopy, 2001. 55: p. 318. [33] Hanley QS, Arndt-Jovin DJ, and Jovin TM, Spectrally resolved fluorescence lifetime imaging microscopy. Applied

Spectroscopy, 2002. 56: p. 155. [34] Hanley QS and Jovin TM, Highly multiplexed optically sectioned spectroscopic imaging in a programmable array

microscope. Applied Spectroscopy, 2001. 55: p. 1115. [35] Hanley QS, et al., Three-dimensional spectral imaging by Hadamard transform spectroscopy in a programmable array

microscope. Journal of Microscopy, 2000. 197(1): p. 5-14. [36] Hanley QS, Verveer PJ, and Jovin TM, Spectral imaging in a programmable array microscope by hadamard transform

fluorescence spectroscopy. Applied Spectroscopy, 1999. 53: p. 1. [37] Bednarkiewicz A, Bouhifd M, and Whelan MP, Digital micromirror device as a spatial illuminator for fluorescence lifetime

and hyperspectral imaging. Applied Optics, 2008. 47(9): p. 1193-1199. [38] Hagen GM, et al. Biological applications of an LCoS-based programmable array microscope. in Imaging, Manipulation, and

Analysis of Biomolecules, Cells, and Tissues V. 2007. San Jose, CA, USA: SPIE. [39] Hanley QS, et al., Fluorescence lifetime imaging in an optically sectioning programmable array microscope (PAM).

Cytometry A, 2005. 67A(2): p. 112-118. [40] Bednarkiewicz A and Whelan MP, Global analysis of microscopic fluorescence lifetime images using spectral segmentation

and a digital micromirror spatial illuminator. Journal of Biomedical Optics, 2008. 13(4): p. 041316: 041311-041313. [41] Rector DM, Ranken DM, and George JS, High-performance confocal system for microscopic or endoscopic applications.

Methods, 2003. 2003(30): p. 16-27. [42] Vermolen BJ, Garini Y, and Young IT, 3D restoration with multiple images acquired by a modified conventional microscope.

Microscopy Research and Technique, 2004. 64: p. 113-125. [43] Nikolenko V, et al., SLM microscopy: scanless two-photon imaging and photostimulation with spatial light modulators.

Frontiers in Neural Circuits, 2008. 2: p. doi: 10.3389. [44] Bub G, et al., Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging. Nature Methods, 2010. 7(3):

p. 209-211. [45] Armitage D, Underwood I, and Wu S-T, Introduction to Microdisplays. Wiley-SID series in display technology, ed. Lowe

AC. 2006, West Sussex, England: John Wiley and Sons. [46] Smith PJ, et al., Programmable array microscopy with a ferroelectric liquid-crystal spatial light modulator. Applied Optics,

2000. 39: p. 2664-2669. [47] Taylor C, Smith PJ, and McCabe EM. A programmable array microscope demonstrator: application of a ferroelectric liquid

crystal SLM. in Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing VII. 2000: SPIE. [48] Taylor CM and McCabe EM. Programmable array microscope employing two ferroelectric liquid crystal spatial light

modulators. in Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing VIII. 2001: SPIE. [49] Hagen GM, et al., Dynamics of membrane receptors: Single molecule tracking of quantum dot liganded epidermal growth

factor, in Single Molecule Dynamics in Life Science, Ishii Y and Yanagida T, Editors. 2008, Wiley Press: New York. p. 346. [50] Hoebe RA, et al., Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell

imaging. Nature Biotechnology, 2007. 25(2): p. 249-253. [51] Hoebe RA, et al., Quantitative determination of the reduction of phototoxicity and photobleaching by controlled light

exposure microscopy. Journal of Microscopy, 2008. 231(1): p. 9-20. [52] Caarls W, et al. Arbitrary and Dynamic Patterning in a Programmable Array Microscope. in Focus on Microscopy. 2009.

Krakow, Poland. [53] Lidke DS, et al., Biotin-Ligand Complexes With Streptavidin Quantum Dots for In Vivo Cell Labeling of Membrane

Receptors in Quantum Dots: Applications in Biology, Bruchez MP and Hotz CZ, Editors. 2007, Humana Press: Totowa, New Jersy. p. 69-79.

[54] Giard DJ, et al., In vitro cultivation of human tumors: establishment of cell lines derived from a series of solid tumors. Journal of the National Cancer Institute, 1973. 51(5): p. 1417-1423.

[55] Lidke DS, et al., Reaching out for signals: filopodia sense EGF and respond by directed retrograde transport of activated receptors. Journal of Cell Biology, 2005. 170(4): p. 619-626.

[56] Lidke DS, et al., Quantum dot ligands provide new insights into erbB/HER receptor-mediated signal transduction. Nature Biotechnology, 2004. 22(2): p. 198.

[57] Lidke DS, et al., Imaging molecular interactions in cells by dynamic and static fluorescence anisotropy (rFLIM and emFRET). Biochemical Society Transactions, 2003. 31(Pt 5): p. 1020-1027.

[58] Grecco HE, et al., Ensemble and single particle photophysical properties (two-photon excitation, anisotropy, FRET, lifetime, spectral conversion) of commercial quantum dots in solution and in live cells. Microscopy Research and Technique, 2004. 65(4-5): p. 169-179.

[59] Yi Chin F, Wei Teng L, and Hsien Lin T. High-definition DLP zoom projector lens design with TIR prism for high-definition television (HDTV). 2006: SPIE.

[60] Heintzmann R, Band-Limit and appropriate sampling in microscopy in Cell Biology: A Laboratory Handbook, Celis Julio E, Editor. 2006, Elsevier Academic Press. p. 29-36.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1375

______________________________________________

Page 11: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

[61] Buck I, et al., Brook for GPUs: Stream Computing on Graphics Hardware. ACM Transactions on Graphics, 2004. 23(3): p. 777-786.

[62] Hecht E, Optics. 4th ed. 2002, San Francisco: Addison Wesley. [63] Heintzmann R, Structured Illumination Methods, in Handbook of Biological Confocal Microscopy, Pawley JB, Editor. 2006,

Springer Science + Business Media, LLC: New York. p. 265-279. [64] Neil MAA, Juskaitis R, and Wilson T, Method of obtaining optical sectioning by using structured light in a conventional

microscope. Optics Letters, 1997. 22(24): p. 1905-1907. [65] Monneret S, Rauzi M, and Lenne PF, Highly flexible whole-field sectioning microscope with liquid-crystal light modulator.

Journal of Optics A: Pure and Applied Optics, 2006. 8(7): p. S461. [66] Karadaglic D and Wilson T, Image formation in structured illumination wide-field fluorescence microscopy. Micron, 2008.

39(7): p. 808-818. [67] Schaefer LH, Schuster D, and Schaffer J, Structured illumination microscopy: artefact analysis and reduction utilizing a

parameter optimization approach. Journal of Microscopy, 2004. 216(2): p. 165-174. [68] Bozinovic N, et al., Fluorescence endomicroscopy with structured illumination. Optics Express, 2008. 16(11): p. 8016-8025. [69] Cole MJ, et al., Time-domain whole-field fluorescence lifetime imaging with optical sectioning. Journal of Microscopy, 2001.

203(3): p. 246-257. [70] Walter J, et al., Towards many colors in FISH on 3D-preserved interphase nuclei. Cytogenetic and Genome Research, 2006.

114(3-4): p. 367-378. [71] Weigel A, Schild D, and Zeug A, Resolution in the ApoTome and the confocal laser scanning microscope: comparison.

Journal of Biomedical Optics, 2009. 14(014022): p. DOI:10.1117/1111.3083439. [72] Barlow AL and Guerin CJ, Quantization of widefield fluorescence images using structured illumination and image analysis

software. Microscopy Research and Technique, 2007. 70: p. 76-84. [73] Chasles F, Dubertret B, and Boccara AC, Optimization and characterization of a structured illumination microscope. Optics

Express, 2007. 15(24): p. 16130-16140. [74] Poher V, et al., Optical sectioning microscope with no moving parts using a micro-stripe array light emitting diode. Optics

Express, 2007. 15(18): p. 11196-11206. [75] Poher V and et al., Micro-LED arrays: a tool for two-dimensional neuron stimulation. Journal of Physics D: Applied Physics,

2008. 41(9): p. 094014. [76] Delica S and Blanca CM, Wide-field depth-sectioning fluorescence microscopy using projector-generated patterned

illumination. Applied Optics, 2007. 46(29): p. 7237-7243. [77] Fukano T and Miyawaki A, Whole-field fluorescence microscope with digital micromirror device: imaging of biological

samples. Applied Optics, 2003. 42: p. 4119-4124. [78] Fukano T, et al., Differential Ras activation between caveolae/raft and non-raft microdomains. Cell Structure and Function,

2007. 32(1): p. 9-15. [79] Lane PM, et al., Fiber-optic confocal microscopy using a spatial light modulator. Optics Letters, 2000. 25: p. 1780-1782. [80] Westphal V, et al., Video-Rate Far-Field Optical Nanoscopy Dissects Synaptic Vesicle Movement. Science, 2008. 320: p.

246-249. [81] Willig KI, et al., Nanoscale resolution in GFP-based microscopy. Nature Methods, 2006. 3(9): p. 721-723. [82] Rust MJ, Bates M, and Zhuang X, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM).

Nature Methods, 2006. 3(10): p. 793-795. [83] Betzig E, et al., Imaging intracellular fluorescent proteins at nanometer resolution. Science, 2006. 313(5793): p. 1642-1645. [84] Egner A, Jakobs S, and Hell SW, Fast 100-nm resolution three-dimensional microscope reveals structural plasticity of

mitochondria in live yeast. Proceedings of the National Academy of Sciences of the USA, 2002. 99: p. 3370-3375. [85] Gustafsson MGL, Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. Journal

of Microscopy, 2000. 198: p. 82-87. [86] Heintzmann R and Cremer C, Laterally modualted excitation microscopy: improvement of resolution by using a diffraction

grating. SPIE, 1998. 3568: p. 185-196. [87] Frohn JT, Knapp HF, and Stremmer A, True optical resolution beyond the Rayleigh limit achieved by standing wave

illumination. Proceedings of the National Academy of Sciences of the USA, 2000. 97: p. 7232-7236. [88] Kner P, et al., Super-resolution video microscopy of live cells by structured illumination. Nature Methods, 2009. 6(5): p. 339-

342. [89] Bailey B, et al., Enhancement of axial resolution in fluorescence microscopy by standing-wave excitation. Nature, 1993.

366(6450): p. 44-48. [90] Reymann J, et al., High-precision structural analysis of subnuclear complexes in fixed and live cells via spatially modulated

illumination (SMI) microscopy. Chromosome Research, 2008. 16(3): p. 367-382. [91] Gustafsson MGL, Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically

unlimited resolution. Proceedings of the National Academy of Sciences of the USA, 2005. 102: p. 13081-13086. [92] Heintzmann R, Jovin TM, and Cremer C, Saturated patterned excitation microscopy: a concept for optical resolution

improvement. Journal of the Optical Society of America A, 2002. 19(8): p. 1599-1609. [93] Gustafsson MGL, et al., Three-dimensional resolution doubling in widefield fluorescence microscopy by structured

illumination. Biophysical Journal, 2008. 94(12): p. 4957-4970. [94] Carlton PM, Three-dimensional structured illumination microscopy and its application to chromosome structure. Chromosome

Research, 2008. 16: p. 351-365. [95] Schermelleh L, et al., Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy.

Science, 2008. 320: p. 1332-1336.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

1376 ©FORMATEX 2010

______________________________________________

Page 12: Spatial light modulators in fluorescence · PDF fileSpatial light modulators in fluorescence microscopy Pavel Křížek1,2 and Guy M. Hagen1,2 1 Charles University in Prague, First

[96] Hasse S, OMX – A Novel High Speed and High Resolution Microscope and its Application to Nuclear and Chromosomal Structure Analysis, in Mathematisch-Naturwissenschaftlichen Fakultat I. 2007, Humboldt-Universitat zu Berlin: Berlin. p. 144.

[97] Lin J-Y, et al., Wide-field super-resolution optical sectioning microscopy using a single spatial light modulator. Journal of Optics A: Pure and Applied Optics, 2009. 11: p. 015301.

[98] Hirvonen LM, et al., Structured illumination microscopy of a living cell. European Biophysical Journal, 2009. 38: p. 807-812. [99] Mandula O, Patterned Excitation Microscopy, in Institute of Physics MFF UK. 2008, Charles University in Prague. p. 68. [100] Fiolka R, Beck M, and Stemmer A, Structured illumination in total internal reflection fluorescence microscopy using a spatial

light modulator. Optics Letters, 2008. 33(14): p. 1629-1631. [101] Beckers JM, Adaptive Optics for Astronomy: Principles, Performance, and Applications. Annual Review of Astronomy and

Astrophysics, 1993. 31(1): p. 13-62. [102] Girkin JM, Poland S, and Wright AJ, Adaptive optics for deeper imaging of biological samples. Current Opinion in

Biotechnology, 2009. 20(1): p. 106-110. [103] Booth MJ, Adaptive optics in microscopy. Philosophical Transactions of the Royal Society A: Mathematical, Physical and

Engineering Sciences, 2007. 365(1861): p. 2829-2843. [104] Ji N, Milkie DE, and Betzig E, Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues.

Nature Methods, 2010. 7(2): p. 141-147. [105] Debarre D, et al., Adaptive optics for structured illumination microscopy. Optics Express, 2008. 16(13): p. 9290-9305. [106] Levecq X and Andilla J, Microscopy Imaging: Adaptive optics sharpens biological microscopy. Laser Focus World, 2010.

46(5): p. 66-69.

Microscopy: Science, Technology, Applications and Education A. Méndez-Vilas and J. Díaz (Eds.)

©FORMATEX 2010 1377

______________________________________________