1

CHROMATIC CONFOCAL MICROSCOPY USING LIQUID CRYSTAL PANELS

by

Qi Cui

____________________________ Copyright © Qi Cui 2019

A Report Submitted to the Faculty of the

COLLEGE OF OPTICAL SCIENCES

In Partial Fulfillment of the Requirements

For the Degree of

MASTER OF SCIENCE

In the Graduate College

THE UNIVERSITY OF ARIZONA

2019

2

3

ACKNOWELDGEMENT

First and foremost, I would like to thank my mentor, professor Rongguang Liang. During my three years at U of Arizona, I’ve learnt much from professor Rongguang Liang, not only the knowledge of optical systems, but also the right way to do research. The invaluable experience in Applied Optics Lab has prepared me well to start my PhD, and I believe it would also be precious in the future. Second, I would like to thank professor Dongkyun Kang and professor Leilei Peng for being my Master’s oral exam committee member. I also thank professor Kang for his suggestions and help on my student career. Third, I would like to thank my colleagues in Applied Optics Lab. I thank Xiaobo Tian, Zhihan Hong, Wei-chen Liao, Bofan Song, Wenjun Kang, Shaobai Li, for their help and beneficial talks. They helped me on my research and my courses. Fourth, I would like to thank my friends at U of A. I thank Aoxue Han, Hwang-Jye Yang, Chloë Castle, Tyler Joseph, Jieun Ryu, Moonseob Jin, and other friends. I thank them for their invaluable friendship. Fifth, I would like to thank my professors and TAs at U of A. I especially thank professor Amit Ashok and professor Tom Milster. I thank them for their recommendation letters for my Fall 2018 and Fall 2019 PhD applications. In the end, I thank my families for their endless love and supports in my life.

4

DEDICATION

To my parents, and Tong Shen

5

TABLE OF CONTENTS LIST OF FIGURES .......................................................................................................... 6 ABSTRACT ...................................................................................................................... 7 CHAPTER 1: BACKGROUND ....................................................................................... 8 CHAPTER 2: CHROMATIC CONFOCAL SYSTEM .................................................... 10 CHAPTER 3: SYSTEM CALIBRATION AND CHARACTERIZATION .................... 12 3.1 Source spectrum normalization........................................................................ 12 3.2 Dispersion prism description and evaluation line ............................................ 13 3.3 LCD scanning method ..................................................................................... 15 3.4 Evaluation of line response and axial resolution ............................................. 16 CHAPTER 4: EXPERIMENTAL RESULTS .................................................................. 18 4.1 50 μm step standard ......................................................................................... 18 4.2 Onion epidermis 3D imaging ........................................................................... 19 CHAPTER 5: CONCLUSION ......................................................................................... 20 REFERENCES ................................................................................................................. 21

6

LIST OF FIGURES

Chapter 2 Fig. 1. Schematic diagram of confocal system ················································ 11

Chapter 3 Fig. 2. Spectrum of plasma lamp light after passing through two LCDs ·················· 12 Fig. 3. Zemax simulation on Amici prism ····················································· 14 Fig. 4. Point array pattern and evaluation lines ··············································· 16 Fig. 5. Evaluation line response curve ························································· 17

Chapter 4 Fig. 6. 50 μm step standard measurement ····················································· 18 Fig. 7. Onion epidermis 3D imaging ··························································· 19

7

Abstract:

A chromatic confocal microscopy system in combination with two liquid crystal display (LCD)

panels is proposed and demonstrated for surface profiling. The major advantage of this system is

no mechanical translation is needed for three-dimensional (3D) imaging. The axial scanning is

realized thanks to the chromatic aberration in the objective, whereas the lateral scanning is

achieved by turning on different pixels on LCDs. Chromatic aberration of objective lens is used

to provide wavelength-to-depth coding and decoding is realized by using a dispersion prism.

System performance is validated with a 50 μm step standard and the capability of 3D imaging is

demonstrated with an onion epidermis.

8

Chapter 1: Background

Confocal microscopy [1] is a powerful tool to measure object surface and three-dimensional

volume structures, owing to its optical sectioning ability [2-4]. A traditional confocal system

requires point by point scanning in both lateral (x,y) and axial (z) directions, therefore, the

scanning speed is relatively slow. To improve the scanning speed, slit confocal microscopy

system [5] and spinning-disk confocal microscopy system [6] were developed, but mechanical

scanning is still required for lateral and axial scanning.

Chromatic confocal microscopy [7-9] was proposed to avoid axial mechanical scanning, taking

advantages of wavelength-to-depth coding ability of chromatic aberration. Using the objective

with uncorrected axial chromatic aberration, different wavelengths are focused at different

depths. Depth information can be easily collected without any axial mechanical scanning. The

maximum depth scan range is determined by bandwidth of light source and the dispersion ability

of objective lens. A broader bandwidth light source and strong dispersion objective lens provide

longer depth scan range. Xenon lamp or a supercontinuum laser is often used to achieve large

depth scan range [10,11].

Slit scan method and spinning disk method in combination with chromatic confocal principle

were developed to further increase the scanning speed [12,13]. However, lateral mechanical

scanning is still necessary. In 2000, a non-translational confocal system using digital micromirror

device (DMD) was proposed to avoid any mechanical scanning. Lateral scanning was achieved

by switching DMD mirror states, and a tunable laser was used to achieve wavelength scanning

[14]. Botvinick el al [15] and Wang et al [16] also used DMD to avoid lateral mechanical

scanning, but no axial scanning process was mentioned in their work.

9

Recently, Hillenbrand et al incorporated a dispersion prism into chromatic confocal microscopy

system [17]. The dispersion prism is used to analyze the spectrum of all lateral channels at the

same time, therefore any axial scanning is avoided. However, a movable pinhole array is still

needed to cover the entire field of view.

In this paper, we propose a simple chromatic confocal system without moving component using

two liquid crystal display (LCD) panels to create a virtual movable illumination pinhole array

and the corresponding detection pinhole array. The confocal system would be described in

Section 2, and system characteristics are evaluated in Section 3. In Section 4, experimental

measurements of a 50 μm step standard and an onion epidermis are presented.

10

Chapter 2: Chromatic confocal system

The schematic diagram of the chromatic confocal system is shown in Figure 1. A plasma lamp

with a 450-650 nm bandpass filter (Semrock, FF01-550/200-25) is used as the light source. Light

from the light source is collimated by an achromatic doublet L1 (Thorlabs, AC254-100-A) and

passes through a linear polarizer P1, illuminates the illumination LCD with 1024 × 768 pixels

and then passes through a linear analyzer A1. A point array pattern is displayed on the

illumination LCD. By passing only the light at locations of displayed point array, the

illumination LCD works as the point source array. Light from the illumination LCD is collimated

by a camera lens L2 (f = 105 mm) and focused by a microscope objective L3 (10X, NA = 0.1)

onto the object. The f-number of L2 has been set as 5.6. Owing to remaining chromatic

aberration in the objective, different wavelengths are focused onto different depths onto the

object. Light reflected from the object is collimated by the objective L3 and focused by the

doublet L2 onto the detection LCD, which is conjugate with the illumination LCD. A linear

polarizer P2 is placed before and a linear analyzer A2 after the detection LCD. The same point

array pattern is displayed on the detection LCD, and detection LCD works as detection pinhole

array. For a single wavelength, out-of-focus light is rejected by those pinholes. Light passing

through the detection LCD is collimated by an achromatic doublet L4 (Thorlabs, AC254-050-A-

ML). A custom-made dispersion prism is used to disperse different wavelengths into different

propagation directions. For each detection pinhole, different wavelengths are focused by an

achromatic doublet L5 (Thorlabs, AC254-050-A-ML) onto a gray-scale detector (pco.edge 5.5,

2560 × 2160 pixel resolution, 6.5 μm pixel pitch) at different locations along a line determined

by the dispersion prism. This “evaluation line” represents spectrum of the light reflected from the

object [17]. By analyzing all evaluation lines formed on the detector, we can get depth

11

information of corresponding points on the object. By changing point array pattern displayed on

two LCDs sequentially, lateral scan is achieved without any mechanical translation. Note that

two LCDs are connected to the same PC, therefore point array pattern displayed on two LCDs

are exactly the same and can be changed synchronously.

Fig. 1. Schematic diagram of confocal system.

12

Chapter 3: System calibration and characterization

3.1 Source spectrum normalization

Due to the light filter nature of pinholes, a multi-point scanning confocal system like scanning-

disk confocal system usually has a low light efficiency and suffers from low signal-to-noise

ratio. Using a plasma lamp is an effective way to decrease the exposure time and increase the

signal-to-noise ratio. The spectrum of plasma lamp light passed through two LCDs is measured

by a high resolution spectrometer (Ocean optics, HR2000+) and used to normalize the intensity

distribution on the evaluation lines. Note that spectrometer should be placed after two LCDs,

because LCD has different transmissions over different wavelengths. The measured spectrum is

shown in Figure 2. In our experiment, we only used the wavelengths between 505 nm and 650

nm. The reason is that the light intensity at 500 nm is relatively low. Normalizing with small

intensities would dramatically amplify the noises. This step guarantees that the signal-to-noise

ratio of the normalized intensity of the evaluation line is high enough to extract depth

information accurately.

450 500 550 600 650

wavelength in nm

0

2000

4000

6000

8000

10000

12000

14000

Inte

nsity

spectrum

Fig. 2. Spectrum of plasma lamp light after passing through two LCDs.

13

3.2 Dispersion prism description and evaluation line

In order to analyze spectrum of all lateral channels at the same time, a dispersion prism is used to

disperse different wavelengths to form evaluation lines on the detector. We used a double Amici

prism (DAP) designed by Mu et al [18] in our experiment. The main reason of using this design

is that this prism is free of distortion aberration. Distortion aberration introduced by a normal

dispersion prism would decrease quality of evaluation lines and hence confocal images of the

object. The configuration of double Amici prism is shown in Fig. 3(a). This prism consists of

three wedge elements. The first and third elements are made of H-LAK52 glass and second

element is made of H-ZF88 glass. The optimal angles of each elements are given in the diagram.

To evaluate the prism performance, we input the parameters into ZEMAX in an optical system

with image space F/# = 5 (corresponding image space NA = 0.1). A 50 mm focal length paraxial

lens is added after the prism to focus light onto the image plane without introducing any

aberration. Two fields are examined: the first field is on the optical axis (x = 0˚ and y = 0˚), the

second field is 70 percent of full field of a 50 mm focal lens with a diameter = 25.4 mm (x = 0˚

and y = 10.1˚). Layout of the system, ray aberrations and distortion in percent are shown in

figure 3(b), 3(c) and 3(d). Figure 3(b) shows the system layout. The blue lines represent light

rays at 10.1˚ field, and green lines represent light rays at 0˚ field. Figure 3(c) indicates the lateral

color of the system. The length of evaluation line is determined by the focus position difference

between 450 nm and 650 nm. If the prism is used with a 50 mm focal length imaging lens, the

length of evaluation line is 545 μm. This length would be used to design point array pattern

displayed on LCDs. Figure 3(d) shows the distortion aberration of the prism. The maximum

distortion is 0.3 percent and it is negligible.

14

In general, the length of evaluation line is determined by the spectrum of light source, the

dispersion prism, and the focal length of imaging lens. For a given prism and source bandwidth,

a longer focal length imaging lens would increase the length of the evaluation line. Note that

there is a trade-off between spatial and spectral resolution due to the fixed pixel number on the

detector. A longer evaluation line on the detector means more pixels are used to represent

spectrum of an object point, hence the spectral resolution is increased, but decreasing the spatial

resolution because smaller number of object points could be sampled. The axial resolution of the

chromatic confocal imaging system is not determined by the full width half maximum (FWHM)

of the longitudinal point spread function (PSF). Instead, it is determined by the length of the

evaluation line and the chromatic aberration in the objective.

Fig. 3. (a) Configuration of double Amici prism, (b) the layout of DAP in F/5 optical system, (c) positions of the rays with different wavelengths for the field of 10.1˚, and (d) distortion aberration in percent.

15

3.3 LCD scanning method

The LCD consists of 1024 × 768 pixels, the pixel size is 12 μm × 12 μm. The light can pass

through LCD only if some pixels are turned on. After passing through LCD, the polarization

direction of the incident beam rotates by 90 degrees. By turning on some pixels while turning

off the others, a point array pattern is generated on two LCDs, which can be used as multi-point

light source and detection pinhole array, respectively.

In our experiment, a single pixel is used as a pinhole. The structure of point array pattern is

designed to achieve zero cross-talk and maximum utilization of detector pixels. The spacing

between adjacent points on LCDs in x direction is 48 μm, and in y direction is 516 μm. A

schematic diagram of point array pattern is shown in Fig. 4(a). The spacing in y direction is

much larger than that in x direction, because a point light source passing through the prism

would spread a 504 μm evaluation line in y direction, therefore the spacing in y direction needs

to be large enough to avoid cross-talk. Note the measured length of evaluation line is smaller

than our simulation result, which is 545 μm. The reason is that intensity of wavelengths between

450-460 nm is very small, this part on evaluation line is overwhelmed by noises and hard to

discriminate. The 48 μm spacing in x direction guarantees that no cross-talk in x directions and it

provides some tolerances for system misalignment. An evaluation line pattern is shown in Fig.

4(b), each evaluation line is generated by a point source in Fig. 4(a) and its orientation is

determined by the dispersion prism.

To achieve lateral scanning, a point array pattern on illumination LCD is shifted in raster

scanning manner to cover the entire field of the objective. By shifting the corresponding pinhole

array in detection LCD, the confocal images can be taken without any mechanical translation.

16

The frame rate of a commercial LCD is 60 Hz, which limits the imaging speed of this confocal

system.

Fig. 4. (a) Point array pattern. (b) Evaluation lines.

3.4 Evaluation of line response and axial resolution

Two important characteristics of a chromatic confocal system are chromatic focus shift and axial

resolution. The chromatic focus shift is characterized as a function of focus position versus

wavelength, it is used to find surface profile of the object. A common method to characterize

chromatic focus shift is realized by using a tunable laser light source and a mirror object [14]. A

single pixel is opened on the optical axis and used as a point light source. For each tuned

wavelength, the mirror is scanned along z axis to find a depth which results in maximum

detected intensity. By finding all wavelength-to-depth mappings, chromatic focus shift

relationship can be determined.

17

To extract the depth information from the normalized evaluation line, we first established the

relationship between the imaging depth and the pixel location of the maximum normalized

intensity. We used a flat mirror as the object and attached it to a z-direction motorized translation

stage (Thorlabs, MT1-Z8). The mirror was scanned in a range of 100 μm, and at every registered

depth the location of maximum normalized intensity on the evaluation line was recorded. The

plot is shown in Fig. 5. The fitted function is shown on the top left of the figure. In our

experiment, the length of evaluation line is 77 pixels on the detector. We fitted the measured data

with a third order polynomial model.

The axial resolution is another important characteristic of the confocal system. As we mentioned

in Section 3.2, the axial resolution is determined by the length of evaluation line and the

chromatic focal shift. With a longer evaluation line, more pixels can be used to image the

spectrum, and therefore more precise the wavelength can be used to find the corresponding axial

depth. In our experiment, the scanning depth is 100 μm, the length of evaluation line is 77 pixels,

therefore the axial resolution is 1.3 μm.

Fig. 5. Depth versus the location of normalized maximum intensity on the evaluation line.

18

Chapter 4: Experimental results

4.1 50 μm step standard

To verify the surface profiling capability of our system, we first measured a 50 μm step standard

(VLSI, SHS-50.0QC). Wide-field image of this step standard is shown in Fig. 6(a). This image

was taken when all LCD pixels were turned on and the dispersion prism was removed. To obtain

confocal images, a point array pattern described in Section 3.3 was displayed and shifted across

all areas on the illumination LCD for lateral scanning. The scanning step is 1 pixel and the total

scanning step is 172. During the scanning process, the evaluation lines of all object points were

recorded. Depth of each object point was deduced from the evaluation lines using the method

discussed in Section 3.4. The measured surface profile is shown in Fig. 6(b). The measured

height is 50.86 μm and root mean square (RMS) error is 1.43 μm. The measurement error is less

than 1.3 μm, which is the theoretical axial resolution calculated in Section 3.4. RMS error is

slightly greater than 1.3 μm, this may be due to the measurement error at some object point

locations. This experiment demonstrates that the proposed chromatic confocal system is capable

to do surface profiling based on evaluation-line processing.

Fig. 6. (a) Wide field image of a 50 μm step standard. (b) Reconstructed surface profile.

19

4.2 Onion epidermis 3D imaging

To demonstrate three-dimensional imaging ability of the proposed system, we measured an

onion epidermis. We used the same lateral scanning procedure described in Section 4.1. The

evaluation lines of all object points were recorded and used to reconstruct 3D image of the onion

epidermis by finding maximum intensity on each evaluation line. Fig. 7(a) shows a reconstructed

image of the onion epidermis. In this figure, every object point is in focus. Fig. 7(b) provides

surface profile of this onion epidermis. The experiment further demonstrates the 3D imaging

capability of the proposed confocal imaging system without any moving component.

Fig. 7. (a) Reconstructed 3D image of an onion epidermis and (b) surface profile.

20

Chapter 5: Conclusion

Chromatic confocal microscopy is a promising tool to measure object surface and three-

dimensional volume structures. In this paper, we present a chromatic confocal system without

moving components for surface profiling. Two LCD panels are incorporated into the system and

used as multi-point light source and detection pinhole array. By changing point array pattern

displayed on two LCDs, lateral mechanical scan can be avoided. Axial scan is avoided by using

an objective lens with uncorrected chromatic aberration and a dispersion prism. The speed of this

system is limited by LCD frame rate. The axial resolution of our system is 1.3 μm and can be

further increased by using a longer focal length imaging lens. However, the trade-off between

lateral and axial resolution always exists and must be carefully balanced. The future research

would focus on how to improve lateral and spatial resolution at the same time and how to reduce

the number of necessary images to reconstruct the confocal image faster.

21

References:

1. M. Minsky, “microscopy apparatus,” U.S. Patent 3,013,467 (19 Dec. 1961). 2. D. K. Hamilton and T. Wilson, “Three-dimensional surface measurement using the confocal

scanning microscope,” Appl. Phys. B 27, 211–213 (1982). 3. K. Carlsson and N. Aslund, "Confocal imaging for 3-D digital microscopy," Appl. Opt. 26,

3232-3238 (1987). 4. D. K. Hamilton, T. Wilson, and C. J. R. Sheppard, “Experimental observations of depth-

discrimination properties of scanning microscopes,” Opt. Lett. 6, 625–626 (1981). 5. C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt.

35, 1169–1185 (1988). 6. M. Petran, M. Hadravsky, M. D. Egger, and Galambos, “Tandem-scanning reflected-light

microscope,” J. Opt. Soc. Am. 58, 661-664 (1968). 7. G. Molesini, G. Pedrini, P. Poggi and F. Quercioli, “Focus-wavelength encoded optical

profilimeter,” Opt. Commun. 49, 229-233 (1984). 8. M. A. Browne, O. Akinyemi, and A. Boyde, “Confocal surface profiling using chromatic

aberration,” Scanning 14, 145–153 (1992). 9. M. Maly and A. Boyde, “Real-time stereoscopic confocal reflection microscopy using

objective lens with linear longitudinal chromatic dispersion,” Scanning 16, 187–192 (1994). 10. K. Shi, P. Li, S. Yin, and Z. Liu, "Chromatic confocal microscopy using supercontinuum

light," Opt. Express 12, 2096-2101 (2004). 11. R. J. Garzón, J. Menese, G. Tribillon, T. Gharbi, and A. Plata, “Chromatic confocal

microscopy by means of continuum light generated through a standard single-mode fibre,” J. Opt. A: Pure Appl. Opt. 6, 544–548 (2004).

12. P. C. Lin, P. Sun, L. Zhu and Y. Fainman , “Single-shot depth-section imaging through chromatic slit-scan confocal microscopy,” Appl. Opt. 37, 6764-6770 (1998).

13. H. Tiziani and H. Uhde, "Three-dimensional image sensing by chromatic confocal microscopy," Appl. Opt. 33, 1838-1843 (1994).

14. S. Cha, P. C. Lin, L. Zhu, P. Sun and Y. Fainman, “Nontranslational three-dimensional profilometry by chromatic confocal microscopy with dynamically configurable micromirror scanning,” Appl. Opt. 39, 2605-2613 (2000).

15. E. L. Botvinick, F. Li, S. Cha, D. A. Gough, Y. Fainman, and J. H. Price, “In-vivo confocal microscopy based on the Texas Instruments digital micromirror device,” in Optical Diagnostics of Living Cells III (2000).

16. Q. Wang, J. Zheng, K. Wang, K. Gui, H. Guo, and S. Zhuang, "Parallel detection experiment of fluorescence confocal microscopy using DMD," Scanning 38, 234 (2016).

17. M. Hillenbrand, R. Weiss, C. Endrödy, A. Grewe, M. Hoffmann, and S. Sinzinger, "Chromatic confocal matrix sensor with actuated pinhole arrays," Appl. Opt. 54, 4927-4936 (2015).

18. T. Mu, S. Pacheco, Z. Chen, C. Zhang, and R. Liang, “Snapshot linear-Stokes imaging spectropolarimeter using division-of-focal-plane polarimetry and integral field spectroscopy,” Sci. Rep. 7, 42115 (2017).