Adaptive Optics and the cone mosaic

• What can you use it for?

• Why is it interesting for color vision science?

• What is it, what are its limitations and where is it going?

Early AO for imaging cone mosaic

• Adaptive optics was first envisioned by Horace W. Babcock in 1953, but it did not come into common usage until advances in computer technology (1990).

• Adaptive optics (AO) is a technology used to improve the performance of optical systems by reducing the effect of wavefront distortions. It is used in astronomical telescopes and laser communication systems to remove the effects of Earth’s atmospheric distortion.

• Adaptive optics works by measuring the distortions in a wavefront and compensating for them with a device that corrects those errors such as a deformable mirror or a liquid crystal array.

Sodium beacons are created by using a laser specially tuned to 589.2 nanometers to energize a layer of sodium atoms which are naturally present in the mesosphere at an altitude of around 90 kilometers. This laser creates a wavefront reference source in order to correct atmospheric distortion of light. The distortions detected by the reflected light from this laser source from the mesophere is used to guide the wavefront distortion of the deformable mirror.

Laser Guide Stars:Adaptive optics (AO) systems require a wavefront reference source in order to correct atmospheric distortion of light (called astronomical seeing). Sufficiently bright stars are not available in all parts of the sky, which greatly limits the usefulness of natural guide star adaptive optics. Instead, one can create an artificial guide star by shining a laser into the atmosphere.Because the laser beam is deflected by astronomical seeing on the way up, the returning laser light does not move around in the sky as astronomical sources do. In order to keep astronomical images steady, a natural star nearby in the sky must be monitored in order that the motion of the laser guide star can be subtracted using a tip–tilt mirror. However, this star can be much fainter than is required for natural guide star adaptive optics, because it is only used to measure tip and tilt and all higher order distortions are measured with the laser guide star.Sodium beacons are created by using a laser specially tuned to 589.2 nanometers to energize a layer of sodium atoms which are naturally present in the mesosphere at an altitude of around 90 kilometers. The sodium atoms then re-emit the laser light, producing a glowing artificial star.

Schematic of an AO system

Schematic illustration of an adaptive optics (AO) system. Light is shown with dotted lines, control connections with dashed lines. The wavefront enters the AO system at the top. The light first hits a tip–tilt (TT) mirror and is then directed to a deformable mirror (DM). The wavefront is corrected and part of the light is tapped off by a beamsplitter (BS). The residual errors (due to system latency, finite number of actuators/sensors etc.) are measured by a wavefront sensor (Shack-Hartmann in this case) and the control hardware then sends updated signals to the DM and the TT mirror. The two filterwheels (FW1 and FW2) are used during calibration only.

-- Light coming through the atmosphere (and most other media) suffers dispersion, where different wavelengths of light are refracted by different amounts.

-- Shorter wavelengths are refracted more, and longer wavelengths are refracted less, so the image of an object which suffers a high refraction and associated dispersion actually consists of a series of images, each of a different wavelength, slightly shifted relative to each other, with the shorter wavelength images at higher altitudes, and the longer wavelength images at lower altitudes.

Atmospheric Chromatic Dispersion

When light hits the lens it is refracted, ie, it changes direction. As most people probably know just by looking at the rainbow, visible light is comprised of different wavelengths which are perceived as different colors. What happens in this situation is that the different wavelengths are bent differently by the lens, causing them to diverge (a phenomenon known as dispersion) and hit different planes, as shown in the following diagram:

The “adaptive” part

Measurements or pictures come out of the beam pointing to science.

The “adaptive” part

Measurements or pictures come out of the beam pointing to science.

Shack Hartmann wavefront sensor

Schematic illustration of a Shack Hartmann wavefront sensor using a so-called lenslet array. The perturbed wavefront coming in at the top (black solid lines) is imaged by an array of small lenses. Each of these lenses only samples a small portion of the full wavefront. The position of the image created by each of these lenses on the camera depends on the local slope of the wavefront (red dashed lines). Thus, by measuring this image displacement Δx for all lenslets, the local wavefront slopes can be obtained and the full wavefront can be reconstructed.

From Star maps or retinal mosaic maps:From extraterrestrial exploration to

intercorporeal exploration

Early AO for imaging cone mosaic (~1996)

• Main Problems: Eye motion & optical imperfections making imaging individual cone cells (typically 40–50 µm long; diameter between 0.5 to 4.0 µm, being smallest in the fovea. S cones are larger than the M/L cones. (fyi: a human hair width ranges 17-181 µm)

AO in retinal imaging

Distribution of cone cells in the fovea of a individual with normal color vision (left), and a color blind (protanopic) retina. The center of the fovea holds very few blue-sensitive cones.

AO in retinal imaging

Distribution of cone cells in the fovea of a individual with normal color vision (left), and a color blind (protanopic) retina. The center of the fovea holds very few blue-sensitive cones.

~2.5 dg.s

• ~1996. D. Willams et al. Using a Shack Hartmann wavefront sensor.

• In microscopy and retinal imaging systems it is used to reduce optical aberrations.

• Ocular aberrations are distortions in the wavefront passing through the pupil of the eye. These are the same aberrations diminish the quality of the image formed on the retina, sometimes necessitating the wearing of spectacles or contact lenses.

• In the case of retinal imaging, light passing out of the eye carry similar wavefront distortions, leading to an inability to resolve the microscopic structure (cells and capillaries) of the retina.

AO in retinal imaging

• Spectacles and contact lenses correct "low-order aberrations", such as defocus and astigmatism, which tend to be stable in humans for long periods of time (months or years).

• While correction of these is sufficient for normal visual functioning, it is generally insufficient to achieve microscopic resolution.

• Additionally, "high-order aberrations", such as coma, spherical aberration, and trefoil, must also be corrected in order to achieve microscopic resolution. High-order aberrations, unlike low-order, are not stable over time, and may change with frequencies between 10 Hz and 100 Hz. The correction of these aberrations requires continuous, high-frequency measurement and compensation.

AO in retinal imaging

In optics (especially telescopes), the coma, or comatic aberration, in an optical system refers to aberration inherent to certain optical designs or due to imperfection in the lens or other components that results in off-axis point sources such as stars appearing distorted, appearing to have a tail (coma) like a comet. Coma is common in patients with decentred corneal grafts, keratoconus, and decentred laser ablations.Trefoil produces less degradation in image quality compared with coma of similar RMS magnitude.[6]

Taxonomy of Optical abberations

Plots of Zernike polynomials in the unit disk

In optics (especially telescopes), the coma, or comatic aberration, in an optical system refers to aberration inherent to certain optical designs or due to imperfection in the lens or other components that results in off-axis point sources such as stars appearing distorted, appearing to have a tail (coma) like a comet. Coma is common in patients with decentred corneal grafts, keratoconus, and decentred laser ablations.Trefoil produces less degradation in image quality compared with coma of similar RMS magnitude.[6]

AOSLO & AO-OCT• Adaptive Optics Scanning Laser Ophthalmoscopy (AOSLO)

- AO flood illumination and scanning laser ophthalmoscopy (SLO) have been used to study many properties of the cones, such as: arrangement and packing in normal and defective retinas; changes in reflectance over time; and sampling of the ocular image.

• AOSLO handles eye movement way better than gaze tracking tech.

• Uses real-time retinal image motion signals in combination with high speed modulation of a scanning laser. Show in good fixation stability Subjects to yield stimulus location accuracy averaged 0.26 arcminutes or approximately 1.3 microns, which is smaller than the cone-to-cone spacing at the fovea (2007. Arathorn, Yang, Vogel, Zhang, Tiruveedhula & Roorda.).

The next generation of AO is AOSLO & AO-OCT. These two new methods aim to address the original problems Williams and colleagues faced in early retinal AO.

AOSLO & AO-OCT

• AO was first attempted for SLO in the 1980s. This first attempt did not use wavefront-detecting technology with a deformable mirror but instead estimated aberrations through pre-measured factors such as astigmatism (not ideal).

• Now AOSLO the Shack-Hartmann wave-front detector for the apparatus produced images of the retina with much higher lateral resolution.The addition of microelectricalmechanical (MEMs) mirrors instead of larger, more expensive mirror deformable mirror systems to the apparatus made AOSLO further usable for a wider range of studies and for use in patients.[9]

AOSLO & AO-OCT• Scanning laser SLO utilizes horizontal and vertical scanning

mirrors to scan a specific region of the retina and create raster images viewable on a television monitor. While it is able to image the retina in real time, it has issues with reflections from eye astigmatism and the cornea. Eye movements additionally can confound the data from SLO.

“Optomap” -- your eyedoc.

The next generation of AO is AOSLO & AO-OCT. These two new methods aim to address the original problems Williams and colleagues faced in early retinal AO.Once the subject is properly placed, wavefront correction and imaging takes place. A laser is collimated and then reflected off of a beam-splitting mirror. As in confocal SLO, light must pass through both a horizontal and a vertical scanning mirror before and after the eye is scanned to align the moving beam for eventual retinal faster images of the retina. Additionally, the light is reflected off of a deformable mirror before and after exposure to the eye to diffuse optical aberrations. The laser enters the eye through the pupil to illuminate the region it has been focused onto and light reflected back leaves the same way. Light returning from the mirrors passes through the first beam splitter onto another beam splitter where it is directed simultaneously toward a photomultiplier tube (PMT) and toward a Shark-Hartmann wavefront sensor array. The light going toward the photomultiplier is focused through a confocal pinhole to remove light not reflecting off of the plane of interest and then recorded in the PMT. Light directed to the wavefront sensor array is split up by the lenslets in the array and then recorded onto a Charge-coupled device (CCD) camera for detection of optical aberrations. These aberrations are then subtracted from the images recorded in the PMT to vastly increase lateral resolution.[3][5][8][10]

AOSLO & AO-OCT

• S

When Longitudinal Chromatic Abberation is corrected, retinal images for all wavelengths (infrared, red, green) are in focus but offset laterally due to TCA.

Chromatic dispersion in AOSLO imaging. (A) When Longitudinal Chromatic Aberation LCA is corrected, retinal imagesfor all wavelengths (infrared, red, green) are in focus but offset laterally due to Transverse Chromatic Aberation TCA. Theimaged area of the retina is outlined on the fundus photograph (top). TCA offsets betweencolors are highlighted with example cone outlines. Dark regions in the AOSLOmicrophotographs are caused by blood capillary shadows. Cone images are constructed byreferenced averaging of ~150 individual video frames, and are individually normalized fordisplay purposes. Scale bar: 2 deg (top), 2 arcmin (bottom). (B) Schematic explanation ofchromatic offsets in AOSLO imaging. Two aligned input beams (red and green) reach adispersing lens and land at different locations on the retina. The reflected light, passing backthrough the lens, is realigned into one beam and is captured by two imaging devices. Althoughthe effects of TCA are cancelled on the second pass through the lens and the outward beamsare re-aligned, one can determine how much dispersion occurred between the two beams,because the retina being imaged has spatial structure. This principle holds for our threewavelength AOSLO setup.

AOSLO & AO-OCTTwo aligned input beams (red and green) reach a dispersing lens and land at different locations on the retina.

The reflected light, passing back through the lens, is realigned into one beam and is captured by two imaging devices.

Although the effects of TCA are cancelled on the second pass through the lens and the outward beams are re-aligned, one can determine how much dispersion occurred between the two beams, because the retina being imaged has spatial structure. This principle holds for a 3 wavelength AOSLO setup.

AOSLO Harmening, Tiruveedhula, Roorda, Sincich 2012. BioMed.Opt.Ex.

• Adaptive Optics - Optical Coherence Tomography (AO-OCT) (~2003)

• The combination of AO with optical coherence tomography (OCT) has permitted cones to be resolved in depth as well, facilitating study of cones’ three dimensional structure, waveguiding properties, polarization behavior, and volumetric changes over time.

• The technical benefits of adding AO to OCT (increased lateral resolution, smaller speckle, and enhanced sensitivity) increase the imaging capability of OCT in ways that make it well suited for three-dimensional (3D) cellular imaging in the retina. AO-OCT systems provide ultrahigh 3D resolution (3 × 3 × 3 µm³) and ultrahigh speed (up to an order of magnitude faster than commercial OCT). AO-OCT systems have been used to capture volume images of retinal structures, previously only visible with histology, and are being used for studying clinical conditions.

AOSLO & AO-OCT

emphasis is structural imaging

AOSLO & AO-OCT

AO-OCT • To make an OCT scan of the retina, a beam of light is split: one

portion is scattered off the retina while the other is the reference beam. The beams are interfered, and the resulting phase information used to procure a precise measurement of a sample’s position. (dealing with in vivo moving eyes). Instead of measuring the phase of a single interference pattern, it measures phase differences between patterns originating from two reference points within the retinal cells: the top and bottom of the outer segment.

• E.g., Using this hidden phase information Ravi et al (2012) measured microscopic changes in hundreds of cones, over a matter of hours, in two test subjects with normal vision. They found they could resolve the changes in length down to about 45 nm size, which is just slightly longer than the thickness of a single one of the stacked discs that make up the outer segment.

• FYI: Their work shows that the outer segments of the cone cells grow at a rate of about 150 nm per hour. (Ravi S. Jonnal et al., Biomedical Optics Express, Vol. 3, Issue 1, pp. 104-124 (2012)).

An interference filter or dichroic filter is an optical filter that reflects one or more spectral bands or lines and transmits others, while maintaining a nearly zero coefficient of absorption for all wavelengths of interest. An interference filter may be high-pass, low-pass, bandpass, or band-rejection.

Review: using AO Hofer et al. (2005) JOV.

Highly Variable Mosaicism in Human Retinas

L-RichM-Rich

Example of a retinal sensitivity map and retinal stimulus profile used to model the microstimulation of the mosaic.

Hofer H et al. J Vis 2005;5:5

©2005 by Association for Research in Vision and Ophthalmology

Flashes were presentedwithout aberration correction to a single location at1.25 deg retinal eccentricity.

Figure 5. Example of a retinal sensitivity map and retinal stimulusprofile used to model the microstimulation of the mosaic. L andM cones have been colored red and green to aid in their identification.The full width at half maximum of the retinal profile of thespot imaged with adaptive optics is smaller than the radius ofindividual cone inner segments near 1 deg.

Hofer et al. 2005.• They achieved AO corrected cone size (a major feat), but

uncontrolled eye motion made it impossible to do repeat assessment of stimulated cones, AND the “aim” of the stimulation was off because of chromatic abberation.

Figure 4. The color sensations reported by subjects when viewinga small spot of 550-nm light. At this wavelength only L and Mcones participate in detection. Shown are the percentages ofwhite and colored responses that were placed in each responsecategory, interpolated at 50% frequency of seeing. Percentagesfor BS are white, 56%; red, 42%; yellow-green, 0.7%; green,0.7%; blue-green, 0.3%; and blue, 0.8%. In addition to white,each subject used at least five different hue categories.

Measuring Color Vision on a Cellular Scale in an AdaptiveOptics Scanning Laser OphthalmoscopeSabesan, Tuten, Harmening, Carney, Klein, & Austin Roorda. Optometry, University of California, BerkeleyImaging and Applied Optics © OSA 2013

• They fix problems Hofer et al. and Brainard et al. encountered.• Problems? Special challenges arise when pursuing multi-

wavelength imaging of retinal tissue in vivo, because the eye’s optics must be used as the main focusing elements, and they introduce significant chromatic dispersion.

• Using sophisticated high-speed retinal tracking, Roorda and colleagues present an image-based method to measure and correct for the eye’s transverse chromatic aberrations rapidly, non-invasively, and with high precision. This method also addresses ubiquitous eye movements.

Measuring Color Vision on a Cellular Scale in an AdaptiveOptics Scanning Laser OphthalmoscopeSabesan, Tuten, Harmening, Carney, Klein, & Austin Roorda. Optometry, University of California, BerkeleyImaging and Applied Optics © OSA 2013 • They validate the technique against hyperacute

psychophysical performance and the standard chromatic human eye model.

• In vivo correction of chromatic dispersion enables confocal multi-wavelength images of the living retina to be aligned, and allows targeted chromatic stimulation of the photoreceptor mosaic to be performed accurately with sub-cellular resolution.

Circular stimuli at green 543 nm wavelength and measuring 0.6 arc-min in diameter were delivered to 3 retinal locations at 4 deg eccentricity and subjects were asked to report their color percepts.

Figure 1: Single cone color naming. A) Circular stimuli at green 543 nm wavelength and measuring 0.6 arc-min in diameter were delivered to 3 retinal locations at 4 deg eccentricity and subjects were asked to report their color percepts. 30 trials per location were tested. The color of each spot corresponds to the color percepts reported by the subject, while its x,y coordinate represents the relative location where it was delivered on theretina. Note that the shown location does not correspond to the actual cones that were stimulated in the experiment. B) Histograms show the number of responses reported for each color at the 3 locations.

Results: High-speed retinal tracking allowed repeatable targeted stimulation of the three retinal locations. Mean stimulus delivery errors of 0.5 arc-min - within the size of a single cone photoreceptor at 4 deg eccentricity. Repeatable color percepts arising from the same retinal location were measured psychophysically as shown in figure 1. The color of each spot corresponds to the color percepts reported by the subject, while its x,y coordinate represents the relative location where it was delivered on the retina. The histograms show the number of responses reported for each color at the 3 locations. Mean purity of color percepts, calculated as the percentage named either red or green, was 80 ± 3.33 %.

Future stuff AO may look at:

• Can we verify whether heterozygous opsin genotypes give rise to retinas with 4 distinct populations of cones?

Observe Four expressed cone types?

‘simulated’ 4-cone human retina Graphic from M. Neitz and J. Neitz (1998)

A remaining AO challenge? differentiating several different M- & L-photopigment variants:

L-alanine552nm

L-serine557nm

Figure from M. Neitz and J. Neitz (1998)

Future stuff AO may look at:

• ala Brainard et al. (2008): does a single cone’s percept also depend on it’s neighborhood network (or the types of cones in the local surround)?

• Waveguiding properties of the different cells and pigments -- are they uniform?

Future stuff AO may look at:

• Test the idea that highly biased L/M cone ratios in female CVD carriers, or in actual color deficients, tracks chromatic discrimination abilities.

• Ask if at isoluminance, relatively equal numbers of L & M cones optimize detection of any chromatic contrast.

• Ask, if cone ratios biased are does this favor detection in one chromatic direction only -- and therefore impair color discrimination in other directions?