24 IEEE Spectrum | May 2004 | NA

ILLU

STR

ATIO

N: J

OH

N M

AC

NEI

LL

PH

OTO

GR

AP

H: M

ICR

OV

ISIO

N IN

C.

+COVER STORY

IN THE EYEOF THE

BEHOLDER

IN THE EYEOF THE

BEHOLDER

Our window into the digital universe has longbeen a glowing screen perched on a desk. It’scalled a computer monitor, and as you stare atit, light is focused into a dime-sized image onthe retina at the back of your eyeball. The retina

converts the light into signals that percolate into your brainvia the optic nerve.

Here’s a better way to connect with that universe: elim-inate that bulky, power-hungry monitor altogether bypainting the images themselves directly onto your retina.To do so, use tiny semiconductor lasers or special light-emitting diodes, one each for the three primary colors—red, green, and blue—and scan their light onto the retina,mixing the colors to produce the entire palette of humanvision. Short of tapping into the optic nerve, there is nomore efficient way to get an image into your brain.

The advantages, at least for some viewing situations,would be overwhelming. Scanning the light into only oneof your eyes, for instance, would allow images to be laidover your view of real objects, giving you an animated,X-ray–like glimpse of the simulated innards of some-thing—a car’s engine, say, or a human body. Alternatively,scanning slightly different images into each eye could render

grippingly vivid three-dimensionalscenes with pure, jewel-like spectral col-ors. Gamers could experience a heightenedsense of reality that liquid-crystal-display gog-gles could never provide, because the laser orlight-emitting diode system could dynamicallyrefocus to simulate near and distant objects withutter realism.

Best of all, the system would waste essentially nophotons, so it would be fantastically efficient and verywell suited to the low-power requirements of mobiledevices. In round numbers, lasers or LEDs would use hun-dreds of times less power than a small LCD screen typi-cal of a subnotebook or handheld personal digital assistant.Imagine a cellphone or a PDA with a small, cameralikeviewfinder that, by stimulating your retina when peeredinto, would show you an image rich in color and detail.The image would appear to your brain as a large, brightlylit display screen perhaps 65 centimeters away, whichcould be reconfigured quickly from, say, a traditional,boxy 4:3 format to the widescreen 16:9 format.

The forerunners of such systems, known asscanned-beam displays, are just now hitting the

Scanning light beams to the retina could revolutionize displays for everything from cellphones to games BY JOHN R. LEWIS

Lens

Pupil expander

Mirror

MEMS scanning platform

LensLasers (red, green, blue)

Light beam

Reflector

DIRECT VIEW: In this scanned-beam display headset

[left], the viewer sees an image when modulated

signals from laser diodes sweep across the retina.

A microcontroller in the visor selects the image from

the view memory [top] and passes it to digital-to-

analog converters. These produce signals that control

lasers—red, green, and blue—for a full-color display.

The modulated light passes to a tiny scanning mirror

and then to a pupil expander that allows for eye

movement by enlarging the image. Next, the image

is reflected into the eye and onto the retina. This

illustration is based on a future version of the Nomad

Expert Technician System.

One commercial version of a monochrome Nomad

[immediately above] produces bright red instructions

from a repair manual that seem to float in the same

plane in which a technician is working.

Wirelessinterface

Userinterface

Digital-to-analog

converter(DAC)

Analog-to-digital

converter

Electronic controls

To MEMSscanner

To lasersDAC

RedGreenBlue

Viewmemory

Micro-controller

Processors and memory

May 2004 | IEEE Spectrum | NA 25

26 IEEE Spectrum | May 2004 | NA

PR

ECED

ING

PA

GE:

ILLU

STR

ATIO

NS

: JO

HN

MA

CN

EILL

; A

LL P

HO

TOG

RA

PH

S: M

ICR

OV

ISIO

N IN

C.

market. They are moving into several industries, includingautomotive service, to help service technicians keep track of thehuge and ever-changing reams of repair data and display it pre-cisely where and when they need it—in the service bay, while theyare working on a car. This first-generation system, from the com-pany I work for, Microvision Inc. of Bothell, Wash., was introducedto auto dealers earlier this year at the National Automobile DealersAssociation Convention & Exposition in Las Vegas. The systemis built around a lightweight display mounted on a baseball capor visor [see diagram, “Direct View”]. In the current version, awireless computer with a touch-pad control is worn on the belt.

Like a high-tech monocle, a clear, flat window angled in frontof the technician’s eye reflects scanned laser light to the eye.That lets the user view automobile diagnostics, as well as repair,service, and assembly instructions superimposed onto the fieldof vision. The information that the device displays comes froman automaker’s service-information Web site through a com-puter running Microsoft Windows Server 2003 in the dealer-ship or repair shop. The data gets to the display via an ordinaryIEEE 802.11b Wi-Fi network, and all the technicians in the serv-ice center are able to access different information simultane-ously from one server.

Those of you who remember the stern warnings printed on theside of the lasers in your school physics lab are probably won-dering about the safety of aiming laser light directly into theeye. To ensure that its device is safe, Microvision applied rigor-ous safety standards from the American National StandardsInstitute, Washington, D.C., and the International ElectrotechnicalCommission, Geneva, derived from years of studying the effectsof light on the eye. Laser light can be harmful because its beamis intense, capable of concentrating its power in a tiny area of inci-dence. This could be a problem if a fixed beam—as opposed to ascanned beam—were allowed to dwell on just one spot. We ensurethat the retina is never overwhelmed by limiting the power of thelaser light entering the eye to about a thousandth of a watt andusing a high-reliability interlock circuit that turns on the laseronly when the beam is scanning. Furthermore, because this verylow-power light is continuously scanned onto the retina, its energyis dispersed over an area hundreds of thousands of times largerthan a single spot of an incident beam.

Even at this very low power level, the monochrome system now

being marketed, called the Nomad Expert Technician System,delivers images that are bright enough, and in a color distinctenough—a vivid red—that they can easily be seen over the back-ground, even outdoors [see “Direct View”]. A test of the Nomadat the American Honda Motor Co. training center in Torrance,Calif., showed that skilled service technicians performed com-plex repair procedures in 39 percent less time, on average. Even athalf that efficiency gain, a dealership would realize a net returnon investment of US $2292 per technician per month, accordingto the Honda study. Honda has announced its intention to buy3800 Nomad systems, which retail for $3995 each. Microvision hasheld trials with dealerships of many other leading automakers,including GM, DaimlerChrysler, and Volvo.

Much as mechanics do, medical doctors fix extremely complexmachinery—the human body. Surgeons at the Wallace-KetteringNeuroscience Institute, the Baylor College of Medicine, and theCleveland Clinic Foundation’s Minimally Invasive Surgery Centerhave tested Microvision’s see-through, or augmented-vision, laser-based display. As they operate, the surgeons are viewing vitalpatient data, including blood pressure and heart rate. And in suchprocedures as the placement of a catheter stent, overlaid imagesprepared from previously obtained magnetic resonance imaging orcomputed tomography scans assist in surgical navigation.

Several military units, including the U.S. Army’s StrykerBrigade, are using adaptations of the system. The commander ofa Stryker, an eight-wheel light-armored vehicle, can view itsonboard battlefield computer with a helmet-mounted daylight-readable display. This enhances the commander’s ability to observethe surroundings, choose the optimum path, command the vehi-cle, and use tactical information advantageously. Other militaryapplications include a series of prototype helmet-mounted dis-plays developed with the U.S. Army and Boeing Co. of Chicago.Currently in the initial stages of flight-testing, the system couldbe a relatively inexpensive way to provide utility- and attack-helicopter pilots with a digital display of the battle space.

The military gave scanned-beam technology its start in the 1980sas part of the U.S. Air Force’s Super Cockpit program. Its team,led by Thomas A. Furness III, now at the University of Washington,Seattle, produced helmet-mounted displays with an extremely largefield of view that let fighter pilots continuously see vital data suchas weapons readiness. The displayed information moved with thepilot’s head, giving him an unobstructed view of what was goingon in front of him and helping him to distinguish friend from foe.

Now offshoots of that technology may even wind up in suchmass-market products as digital cameras, where scanned-beamdisplays provide better image quality at lower power and costthan liquid-crystal-on-silicon and organic LED displays. Insteadof using lasers, which provide the power necessary for bright,see-through, head-up displays, the non-see-through Microvisiondisplays that require full color at low power and low cost—desir-able for viewfinders and near-to-eye cellphone displays—relyon LEDs. Microvision is working with Canon Inc., Tokyo, on anear-to-eye microdisplay with several advantages over conven-tional optical and LCD electronic viewfinders. Linking themicrodisplay to the camera’s image sensor would give digitalcameras the full viewfinder capabilities of a premium single-lens reflex camera. The user could preview a smooth, high-resolution, full-color image, allowing critical focus-control anddepth-of-field adjustments to be made [see diagram, “What YouSee Is What You Get”].

There are just four primary components of a scanned-beamdisplay: electronics, light sources, scanners, and optics. Yet witha modular approach, these simple elements can be combined

IMAGE PAINTER: The chiplike brush that paints an image, the MEMS scanner, con-

sists of a 1.5-millimeter-diameter scan mirror and the torsion flexures that secure it.

ILLUS

TRATIO

N: JO

HN

MA

CN

EILL

May 2004 | IEEE Spectrum | NA 27

to yield many different products [see dia-gram, “Direct View,” again].

Electronics acquire and process signalsfrom an image or data source, such as aWeb page or video camera. The processedsignals contain information for the intensityand mix of color that best renders the intendedimage at each location that will be scanned, in sequence.These values are the individual picture elements—pixels—thatmake up the image. This information is stored in memory untilneeded, when the data pass through a digital-to-analog converterthat controls the light source. Once the image has been renderedinto memory, there is no need to recalculate it unless some-thing has changed. The data can simply be replayed from mem-ory, a feature that can be exploited to cut costs or save power.

Depending on the application and cost and size requirements,we can use single color or multiple low-power solid-state lasers,laser diodes, or LEDs as the light source. In the case of a full-colorelectronic viewfinder display on a camera where low cost andpower consumption are critical, modulated red, green, and blueLEDs produce color pixels of varied intensities to generate a com-plete palette of colors and shades.

If the light source is the paint, Microvision’s proprietary micro-electromechanical systems (MEMS) biaxial scanner is the brushthat applies the image to the retina. The scanner’s main compo-nent is a mirror 1.5 millimeters in diameter that rapidly sweeps thelight beam horizontally to position the pixels in a row, also mov-ing the beam downward, to draw successive rows of pixels. Thisprocess continues until an entire field of rows has been placed anda full image appears to the user—quite similar to the process ina regular cathode-ray television, in which the magnetic deflectioncoils direct the electron beam to scan the phosphor-coated screen.But while a conventional display can create jagged edges on imagesbecause the pixels are fixed onto screen hardware, a scanned-beamdisplay has no hard pixels: the continuously scanning beam cre-ates a much smoother image.

For applications in whichthe scanned-beam display is to beworn on the head or held closely to the eye, we need to deliver thelight beam into what is basically a moving target: the human eye.Constantly darting around in its socket, the eye has a range ofmotion that covers some 10 to 15 mm. One way to hit this targetis to focus the scanned beam onto an optical element called an exitpupil expander. When light from the expander is collected by alens, and guided by a mirror and a see-through monocle to the eye,it covers the entire area over which the pupil may roam. For appli-cations that require better image quality using less power, we candispense with the exit pupil expander altogether either by usinga larger scan mirror to make a larger exit pupil or by actively track-ing the pupil to steer light into it.

THE SIMPLICITY AND ELEGANCE of the scanned-beam conceptbelies the underlying complexity of the enabling advancementsover the past four decades in scanning, light-source, and image-processing technologies.

Early on, Microvision researchers identified the scanner asthe crucial element in this emerging technology. Eight years ago,we scanned using a polished metal plate that combined the scanmirror and a stiff torsion spring that had a resonance of about20 kilohertz. When driven by magnetic coils, the plate scans in a

WHAT YOU SEE IS WHAT YOU GET: The basic

technology used in the Nomad head-up display is

being adapted to smooth the viewfinder image on

a digital camera [right]. The picture captured by a

charge-coupled device chip is processed to con-

trol the intensities of red, green, and blue

light-emitting diodes, as well as the motion of a

MEMS scanner. The image is next focused

through the viewfinder and onto a user’s retina.

A blown-up detail of a test image [top left]

shows the difference between the hard pixels of a

digital display [top middle] and the much

smoother appearance of a scanned-beam display

[top right].

Mirror

Lens

Viewfinder

Lens

Lens

Charge-coupled device

Electronics(user interface

processor)

Edge-emitting LEDs(red, green, blue)

MEMS scanningplatform

ORIGINAL DIGITAL DISPLAY SCANNED-BEAM DISPLAY

AREA ENLARGED

28 IEEE Spectrum | May 2004 | NA

large, twisting, resonant motion. With that proof of principle inhand, we developed a MEMS version of the scanner. MEMS areelectromechanical devices that are photolithographically definedon a silicon wafer, much as integrated circuits are made, and inquantities of more than 100 per wafer.

A typical MEMS scanner today measures about 5 mm across, witha 1.5-mm-diameter scan mirror capable of motion on two scan axessimultaneously [see photo, “Image Painter”]. Using MEMS allowsus to integrate the scanner, coil windings, and angle-sensor func-tions all on one chip. Such a scanner provides SVGA (800-by-600)equivalent resolution at a 60-hertz refresh rate and is now inproduction and in products. We expect a higher performance perscanner as we more fully exploit the basic advantages of MEMS,which include the potential of very low costs in small packages. Inaddition, multiple scanners could provide higher-resolution imagesby each providing full detail in a tiled subarea. Eventually, costswill become low enough to make this practical, allowing thescanned-beam approach to surpass the equivalent pixel count ofany other display technology.

While the MEMS scanner is arelatively recent development, thelaser, another indispensable ele-ment of the scanned-beam display,traces its origins back to 1960 andprovides a compact source of spec-trally pure, focused, virtually noise-free light. Microvision uses laserlight sources in many of its see-through products because our cus-tomers’ applications demand display performances with color-gamut and brightness levels far exceeding the capabilities of flatpanel displays, notebook displays, and even higher-end desktopdisplays. For today’s commercial products, only red laser diodesare small enough, efficient enough, and cheap enough to use insuch see-through mobile devices as Nomad. Blue and green diode-pumped solid-state lasers are still too expensive for bright, full-color, head-up or projection displays for mainstream markets, butthat could change soon. In the mid-1990s Shuji Nakamura of NichiaChemical Industries Ltd. (now Nichia Corp., Tokushima, Japan)demonstrated efficient blue and green LEDs, and then blue laserdiodes made of gallium nitride. When these designs and mate-rials are extended to green laser diodes, we’ll be able to buildbright, full-color see-through displays.

On another front, Microvision recognized that the totalamount of light that enters your eye from a desktop display isactually less than a microwatt, and that this is small com-pared with what an LED can supply. Although the powerrequired is low, the light must be collected and focused downto a pinpoint—easy to do with a laser, but not so easy withan LED. A scanned-beam display placed near the eye, such asa camera viewfinder, wastes little light, especially if it does nothave to overcome a background scene. Even so, we’ve neededadvances in LED technology to further concentrate the lightcoming from these devices.

Enter the edge-emitting LED. Unlike conventional LEDs,which emit light from the surface of the chip, an edge-emittingLED has a sandwich-like physical structure similar to that ofan injection-laser diode, but it operates below the lasingthreshold. These LEDs emit incoherent beams of light that,while not so fine as a laser’s beam, provide a tenfold increasein brightness. We also use multiple inexpensive surface-emittingLEDs, each contributing a portion of the overall power, toachieve high brightness. Further performance improvements

of LED materials driven by huge investments aimed at gen-eral lighting applications will increase the brightness and rangeof applications for scanned-beam displays based on green andblue gallium nitride devices and aluminum gallium indiumphosphide red LEDs.

On top of improvements in LEDs, lasers, and MEMS, memorydensity and processor power are expected to double every two years,translating directly into better performance from our displays.Increased memory and computational capacity boost the updaterate of the light source and refine its control, increasing resolu-tion even further.

IN ADDITION TO DISPLAYING IMAGES, the scanned-beam tech-nology can capture them. In a display, the data channel through adigital-to-analog converter controls the light source to paint a pic-ture on a blank canvas. In image capture, the light source is steadilyon, and the data channel looks at the reflections from the objectthrough an analog-to-digital converter connected to a photodiode.

The light source, beam optics, andscanner are essentially the same inboth applications.

Exploiting this versatility, wedeveloped a design for an endo-scope, the long, slender medicalinstrument that is used for exam-ining the interior of a bodily organor performing minor surgery. Com-posite red, green, and blue lightfrom lasers travels down a single-

mode fiber to the far tip of the endoscope. There at the tip, asimple lens collects the light into a single beam that then cul-minates in a fine point. A MEMS scanner directs the fine pointof light over an area that is 10–100 mm distant from the tip.Reflected light collected by fibers and conducted back to detec-tors contains the information about objects encountered. Thedetected light is digitized and, with software, reconstructed intoan image of the object encountered by the scanned beam.

Microvision just completed a study showing that a 2.5-mm-diameter MEMS chip scanning at large angles would provideresolution as good as that of the leading endoscopes. ThisMEMS chip is smaller than the sensor chip that is used inCMOS or charge-coupled-device imagers. Small size, of utmostimportance for minimally invasive surgery, combined with sim-ple optics, results in a disposable endoscope probe. Such a probewould reduce the cost of medical procedures by saving on thetime and cost of sterilization while minimizing the risk ofcross-contamination.

Medical-device applications will take more time to developand to qualify for use. Meanwhile, to provide revenue and gainexperience in high-volume manufacturing, Microvision is apply-ing this rather exotic technology to the $1.8-billion bar-code-scanner market with the $99.95 Flic laser bar-code scanner, whichwe introduced in September 2002. The resolution and scan speedscan be much lower than those needed for display applications,so we can reduce costs by using a plastic multifaceted scan mir-ror operated by the energy harvested from pressing the scan but-ton. NCR Corp., Dayton, Ohio, has recently introduced the Flicscanner under its own label, called the RealScan Companion.Consumer applications will take longer to come to market, butwe expect that in the next five years, our displays will pop up incellphones and cameras, giving users an HDTV experience on thego, and at a fraction of the power, weight, and cost required bytoday’s devices. �

With green laserdiodes, we’ll be able tobuild bright, full-color

SEE-THROUGH displays