Practical Spectral Photography

Ralf Habel1

Michael Kudenov2

Michael Wimmer1

Institute of Computer Graphics and AlgorithmsVienna University of Technology1

Optical Detection LabUniversity of Arizona2

Ralf Habel 2

Motivation

Spectroscopy is most important analysis tool in all natural sciences

Astrophysics, chemical/material sciences, biomedicine, geophysics,…

Industry applications:Mining, airborne sensing, QA,…

In computer graphics:Colors

Material reflectance

Spectral/predictive rendering

…

Ralf Habel 3

Spectral Imaging

Records image at narrow wavelength bandsIn visible range not only RGB (3 channels)but many more (6-400 channels)

Result: 3D data cube2 spatial image axis

1 wavelength axis

Ralf Habel 4

Spectral Imaging

Usually done with highly specialized devicesMany methods to build devices

Scanning slits, rotating mirrors, special sensor, filters, prisms, …

Usually scan along one of the data cube axis

All very costly due to opto-mechanical components

“Simplest” spectral imager:Camera + band filters

Requires switching of filters

Limited in number of bands

Ralf Habel 5

Motivation

Why not use consumer cameras and equipment for spectral imaging?

High quality, very sensitive

Highly accurate lenses

Practical Constraints:No camera modification

No lab/desktop/optical bench setup

No expensive components

Ralf Habel 6

CTIS Principle

Computed Tomography Image SpectrometerDiffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):

Ralf Habel 7

CTIS Principle

Computed Tomography Image SpectrometerDiffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):

Sensor records projections of 3D data cube All information needed is recorded in one image

“Snapshot” spectrometry

Challenge is to reconstruct 3Ddata cube from projections

Tomographic rec. with ExpectationMaximization

More details in paper

Ralf Habel 8

CTIS Principle

Ralf Habel 9

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Ralf Habel 10

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Collimating lens makes light parallel

Ralf Habel 11

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Collimating lens makes light parallel

Diffraction grating creates projections

Ralf Habel 12

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Collimating lens makes light parallel

Diffraction grating creates projections

Re-imaging lens focuses on sensor

Ralf Habel 13

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Collimating lens makes light parallel

Diffraction grating creates projections

Re-imaging lens focuses on sensor

Ralf Habel 14

CTIS Optical Path

Built with:Drain pipe & duct tape

50mm, 17-40mm and macro lens

Diffraction gel ($2 per sheet) in gel holder

Ralf Habel 15

CTIS Camera Objective

Ralf Habel 16

CTIS Camera Objective

Ralf Habel 17

HDR Image Acquisition

No overexposed pixels allowed

Projections (diffractions) weaker than center image

Avoids noisy signal where camera response is weak

Ralf Habel 18

Spatial Wavelength Calibration

Mapping from 3D data cube into projectionsLaser pointers (red, green and blue) with known wavelengths shot through a diffusor and pinhole

Monochromatic point light source

Pictures of pinhole give mapping of one voxel in 3D data cube

All other projections valuesinterpolated/extrapolated

Ralf Habel 19

CTIS Principle

Ralf Habel 20

Spatial Wavelength Calibration

Ralf Habel 21

Spectral Response Calibration

Spectral response of the diffraction grating + RGB sensor for red, green and blue

Picture of light source with continuous known spectrum

We use calibrated halogen lamp

Ralf Habel 22

Spectral Photography Results

Take HDR picture with CTIS camera objective

Reconstruct 3D data cube for red, green and blue image color channels

Mapping from spatial calibration

Combine RGB spectral response of each pixel to true spectrum with spectral de-mosaicking

Mapping from spectral response calibration

Ralf Habel 23

Spectral Photography Results

Protoype data cube resolutions:

120x120 pixels4.59 nm (54 channels)

Accuracy reduced in high blue and low reds dueto color filters

Slight Expectation Maximization reconstruction artifacts

Nowhere near possible optimum!

Ralf Habel 24

Spectral Photography Results

Ralf Habel 25

Spectral Photography Results

Ralf Habel 26

Future

Better CTIS objectiveDrain pipes and duct tape have their limits…

Optimized optical path and components

More compact/integrated device

Increase data cube resolution/accuracy:Structured aperture

Digital holography – form diffraction/projections in any way

Better solutions to tomographic reconstruction

Is active research in optics

No vision based approach yet!

Ralf Habel 27

Future

Turning mobile devices into spectrometers - consumer spectroscopy?

8 MP high sensitivity sensors

HDR capabilities

Very low cost!

“Snapshot” capability: Spectral movies with consumer cameras?

Not only good for computer graphics:Blood sample analysis

Water contamination analysis

As part of a TricorderTM

Ralf Habel 28

Practical Spectral Photography

Thank You!