Multispectral imaging. Applications - Vaasan yliopistolipas.uwasa.fi/~TAU/AUTO3090/Slides/lect_multispec.pdf · may show information invisible by a human eye. ... Multispectral imaging

Multispectral imaging.Applications

Vladimir Bochko Jarmo Alander

University of Vaasa

Vladimir Bochko Multispectral imaging. Applications 1/26

Contents

• Introduction.

• PCA.

• Skin wound classification.

• Dichromatic reflection model.

• Image colorization.

• Image fusion using image processing.

• Estimating spectral reflectance of painting.

• Electronic endoscope. Image fusion using variableselection.


Multispectral imaging


Why multispectral and hyperspectralimaging systems?

The color models is based on the CIE 1931 Standard Observer. TheStandard observer is assumed to integrate the light source power spectrumS(λ) and the spectral reflectance of the object R(λ) via three matchingfunctions x(λ), y(λ) and z(λ). There are problems with color imagingsystems composed by 3 channeles Red, Gree and Blue [5]:

• The spectral characteristics of digital cameras are different frommatching functions and differ from each other.

• Difficultes in reproduction images under particular illuminant if imagesare captured under different illuminant.

• Metamerism. Two different spectra may be reproduced as the samecolor at the particular illuminant and to be different colors at anotherilluminant.

The solution is multispectral imaging systems.


Visual and near-infrared images(VIS-NIR)

• A visible (VIS) and near-infrared (NIR) range are desirable in manyapplications.

• The VIS range is from 380 to 780 nm. The components are reproducedas grayscale images or an RGB color image.

• The NIR range is from 780 to 2500 nm. The components arereproduced as grayscale images or any three components can be takenand used as R,G,B components. i.e. pseudocolor reproduction.

• The NIR light penetrate deeper in materials, e.g. paint or tissue, andmay show information invisible by a human eye.

• In addition, the contrast agent may be injected and propagated intoblood or lymphatic vessels. For example, idocyanine green gives afluorescent light at 830 nm after the exciting illumination at 780 nm. Thefluorescent light highlight the vessels in the NIR image. This is used foranalysis in medicine.



• Professor Yoichi Miyake (left), Chiba University, Japan. Talk:Multispectral imaging and its application to medicine: The developmentof diagnosis system for life style diseases using color information,University of Vaasa, June, 2011.



• Mitsubishi 5-band spectral camera.

• Canon 8 ink jet printer.

• Spectral scanner by Mitsubishi Heavy Industry. The sensorrange 380-860nm, at 10nm.


Image reshaping for computation

• We reshape a 3D image using concatenation along columns or rows.This is needed for PCA, MLR, PCR and PLS. After computing forreproduction we reshape data to obtain the 3D image.

• In SOLO [9] it is called folded (left) and unfolded (right) images.

100

100

1Multispectralimage

44Com

pone

nts

Variables/ComponentsSa

mpl

es/P

ixel

s10000

2D matrix

3D matrix


Spectral images and colorreproduction

• Multispectral image 81 components taken in the range380-780nm, at 5nm, by a Specim camera.

• Multispectral image can be presented by a cube with twospatial and one spectral (wavelength) dimension.

• The RGB color image can be reproduced from themultispectral image using the color matching functionsx(λ), y(λ) and z(λ) and a spectral power density of thelight souce S(λ)(e.g. D65).

• The spectral reflectance of the object R(λ).



In Fig. S(λ) is normalized to 100 at 560 nm.



X = k

∫

780

380

R(λ)S(λ)x(λ)dλ

Y = k

∫

780

380

R(λ)S(λ)y(λ)dλ

Z = k

∫

780

380

R(λ)S(λ)z(λ)dλ

k =1

∫

780

380S(λ)y(λ)dλ

• x, y, z, are the color matching functions.

• R(λ) is a spectral reflectance factor ofthe material.

• S(λ) is a relative spectral powerdistribution of the light source.

• λ is a wavelength.

• k is a normalizing constant.

• Note: in colorimetry X, Y and Z aremeasured in percents and k ismultiplied by 100.

∣

∣

∣

∣

∣

∣

R

G

B

∣

∣

∣

∣

∣

∣

=

∣

∣

∣

∣

∣

∣

a1,1 a1,2 a1,3

a2,1 a2,2 a2,3

a3,1 a3,2 a3,3

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

∣

X

Y

Z

∣

∣

∣

∣

∣

∣


PCA

k=1, 89.25% k=2, 97.78% k=3, 99.31% k=10, 99.92%


Skin wound image analysis• The RGB color image (left). In the center of yellow area the spot is gray.

• The red component of the RGB color image was fused with the NIRimage using the multiresolution pixel fusion and replaced by the fusedimage.

• The RGB color image with the fused red component (right). In thecenter of yellow area the spot is red.


Skin wound image analysis• The developed multispectral camera acquires images: NIR, red, green

and blue, using LED light. The red, green and blue are reproduced as acolor image [1].

• Preprocessing. Image calibrated using reference white and black like inspectrophotometry.

• Support Vector (SV) classifier.

• Postprocessing. Either morphological operations (dilation, erosion) ormedian filtering.

SVM modelSV classifier

Preprocessing

Test image

Normalized image

Segmented image

Smoothed image

Postprocessing


Skin wound image analysis• Skin wound image segmentation. The NIR and color images are

combined and used as a four component image.• After segmentation we measured area of skin wound and observed a

wound area change over the time. Color images (top row), NIR images(middle row) and segmented images (botttom row).


Dichromatic reflection model• The Dichromatic Reflection Model describes light reflection from

inhomogeneous dielectric materials as a function of two independentreflection processes: body (diffuse) and surface (specular) reflection [6].

• The model is applicable to: plastics, paints, papers, ceramics, textiles,vinyl, tiles, fruits, leaves, woods and human skin. It is not applicable formetals.

Colorants

Surface

Surface reflection

Body reflection

Incident lightnormalSurface

Medium


Dichromatic reflection model• We consider the segmented uniformly colored regions (dielectric

surfaces) under single color illumination.

• The physical approach suggests the dichromatic reflection model fordescribing the data structure in the RGB color space.

• For analysis only regions with cherries are used. The highlight clustercaused by surface reflection.


Image colorization

• The dichromatic reflection model is used for colorization.

• The orange and red pepper colors were used for colorization of asynthetic teapot [3].


Image fusion using image processing• Brain surgery images, color (upper-left) and NIR (upper-right). Images courtesy

of Mr. Kari Koivuporras.• They can be fused using alpha blending (bottom-left) or image colorization

(bottom-right).• The other approach based on PCA and pseudocoloring will be implemented in

SOLO.


Estimating spectral reflectance ofpainting

The system consists of the single chip high quality CCD camera and therotating color filter wheel comprising several color filters [2].

filtersColor

Painting

Digital camera

Illuminant


Estimating spectral reflectance ofpainting

Hermitage. Scientists of the department of scientific and technicalexamination use NIR imaging for authentication of work of art.


Spectral electronic endoscope.Image fusion using variable selection.

Recently, a new electronic endoscope based on a conventional electronicendoscope was developed to reproduce arbitrary multispectral images of themucous membrane such as gullet, colon and stomach [7].

The conventional electronic endoscope for color images is shown in Figure.


Spectral electronic endoscope

• Original color image (left). Selection of components by doctors (middleimage) [7].

• The way for automatic component selection using a Fourier transformand an area defined by a chromaticity histogram (right) [4].


Spectral electronic endoscope• Color images (top).

• The automatically fused components of multispectral images (bottom).


ReferencesBochko V., Vlisuo P., Harju T. and Alander J., Lower Extremity Ulcer Segmentation of Visual and

Near-Infrared Imagery, Skin Research and Technology, Vol. 16, 2010, pp. 190-197. doi:10.1111/j.1600-0846.2009.00415.x.

Bochko V., Tsumura N. and Miyake Y., Spectral Color System for Estimating Spectral reflectance of Paint,

Journal of Imaging Science and Technology, Vol. 51, No. 1, 2007, pp. 70-78.

Pipirigeanu A., Bochko V. and Parkkinen J., A Computationally Efficient Technique for Image Colorization,

The Computational Color Imaging Workshop (CCIW’09), Saint Etienne, France, March 26 and 27, 2009.

Bochko V., Miyake Y., Tsumura N., Nakaguchi T., Image Fusion in Spectral Electronic Endoscope, The Ninth

International Symposium on Multispectral Color Science and Application, Taipei, Taiwan, 30 May - 1 June,2007. pp. 109-115.

Hill B., High Quality Color Image Reproduction: The Multispectral Solution. The Ninth International

Symposium on Multispectral Color Science and Application, Taipei, Taiwan, 30 May - 1 June, 2007. pp. 1-7.

Klinker, G., J., Shafer, S., A., Kanade, T.: A Physical Approach to Color Image Understanding. International

Journal of Computer Vision 4 (1990) 7-38.

Miyake Y., Kouzu T. Takeuchi S., Yamataka S. Nakaguchi T., Tsumura N., Development of New Electronic

Endoscopes Using the Spectral Images of an Internal Organ, The 13th Color Imaging Conference CIC

2005, Scottsdale, Arizona, pp. 261-263, 2005.

Miyake Y., Evaluation of Image Quality Based on Human Visual Characteristics, The First International

Workshop on Image Media Quality and its Applications, Nagoya, Japan, pp. 10-14, 2005.

SOLO toolbox: http://wiki.eigenvector.com/index.php?title=Main Page


Questions


Documents

Multispectral imaging. Applications - Vaasan yliopistolipas.uwasa.fi/~TAU/AUTO3090/Slides/lect_multispec.pdf · may show information invisible by a human eye. ... Multispectral imaging