Image and Video Processing
Image Formation and Color
Thomas Breuel
light
light
electromagnetic radiation
properties of both particle and wave
diffraction, interference
different frequencies/wavelengths
narrowband appears as pure color
white light = even mixture
what do we perceive?
radianceenergy emitted/received as light (W)
luminanceradiance adjusted by sensitivity of the eye (lm)
brightnessperceptual quantity (which is brighter?)
image processing goals
prepare images for visual inspectionbased on properties of
visual system
replicate performance of visual system
prepare images for automated analysisbased on physical
properties
solve a physical problem
perception vs physical reality
discounting the illuminant
purveslab.net
discounting the illuminant
reasoning
purveslab.net
sunrise/sunset
midday
incandescent light
daylight
color constancy
Even low-level perception (color etc.)
interprets images.
Humans do not consciously perceive
physical intensities or light frequencies.
definition of color
Farbe ist diejenige Gesichtsempfindung eines dem Auge des
Menschen strukturlos erscheinenden Teiles des Gesichtsfeldes, durch
die sich dieser Teil bei einugiger Beobachtung mit unbewegtem Auge
von einem gleichzeitig gesehenen, ebenfalls strukturlosen
angrenzenden Bezirk allein unterscheiden kann.
Color is a percept that allows humans to distinguish two
untextured, flat, uniform, unmoving surface patches from one
another by looking at them with one eye.
trichromatic vision
experimental evidence (1852)
experimentobservers match given color by mixing different
numbers of base colors
resultcombinations of three base colors are sufficient
Maxwell Helmholtz experiments
sensors
structure of the retina
structurecones are in the center
rods are in the periphery
implications...
lower density of blue receptors
absent from the center of the fovea
retina, rods, cones...
structureretina is inverted
cones in fovea, rods in periphery
low density of blue, absent from center of fovea
why?no engineer would design such a system (?)
actually, very finely tuneddensities and curves optimized for
fine spatial discrimination in the presence of chromatic
aberration
lots of other complex design components
some historical accidents
direct imaging of photoreceptors
live retinal imaging using adaptive optics
spectral sensitivity
maximum sensitivity
cones aren't really RGBM and L are close together
CRT RGBspaced apart further
less overlap
peaky spectrum in red
practical applications
foundation ofdisplay devices
printing
color calibration
image processing
digital photography
significant foraccessibility (color blindness)
display design
color vs frequency
color perception
lightspectrum3DRGB vectorcolorpercept
spectral sensitivity of cones
contextual interpretation
chromaticity diagram
x,y,z percentages of RGB
trichromatic coefficients
chromaticity = hue+saturation
brightness not visible in diagram
spectrum and color
Newton: spectrum, additivity, hue circle
metamerism
vv
http://www.visualmill.com/
physiological color space
unrealizablebecause ofoverlap ofresponse curves
response
vv
wavelength vs rgb
more aspects of color
color difference perception
color gamut
color spaces
color spaces
RGB red green blue
CMY, CMYK cyan magenta yellow (black)
HSI / HSV / HSL hue saturation intensity...
XYZ perceptual space
L*a*b* perceptual space (normalized dist)
YUV / YIQ / YCbCr TV spaces
3D RGB space
pixel and color values
colors for all on/off components
CMY(K) space
HSI space
decouples intensity from chromaticity
hue, lightness, saturation
decomposing the image to Lab
decomposing the image to Lab
digital cameras
image formation in cameras
geometry
optics
sensor
signal processing
pinhole camera model
projection, translation, rotation
camera equation
often write this as matrix multipliations in homogeneous
coordinates
color and surfaces
n
illuminant
surface reflectance
lens
retina, film
The light that arrives at the eye/camera
is the product of the spectrum of the illuminantand the surface
reflectance of the object(at each wavelength)
image formation model
digital camera sensors
CCD vs CMOS
linear response
logarithmic response
color sensors (Bayer pattern)
higher resolution in green (like human eye)
output in RAW format
final image interpolated (demosaicing)
Kodak KAI-11000 CCD
4008x2672 pixels (11MP)
KAI-11000 quantum efficiency
digitization
digitization
spatial quantizationcontinuous coordinate system
image plane divided into buckets (pixels)
intensity quantizationphoton counting (some specialized
sensors)
discretization of voltage
intensity quantization
quantization
spatial digitization
sampling vs digitization
sampling
digitization
(equivalent to convolution with a square pillbox, followed by
sampling)
quantization
uniform quantizationdigital version of analog quantity
gamma-corrected / logarithmic / non-uniformtry to get more
resolution for interesting values
mimic human JNDs
humans see logarithmically
binarizationspecial quantization for document images, industrial
vision
effect of digitization
effects of quantization
iso-preference curves
vary resolution (N) and quantization (k)
ask for preference between versions
image storage
image storage
raster storage convention (used by SciPy display/load/save)
image formats
JPEGlossy compression, natural images
PNGlossless compression, graphics
TIFFmultipage, extensible >8bpp, document images
GIFgraphics, palette, animation
JPEG2000lossy wavelet compression, >8bpp
storage
raw size 12 Mpixel, RGB, 8bpp
same in floating point? double?
factorsimage type (natural, graphics, text, ...)
compression method
Click to edit the title text format
Click to edit the outline text formatSecond Outline LevelThird
Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline
LevelSeventh Outline LevelEighth Outline LevelNinth Outline
Level
Click to edit the title text format
Click to edit the outline text formatSecond Outline LevelThird
Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline
LevelSeventh Outline LevelEighth Outline LevelNinth Outline
Level
/home/tmb/teaching/ipiu/lecture02/slides02.odp
