View
220
Download
2
Category
Preview:
Citation preview
EE 7700
Color
Bahadir K. Gunturk 2
References On Color: Wikipedia, Gonzalez, Poynton, many others… On HDR: Slides and papers by Debevec, Ward, Pattaniak, Nayar,
Durand, et al… http://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/
Bahadir K. Gunturk 3
Color
Color is a perceptual property. It comes from the spectrum of light (energy distribution of
light versus wavelength) interacting with the spectral sensitivities of the light receptors (photoreceptors) in the eye.
Bahadir K. Gunturk 4
Human Visual System
Human visual system is sensitive to a narrow range of the electromagnetic spectrum. (Approximately from 380nm to 740nm.)
Bahadir K. Gunturk 5
Human Visual System
•The diameter of the eyeball is around 22mm.•Retina is a thin layer of neural cells that lines the back of the eyeball.•Retina contains photoreceptors (rods and cons) that respond to light.•Fovea is the most sensitive part of the retina; it is responsible for our sharp central vision.•Some birds (such as hawks) have more than one fovea. (two).• The axons (coming from receptors) exit the eye at the optic disc (blind spot), forming the optic nerve.• There are 1.2million axons in the optic nerve. • There are 130million photoreceptors. A large amount of pre-processing is done within the retina. 10% of the axons are devoted to the fovea area.
Bahadir K. Gunturk 6
0.5mm
Bahadir K. Gunturk 7
Human Visual System There are two classes of receptors: cones and rods. Cones:
Sensitive to color (there are three cone types in humans) Produces high-resolution vision 6-7 million cone receptors, located primarily in the central portion of
the retina Rods:
Not involved in color vision 75-150 million rod receptors, distributed over retina Sensitive to low levels of illumination. Not effective in bright light. Produces lower-resolution vision
Bahadir K. Gunturk 8
Human Visual System
There are three types of cones in humans
A side note:•Humans and some monkeys have three types of cones (trichromatic vision); most other mammals have two types of cones (dichromatic vision).• Marine mammals have one type of cone.• Most birds and fish have four types. •Lacking one or more type of cones result in color blindness.
Human lens and cornea are increasingly absorvative to smaller wavelengths, which sets wavelength sensitivity limit to around 380nm. Humans lacking lens reported to see ultraviolet.
65% sensitive to Long-wavelength33% sensitive to Medium2% sensitive to Small
Bahadir K. Gunturk 9
Human Visual System• Light is reduced to three color components by the eye.• These values are called tristimulus values. • The set of all possible tristimulus values determines the human color space.• It is estimated that humans can distinguish around 10million colors.
The mechanisms of color vision within the retina are explained well in terms of tristimulus values. The way the values sent out of eye is little different:A dominant theory says that color is sent out of the eye in three opponent channels: a red-green channel, a blue-yellow channel and a black-white "luminance" channel. These channels are constructed from the tristimulus values.
Bahadir K. Gunturk 10
Human Visual System
Color constancy (Chromatic adaptation): The perceived color of objects remains relatively constant under varying illumination conditions. This helps us identify objects.
A red apple appears red in sunlight, at sunset, in florescent illumination, etc. Of course, this works only if the illumination contains a range of wavelengths. The HVS determines the approximate composition of the illuminating light, and then discounted to obtain the objects “true color” or reflectance.
Bahadir K. Gunturk 11
Human Visual System
Which square is darker? A or B?
Bahadir K. Gunturk 12
Human Visual System
Bahadir K. Gunturk 13
Human Visual System
Bahadir K. Gunturk 14
A Color Blindness Test
2 3 5 5
6 8 26 56
Bahadir K. Gunturk 15
Human Visual System Colors consisting of a single wavelength are called pure spectral or
monochromatic colors. Most light sources are mixtures of various wavelengths of light. If they
produce a similar stimulus in the eye, a non-monochromatic light source can be perceived as a monochromatic light.
For a non-monochromatic light source, we may talk about the dominant wavelength (or color), which identifies the single wavelength of light that produces the most similar sensation.
Of course, there are many color perceptions that cannot be identified by pure spectral colors, such as pink, tan, magenta, achromatic colors (black, gray, white).
Bahadir K. Gunturk 16
Human Visual System Two different light spectra that have the same effect on the three color
receptors will be perceived as the same color.
Most human color perceptions can be generated by a mixture of three colors, called primaries.
This is used to reproduce color in photography, printing, TV, etc.
Bahadir K. Gunturk 17
CIE In 1931, the Commission Internationale de l’Eclairage (CIE) established
standards for color representation. Subjects were shown color patches and asked to match the color by adjusting three monochromatic colors. Based on the experiments, they defined the color-matching-functions:
Bahadir K. Gunturk 18
Tristimulus Let X, Y, and Z be the tristimulus values.
A color can be specified by its trichromatic coefficients, defined as
Xx
X Y Z
Y
yX Y Z
Zz
X Y Z
X ratio
Y ratio
Z ratio
Two trichromatic coefficients are enough to specify a color. (x + y + z = 1)
Bahadir K. Gunturk 19
CIE Chromaticity DiagramInput light spectrum
x
y
Bahadir K. Gunturk 20
CIE Chromaticity DiagramInput light spectrum
x
y
Bahadir K. Gunturk 21
CIE Chromaticity DiagramInput light spectrum
Boundary
x
y
380nm
700nm
Bahadir K. Gunturk 22
CIE Chromaticity DiagramInput light spectrum
Boundary
Bahadir K. Gunturk 23
CIE Chromaticity DiagramLight composition
Bahadir K. Gunturk 24
CIE Chromaticity DiagramLight composition
Light composition
Bahadir K. Gunturk 25
CIE Chromaticity Diagram The CIE chromaticity diagram shows
the human color space as a function of x and y.
Boundary indicates the pure spectrum colors. (Full saturation.)
Inside the boundary shows mixture of spectrum colors.
Boundary
Bahadir K. Gunturk 26
CIE Chromaticity Diagram The CIE chromaticity diagram is helpful to determine the range of colors
that can be obtained from any given colors in the diagram.
Source: http://hyperphysics.phy-astr.gsu.edu/hbase/vision/visioncon.html#c1
Gamut: The range of colors that can be produced by the given primaries.
http://www.brucelindbloom.com/index.html?Eqn_ChromAdapt.html
Bahadir K. Gunturk 27
CIE Chromaticity Diagram
Green: ColorMatch primaries, D50Orange: sRGB primaries, D65
R’G’B’: Gamma corrected valuesGreen: Corresponding RGB with gamma 1.8Orange: … with gamma 2.2
Bahadir K. Gunturk 28
Mixtures of Light The primary colors (primaries) can be added to produce the secondary
colors of light.
Example: Color TV displays use this additive nature of colors. An electron gun hits red, green, blue phosphors (with different energies) in a small region to produce different shades of color.
Bahadir K. Gunturk 29
Mixtures of Light In printing, subtractive primaries are used: Cyan absorbs only Red. Magenta absorbs only Green. Yellow absorbs only Blue.
In printing, dark colors may be obtained by addition of black ink. Such color systems are known as CMYK systems.
Y
M
C
Bahadir K. Gunturk 30
Color Space A color space relates numbers to actual colors; it contains all realizable
color combinations. A color space could be device-dependent or device-independent.
An RGB color space has three components:Red, Green, and Blue. But, it does not specify the exact color unless Red, Green, and Blue are defined.
The sRGB is a device-independent color space. It was created in 1996 by HP and Microsoft for use on monitors and printers.It is the most commonly used color space.
R G
B
Bahadir K. Gunturk 31
Color Space
The Adobe RGB is developed by Adobe in 1998. It was designed for printers; it has a wider gamut than sRGB.
Bahadir K. Gunturk 32
Color Space HSV color space defines color in terms of Hue, Saturation, and Value. Hue is the color type (such as, red, blue, yellow). (0-360 degrees) Saturation is the purity of the color. (0-100%) Value is the brightness of the color. (0-100%)
HSV is not device-independent. It is defined in terms of RGB intensities. It is commonly used in computer graphics applications.
Bahadir K. Gunturk 33
Color Space YUV color space defines color in terms of one luminance (brightness)
and two chrominance (color) components. YUV is created from RGB components.
YUV YCbCr
Bahadir K. Gunturk 34
Color Space
Input device Output device
Color space conversion
•International Color Consortium (ICC) was established in 1993 to create an open color management system.•The system involves three things: color profiles, color spaces, and color space conversion.•The color profile keeps track of what colors are produces for a particular device’s RGB or CMYK numbers, and maps these colors as a subset of the “profile connection space”.
Profile Connection Space
Bahadir K. Gunturk 35
Color Space
Input device Output device
Color space conversion
When there is gamut mismatch, There should be color rendering.
Profile Connection Space
Bahadir K. Gunturk 36
CIELAB (CIE L*a*b*)• It was found that CIExyz is not a perceptually uniform color space: The minimum distance between two discernable colors differs in different parts of the CIExyz diagram.
• Perceptually linear means that a change of the same amount in a color value should produce a change of about the same visual importance. When storing colors in limited precision values, this can improve the reproduction of tones.
• L*a*b* color space was defined in 1976. Conversion from XYZ to L*a*b* is
for otherwise
Xn, Yn and Zn are the CIE XYZ values of the reference white point.
Bahadir K. Gunturk 37
White Point•A white point is the reference point to define the color “white”.
•Primaries plus the white point (indicating power ratio of primaries) should be given.
•Depending on the application, different definitions of white are needed to get acceptable results. For example, photographs taken indoors may be lit by incandescent light, which are relatively orange compared to daylight. Defining “white” as daylight will give unacceptable results when attempting to color-correct a photograph.
A list of common white points:
Name x y Notes
E 1/3 1/3 Equal energy
D65 0.31271 0.32902 TV, sRGB color space
A 0.44757 0.40745 Incandescent tungsten
Bahadir K. Gunturk 38
High Dynamic Range (HDR) Imaging
star lightstar light moon lightmoon light office lightoffice light day lightday light search lightsearch light
1010-6-6 1010-2-2 101011 101022 101044 101088101000
The range of radiances is more than 10^12 candela/m2
Range of human eye at an instant is around 10^4:1 (4log units)Human eye can adapt to see much wider range.
Candela is the unit of luminous intensity (power emitted by a light source in a particular direction, with wavelengths weighted by the sensitivity of the human eye.
Bahadir K. Gunturk 39
HDR
star lightstar light moon lightmoon light office lightoffice light day lightday light search lightsearch light
1010-6-6 1010-2-2 101011 101022 101044 101088101000
The range of radiances is more than 10^12 candela/m2
0 255 0 255
Range of Typical Displays:from ~1 to ~100 cd/m2
Bahadir K. Gunturk 40
1000 cd/m^2
Cone dominated
log Llog Laa
Gainrod
cone
log
Gai
n
0 2 4 6-2-4-6
Sensitivity of Eye
Bahadir K. Gunturk 41
0.04 cd/m^20.04 cd/m^2
Rod dominated
log Llog Laa
Gainrod
cone
log
Gai
n
0 2 4 6-2-4-6
Sensitivity of Eye
Bahadir K. Gunturk 42
Sensitivity of Eye
Bahadir K. Gunturk 43
HDR The range of image capture devices is also low
Bahadir K. Gunturk 44
HDR The range of image capture devices is also low
Bahadir K. Gunturk 45
HDR HDR image rendered to be displayed on a LDR display.
Bahadir K. Gunturk 46
HDR Problems:
• How to capture an HDR image with LDR cameras?• How to display an HDR image on LDR displays?
Bahadir K. Gunturk 47
• Capture multiple images with varying exposure.• Combine them to produce an HDR image.
Bahadir K. Gunturk 48
Creating HDR from Multiple Pictures
Measured intensity, z
t1 t2
t1 t2
Irradiance, E
Bahadir K. Gunturk 49
Creating HDR from Multiple Pictures
Measured intensity, z
t1t2
t1 t2
Irradiance, E
z1
z2
Ez1 = t1 * E z2 = t2 * E
E1=z1/t1
E2=z2/t2
Estimates:Take a weighted sum of E1 and E2:
w1 w2E=( w1*E1 + w2*E2 ) / (w1+w2)
E
Bahadir K. Gunturk 50
Creating HDR from Multiple Pictures
Measured intensity, z
t1t2
t1 t2
Irradiance, E
z1
z2
Ez1 = t1 * E z2 = t2 * E
E1=z1/t1
E2=z2/t2
Estimates:Take a weighted sum of E1 and E2:
wE=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))
z
Bahadir K. Gunturk 51
Creating HDR from Multiple Pictures In general, the camera response is not linear.
t1 t2
z1 = f ( t1 * E ) z2 = f ( t2 * E )
E1= g (z1) / t1
E2= g (z2) / t2
E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))
f
g
w
z
z
Questions: How to estimate g and t?
w is sometimes chosen as the derivative of f. (Mann)
Bahadir K. Gunturk 52
Radiometric Self Calibration
Polynomial model
Exposure ratios:
Cost function
Solve using
If exposure ratios are not known, solve iteratively
(Nayar)
Bahadir K. Gunturk 53
Tone MappingGiven an HDR image, how are we going to display it in an LDR display?
Bahadir K. Gunturk 54
Tone MappingGiven an HDR image, how are we going to display it in an LDR display?
Linear
Nonlinear
Bahadir K. Gunturk 55Durand & Dorsey
Bahadir K. Gunturk 56Durand & Dorsey
Bahadir K. Gunturk 57Durand & Dorsey
Bahadir K. Gunturk 58Durand & Dorsey
Bahadir K. Gunturk 59Durand & Dorsey
Bahadir K. Gunturk 60Durand & Dorsey
Durand & Dorsey
Bilateral filter
Bahadir K. Gunturk 61Durand & Dorsey
Bahadir K. Gunturk 62Durand & Dorsey
Bahadir K. Gunturk 63Durand & Dorsey
Bahadir K. Gunturk 64
Spatially Varying Exposures
Instead of capturing multiple pictures, allow different amounts of light pass for different pixel positions.
Estimate the missing pixels. Combine to obtain an HDR image.
100% 75%
50% 25%
Nayar
Bahadir K. Gunturk 65
Image Reconstruction: Interpolation
Bahadir K. Gunturk 66
Image Reconstruction: Aggregation
Bahadir K. Gunturk 67
HDR image examples
Bahadir K. Gunturk 68
HDR image examples
Bahadir K. Gunturk 69
HDR image examples
Bahadir K. Gunturk 70
Retinex Image Processing
Received intensity is a product of illuminance and reflectance:I = L*R
Illumination components changes slowly. Reflectance component changes fast.
Take the logarithm of I:log(I) = log(L) + log(R)
Apply a high-pass filter to obtain the reflectance.
Homomorphic filter Multi-scale retinex
Bahadir K. Gunturk 71
Retinex Image Processing
0.5
2.0L
H
Bahadir K. Gunturk 72
Retinex Image Processing
http://dragon.larc.nasa.gov/
Bahadir K. Gunturk 73
Retinex Image Processing
http://dragon.larc.nasa.gov/
Bahadir K. Gunturk 74
Retinex Image Processing
Vivek Agarwal
Bahadir K. Gunturk 75
Retinex Image Processing
Recommended