Nuclei of fungus cell Paramecium Caudatum

ter Haar Romeny, FEV

Nuclei of fungus cell Paramecium Caudatum

Spatial gradient Illumination spectrum-invariant gradient

Color RGB original

Color-Scale Differential Structure

Geusebroek et al, LNCS 1852, 459-464, 1999


The color of an object depends on

• color of the illuminating light• illumination intensity• sensor sensitivity• direction of surface normal• surface reflectance properties

Assumptions:• Scene is uniformly illuminated• light source is colored• surface has Lambertian reflectance


What causes color ?

Lamp

objectcolor

spectral

color


400 450 500 550 600 650 700

500 1000 1500 2000

50000

100000

150000

200000

250000

300000

350000

400 450 500 550 600 650 700 400 450 500 550 600 650 700 400 450 500 550 600 650 700

Object reflectance function for the observed spectrum for a resp. 2500K, 6500K and 10,000K light source:

Spectrum reflected froman arbitrary object

8 hc 5E hc

kT 11

Emissionspectrum ofblack bodyradiator


Color receptive fields

x

y

x


Self-organization:receptive fieldsfrom Eigenpatches(12x12 pixels)


Blakemore C, Cooper GF (1970), Development of the brain depends on the visual environment. Nature 228, 477 - 478


Colour receptivefields fromEigenpatches

x


GD Field et al. Nature 467, 673-677 (2010)

doi:10.1038/nature09424


Distribution of cone cells in the fovea of an individual with normal color vision (left), and a color blind (protanopic) retina. Note that the center of the fovea holds very few blue-sensitive cones. [Wikipedia]

ter Haar Romeny, FEVGD Field et al. Nature 467, 673-677 (2010) doi:10.1038/nature09424

Full functional sampling of cone lattice by four RGC types.


The color opponency model

Hering E, 1964. Outlines of a Theory of the Light Sense. Harvard University Press, Cambridge, Mass.


Hering basis

0 10 20 30 40 50 60 70 80 90-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Idea Koenderink: Gaussian derivatives of zero, first and second order in the wavelength domain

wavelength

RFsensitivity

How can weanalyse colordifferential structure?


Taylor color model

Luminance

Blue-yellowness

Purple-greenness

300 400 500 600 700 800 90000.10.20.30.40.50.60.70.80.9

LM

S

Cone sensitivity


Spatial color

Color scale-space starts by probing this space.

y

x

xy

s

Energy densities cannot be measured at a point, …… one probes a certain volume


Reflectance of light

Lamp

objectcolor

spectral

color

What are invariantproperties?


Reflectance model


Transparent materials


)()),,(1()()( 2 RvsneE f

The reflected spectrum is:

v = viewing directionn = surface patch normals = direction of illuminationf = Fresnel front surface reflectance coefficient in vR = body reflectance


),())(1)(,(),( 2 xRxxexE f

Because of projection of the energy distribution on the image plane the vectors n, s and v will depend on the position at the imaging plane. So the energy at a point x is then related to:

We assume an illumination with a locally constant color:

),())(1)(()(),( 2 xRxxiexE f


Aim: describe material changes independentof the illumination.

Rxxie

exRxxi

E

f

f

2

2

))(1)(()(

),())(1)((

),())(1)(()(),( 2 xRxxiexE f

Bothequationshavemanycommonterms


R

xR

e

e

E

EE

),(

1

)(

11ˆ

The normalized differential

determines material changes independent of the viewpoint, surface orientation, illumination direction, illumination intensity and illumination color!


The derivative jet to x and forms a complete family of geometric invariants:

0

ˆˆ ( ; 515 ; 55 )

n

n

EE G nm nm

These are observed properties, so we convolve with Gaussian derivatives

ˆn m

n m

E

x


2 2ˆ ˆx yE E

Color edges can be defined as the thresholding of the spatial gradient (color-invariant equivalent of Lw):

2 2ˆ ˆx yE E

Color invariants


Spatial color model and tracing color edges in microscopy

Influence of illumination color temperature on edge strength, scale is 3.0 px.

Skin tissue section illuminated by a halogen bulb at 4000 K (top) and 2600 K (bottom) color temperature.


1

ee

x. ex, y, eshortnotatione exzx, y, exx, y, ezx, y,

e2

x,. ex, y, eshortnotation1

e3e2 exzzx, y, 2 e exzx, y, ezx, y, exx, y, 2 ezx, y, 2 e ezzx, y,

Color-invariant multi-scale structural operators


Simplifyx2 y2 x, 2 y, 2;shortnotation 1

ex, y, 6 ex, y, 2ex, y, exzx, y, exx, y, ezx, y, 2

ex, y, 2ex, y, eyzx, y, eyx, y, ezx, y, 2 ex, y, 2 exzzx, y, 2 exx, y, ezx, y, 2 ex, y, 2 exzx, y, ezx, y, exx, y, ezzx, y, 2 ex, y, 2 eyzzx, y, 2 eyx, y, ezx, y, 2 ex, y, 2 eyzx, y, ezx, y, eyx, y, ezzx, y, 2

Total edge strength


[im,] 1e

e

color invariant 1e

e

[im,]

first wavelength derivative of

[im,] 22 second wavelength derivative of

g[im,] x2 y2 yellow-blue edges

g [im,] 2x2 2

y

2 red-green edges

g[im,] x2 y2

2x2 2

y

2 total color edge strength

Somecolor

differentialinvariants


Feulgen stain,red-green edges

Paramecium caudatum, Feulgen and Fast green stain

Color canny, red-green normalized edges, scale 3


Hematoxylin eosin stain

Pituitary gland, sheep, adenohypophysis 40x

Cell: E<0, E > 0, scale 1.0

Nuclei: E <0, E > 0, E +E < 0, scale 3.0

additional constraint added to refine selection


Safranin O stain

E > 0, E > 0, scale sigma 1.0

Safranin O stain for proteoglycans (mouse knee

joint)

Courtesy of Koen Gijbels and Paul Stoppie


Oil red O stain

Oil red O stain of fat emboli in

lung

E > 0, E > 0, scale 1.5


PAS stain

Lww > 0, Lvv Lww-Lvw2 > 0, E -E > 0, scale sigma 2.0

P.A.S. stain for carbohydrates (goblet cells, gut)

carbohydrates stain magenta - elliptic patches


Blood smear

Blood smear, Giemsa stain, 100x, JPEG

compression

RBC: E > 0, E +E > 0, scale 0.5

Leucocytes: E < 0, scale 12

Leucocyte nuclei: E < 0, E > 0, scale

3


Blue-yellow edges

Note the complete absence of detection of black-white edges.


Color edges can also be defined as the zero-crossings of the second order derivative in the spatial gradient direction (color-invariant equivalent of Lww):

0ˆˆ

ˆˆˆˆˆ2ˆˆˆ

22

22

yx

xxxxyxyyyyww

EE

EEEEEEEE

Second order color invariants


Color invariant edge detection

Luminance gradientedge detection



Conclusions

• Color ‘scale-space’ compatible with classical luminance

scale-space

• The model enables the design of practical image analysis

‘color reasoning’ solutions, e.g. invariance for illumination

• The color-scale invariant differential operators are building

blocks for a differential geometry on color images

Documents

Nuclei of fungus cell Paramecium Caudatum