View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Properties of Histograms and their Use for Recognition
Stathis Hadjidemetriou, Michael Grossberg,
Shree Nayar
Department of Computer Science
Columbia University
New York, NY 10027
Motivation
• Histogramming is a simple operation:
• Histograms have been used for:– Object recognition [Swain & Ballard 91, Stricker & Orengo 95]
– Indexing from visual databases [Bach et al, 96, Niblack et al 93, Zhang et al 95]
• Histogram advantages:– Efficient– Robust [Chatterjee, 96]
• Histogram limitation: – Do not represent spatial information
Motivation
Overview
•Image transformations that preserve the histogram
•Image structure through the multiresolution histogram
•Multiresolution histogram compared with other features
Invariance of Histogram with Discontinuous Transformations
Cut and rearrange regions
Shuffle pixels
Invariance of Histogram with Continuous Transformations
Rotation
Shear
What is the complete class of continuous transformations that
preserves the histogram?
Model for Image
Continuous domain Image: Map from continuous domain to intensities
Model for Histogram
U
Histogram count for bin U
≡Area bounded by level sets U
U
•Vector fields, X, morph images [Spivak, 65]:
Continuous Image Transformations
X
Gradient Transformations
FX
Original
5.122 )( yxF
40
)(sin
22 yxF
xyF
yxF 2
20
)(sin.
.20
)(sin
yx
yxF
Histograms of Gradient Transformations
Condition 1: Histogram Preservation and Local Area
Histogram preservedLocal area preserved
[Hadjidemetriou et al, 01]
T
……
……
T
Condition 2: Local Area Preservation and Divergence
•Divergence is rate of area change per unit area
Local area preserveddivergence is zero [Arnold, 89]
Small region
•Fields along isovalue contours of an energy function F
Isovalue contours
)sin(xyF
Hamiltonian Fields
•Flow of incompressible fluids [Arnold, 89]
Hamiltonian flow
Computing Hamiltonian Fields
22 yxF Gradient of
1. Compute gradient of F 2. Rotate gradient pointwise 900
jx
Fi
y
FFR
)(90
22 yxF Hamiltonian of
090R
Transformations preserve histogram of all images corresponding field is Hamiltonian
[Hadjidemetriou et al, CVPR, 00, Hadjidemetriou et al, IJCV, 01]
Theorem
Condition 3: Divergence and Hamiltonian Fields
Divergence of field is zero Hamiltonian field [Arnold, 89]
0
jx
Fi
y
Fdiv
Examples of Hamiltonian TransformationsLinear: Translations, rotations, shears
Original 3xF
7.022 )( yxF 5.122 )( yxF 10
)(sin
yxF
yxF 2
Examples of Hamiltonian Transformations
4
)(sin
4
)(sin
yxyxF
40
)(sin
22 yxF
65
coshyx
F
)(3 2244 yxyxF
Border Preserving Hamiltonian Transformations
0border
F
~
FFF win0|
,0|
borderwin
borderwin
F
F
2)2/(
,
2
1wx
xwin eF
0 w
h
Examples of Windowed Hamiltonian Transformations
yxF ~
2~
yF 3~
xF 24~
3xxF )3(
324
24~
yy
xxF
xyF ~
10
)(sin
~ yxF
22
~
yxF 5.122~
yxF
20
sin
.20
sin~
yx
yxF
Examples of Windowed Hamiltonian Transformations
Identical histograms:
4
sin
.4
sin~
yx
yxF
65
cosh~ yxF
yxF 2
~
Weak Perspective Projection
•Depth (z) causes scaling
cos2
2
z
fmw
[Hadjidemetriou et al, 01]
•Planar object tilt causes shearing and scaling
The Hamiltonian transformations is the complete class of continuous image transformations that preserves the
histogram
How can spatial information be embedded into the histogram?
Previous work on Features combining the Histogram with Spatial Information
•Local statistics:−Local histograms [Hsu et al, 95, Smith & Chang, 96, Koenderink and Doorn, 99, Griffin, 97]
−Intensity patterns [Haralick,79, Huang et al, 97]
•One histogram: −Derivative filters [Schiele and Crowley, 00, Mel 97]
−Gaussian filter [Lee and Dickinson, 94]
•Many techniques are ad-hoc or not complete
Multiresolution HistogramG(l2)
Limitations of HistogramsDatabase of synthetic images with identical histograms
[Hadjidemetriou et al, 01]
Matching with Multiresolution Histograms
Match under Gaussian noise of st.dev. 15 graylevels:
Matching with Multiresolution Histograms Match under Gaussian noise of st.dev. 15 graylevels:
How is Image Structure Encoded in the Multiresolution Histogram?
?
Image structure
Differences of histogramsdl
l))(*(d GLh
LImage
h(L*G(l))Multiresolution histogram
Histogram Change with Resolution and Spatial Information
•Bin j: dl
lhdj
))(*( GL
Spatial information
•Averages of bins:
where Pj are proportionality factorsdl
ldh jm
jj
))(*(
1
GLP
ill-conditioned
well-conditioned
Histogram Change with Resolution and Fisher Information Measures
qJ= Generalized Fisher
information measures of order q [Stam, 59, Plastino et al, 97]
≡
)(LqJ L is the image
xdq
D
22
LLL
=
D is the image domain
dl
ldh jm
jqj
))(*(
1
GLP
Averages
Image Structure Through Fisher Information Measures
?
Image structure
Differences of histogramsdl
l))(*(d GLh
LImage
h(L*G(l))Multiresolution histogram
Fisher information measures (Analysis)
P
Jq
Shape Boundary and Multiresolution Histogram15.0)( yxR LSuperquadrics:
=0.56
Histogram change with l is higher for complex boundary
=1.00 =1.48 =2.00
=6.67
Texel Repetition and Multiresolution Histogram
,, pTiling
Histogram change with l is proportional to number of texels (analytically)
Texel Placement and Multiresolution Histogram
Std. dev. of perturbation
Histogram change with l decreases with randomness
Matching Algorithm for Multiresolution Histograms
Burt-Adelson image pyramid
Cumulative histograms
L1 norm
Differences of histograms betweenconsecutive image resolutions
Concatenate to form feature vector
Histogram Parameters
•Bin width
•Smoothing to avoid aliasing
•Normalization:−Image size−Histogram size
179x179
89x89
44x44
5x5
……
Database of Synthetic Images
108 images with identical histograms [Hadjidemetriou et al, 01]
Sensitivity of Matching for Synthetic Images
Database of Brodatz Textures91 images with identical equalized histograms: 13 textures
different rotations
Match Results for Brodatz Textures
Match under Gaussian noise of st.dev. 15 graylevels:
Sensitivity of Class Matching for Brodatz Textures
Database of CUReT Textures 8,046 images with identical equalized histograms : 61
materials under different illuminations [Dana et al, 99]
Match Results for CUReT Textures
Match under Gaussian noise of st.dev. 15 graylevels:
Match Results for CUReT Textures
Match under Gaussian noise of st.dev. 15 graylevels:
Sensitivity of Class Matching for CUReT Textures
100 randomly selected images per noise level
Embed spatial information into the histogram with the multiresolution histogram
How well does the multiresolution histogram perform compared to other image features?
Comparison of Multiresolution Histogram with Other Features
•Multiresolution histogram:−Variable bin width−Histogram smoothing
•Fourier power spectrum annuli [Bajsky, 73]
•Gabor features [Farrokhnia & Jain, 91]
•Daubechies wavelet packets energies [Laine & Fan, 93]
•Auto-cooccurrence matrix [Haralick, 92]
•Markov random field parameters [Lee & Lee, 96]
Comparison of Effects of Transformations on the Features
Feature Translation RotationUniform Scaling
1Fourier power
spectrum annuliinvariant robust equivariant
2 Gabor features invariant sensitive equivariant
3Daubechies wavelet
energiessensitive sensitive sensitive
4Multiresolution
histogramsinvariant invariant equivariant
5Auto-cooccurrence
matrixinvariant robust equivariant
6Markov random field
parametersinvariant sensitive sensitive
Comparison of Class Matching Sensitivity of Features
Database of Brodatz textures
Comparison of Class Matching Sensitivity of Features
•Database ofCUReT textures•100 randomly selected images per noise level
Sensitivity of Features to Matching
FeatureGaussian
NoiseDatabase
size,# classesIlluminati-
on Parameter selection
Fourier power spectrum annuli
sensitive sensitive robust very sensitive
Gabor features robust robust robust sensitive
Daubechies wavelet energies
sensitive robust robust robust
Multiresolution histogram
robust robust robust robust
Auto-cooccurrence matrix
very sensitive very sensitive very sensitive very sensitive
Markov random field parameters
very sensitive very sensitive sensitive N/A
Comparison of Computation Costs of Features
1Markov random field
parametersO(n(2-1)2-(2-1)3/3)
2 Gabor features ( (logn+1)nlogn)
3Fourier power spectrum
annuliO(n3/2)
4 Auto-cooccurrence matrix O(n)
5 Wavelet packets energies O(nl)
6 Multiresolution histograms nn- number of pixels- window widthl- resolution levels
Decreasing cost
The multiresolution histogram compared to other image features is robust and efficient
Summary and Discussion
•Hamiltonian transformations preserve features based on:−Histogram−Image topology
•Multiresolution histograms:−Embed spatial information
•Comparison of multiresolution histograms with other features:−Efficient and robust
Recognition of 3D Matte Polyhedral Objects
•Face histograms:–Magnitude scaled by tilt angle ()
–Intensity scaled by illumination (ai )
•In an object database find [Hadjidemetriou et al, 00]:–Object identity
–Pose ()–Illumination (ai )
•Total histogram: Sum of h(i) of visible faces
A Simple Experiment
Object 1: Object 2:
Object 3: Object 4:
Object Tests Rank=1 Rank=2
Total 40 38 2
Shape Elongation and Multiresolution Histogram
Elongation:
y
x
St. dev. along axes: x, y.
•Gaussian:Sides of base : rx, ry.
y
x
r
r
•Pyramid:
Elongation:
1
(analytically)
Histogram change with l
Are all image resolutions equally significant?
Resolution Selection with Entropy of Multiresolution Histograms
•Entropy-resolution plot [Hadjidemetriou
et al, ECCV, 02] :–Global–Non-monotonic
….
l
…………………………………
Examples of Entropy-Resolution Plots
The entropy of the multiresolution histogram can be used to detect significant image resolutions
Future Work
•Histogram preserving fields:−Transformations over limited regions−Sensitivity of features to image transformation
•Multiresolution histograms:−Color images−Rotational variance with elliptic Gaussians
•Resolution selection:−Preprocessing step−Non-monotonic features