Upload
erick-cross
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Lighting affects appearance
What is the Question ?(based on work of Basri and Jacobs, ICCV 2001)
Given an object described by its normal at each surface point and its albedo (we will focus on Lambertian surfaces)
1.What is the dimension of the space of images that this object can generate given any set of lighting conditions ?
2. How to generate a basis for this space ?
90.797.296.399.5#9
88.596.395.399.1#7
84.794.193.597.9#5
76.388.290.294.4#3
42.867.953.748.2#1
ParrotPhoneFaceBall
(Epstein, Hallinan and Yuille; see also Hallinan; Belhumeur and Kriegman)
5 2DDimension:
Empirical Study
DomainDomain
Lambertian
No cast shadows
(“convex” objects)
Lights are distant
nl
Lambert Law
k(max (cos, 0)
Lighting to Reflectance: Intuition
1
0 1 2 30
0.5
1
0 1 2 30
0.5
1
1.5
2
r Three point-light sources, l), Illuminating a sphere and its reflection r). Profiles of l and r
Lighting to Reflectance: Intuition
Images
...
Lighting Reflectance
lllllS llrrrr ddlkr sin),(),;,(),(2
where )0),max(cos(1
)(),;,( lrlrllrr kk
...
Spherical Harmonics (S.H.)
Orthonormal basis, , for functions on the sphere.
n’th order harmonics have 2n+1 components.
Rotation = phase shift (same n, different m).
In space coordinates: polynomials of degree n.
imnmnm eP
mn
mnnY )(cos
)!(
)!(
4
)12(),(
nmn
mn
n
m
nm zdz
d
n
zzP )1(
!2
)1()( 2
22
nmY
S.H. analog to convolution theorem
• Funk-Hecke theorem: “Convolution” in function domain is multiplication in spherical harmonic domain.filter.
k
nn
nmnnm
kn
YYk
12
4
Harmonic Transform of Kernel
00
( ) max(cos ,0) n nn
k k h
12
( 2)! (2 1)
13
( 1) 2, even2 ( 1)!( 1)!
0 2, od2
2
d2
0
nn
n
n nn
n n
kn
n
n
0nY 2cos16
5cos
2
1
4
1
0.886
0.591
0.222
0.037 0.014 0.0070
0.5
1
1.5
0 1 2 3 4 5 6 7 8
Amplitudes of Kernel
n
nA
n
nmnmn k
nA 2
12
1
Energy of Lambertian Kernel in low order harmonics
Accumulated Energy
37.5
87.599.2 99.81 99.93 99.97
0
20
40
60
80
100
120
0 1 2 4 6 8
kis a low pass filter 2cos16
5cos
2
1
4
1)( k
Reflectance Functions Near Low-dimensional Linear Subspace
0
( )n
nm nm nmn m n
r k l K L h
Yields 9D linear subspace.
2
0
)(n
n
nmnmnmnm hLK
nmY
nmY
Forming Harmonic Imagesnm nmb (p)=λr (X,Y,Z)
Z YX
2 2 22λ(Z -X -Y ) XZ YZ2 2λ(X -Y ) XY
How accurate is approximation?Point light source
Amplitude of k
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
0 1 2 4 6 8
Amplitude of l = point source
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 4 6 8
9D space captures 99.2% of energy
2
00
)()(n
n
nmnmnmnm
n
n
nmnmnmnm hLKhLKlkr nmY nmY
How accurate is approximation? Worst case.
9D space captures 98% of energy
DC component as big as any other. 1st and 2nd harmonics of light could have zero energy
Amplitude of k
-0.50
0.51
1.5
22.5
33.5
0 1 2 4 6 8
Amplitude of l = point source
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 4 6 8
How Accurate Is Approximation?
Accuracy depends on lighting.
For point source: 9D space captures 99.2% of energy
For any lighting: 9D space captures >98% of energy.
Accuracy of Approximation of Images
Normals present to varying amounts.
Albedo makes some pixels more important.
Worst case approximation arbitrarily bad.
“Average” case approximation should be good.
Summary
Convex, Lambertian objects: 9D linear space captures >98% of reflectance.
Explains previous empirical results.
For lighting, justifies low-dim methods.
Models
Query
Find Pose
Compare
Vector: IMatrix: B
Harmonic Images
Recognition
Experiments (Basri&Jacobs) 3-D Models of 42 faces
acquired with scanner.
30 query images for each of 10 faces (300 images).
Pose automatically computed using manually selected features (Blicher and Roy).
Best lighting found for each model; best fitting model wins.
Results
9D Linear Method: 90% correct.
9D Non-negative light: 88% correct.
Ongoing work: Most errors seem due to pose problems. With better poses, results seem near 100%.