A Projective Framework for Radiometric Image Analysis
CVPR 2009
Date: 2010/3/9
Reporter : Annie lin
Image reflectance
Image reflectance : diffuse reflectance + specular reflectance
How to estimate shape from images?
1. shape from shading – add more constrain
2. photometric stereo – take more images -> get “normal field”
but GBR (Generalize bas-Relief) ambiguity
the same image can be produced by different surface under corresponding lighting direction.
- GBR Transformation
Shape: “depth scaling” and ‘additive plane’
yxyxZyxZ ),(),(ˆ
Specular reflectance and GBR • GBR is resolved by ‘almost any’ additive specular reflectance
component.
– Only requirements
• Spatially uniform• Isotropic • Reciprocal
Isotropy + reciprocity:– Local light/view directions equivalent at distinct points– Observed reflectance equal
Outline • Reflectance symmetries on the plane• Isotropic and reciprocal • Isotropic-reciprocal quadrilateral • GBR Transformation
– Reconstruction via photometric stereo and show how to resolve shape ambiguities in both “uncalibrated ” and “calibrated ” cases
• Applications– Uncalibrated photometric stereo– Calibrated photometric stereo
• Conclusion
Reflectance symmetries on the plane
• The real projective plane provides and effective tool for analyzing reflectance symmetries , of which we focus on reciprocity and isotropy in this paper
Isotropic Pair
form an isotropic pair with respect to , ifvs
,
v
s
o
n
'n
d2
d1
n
v S
'n
Gauss sphere
', nn
Isotropic Curve and Symmetry
Isotropic curve:
Union of isotropic pairs Radiance function is symmetric
vs
Gauss sphere, top view
Reciprocal Pair
form an reciprocal pair with respect to ifmn
, vs
,
v
s
o
n 'n
m 'm
d1
d2
d1
d2 'n
n
m
'm
v S
equal BRDF value
Gauss sphere
Reciprocal Curve & Symmetry
Reciprocal curve
Union of reciprocal pairs BRDF symmetry along curve
vs
Gauss sphere, top view
Isotropic – reciprocal quadrilateral
Transformation • Reconstruction via photometric stereo and show how to resolve
shape ambiguities in both “uncalibrated ” and “calibrated ” cases
• Linear transformations of the normal field correspond to projective transformations of the loane , so now disscusses the behavior of our symmetry-induced structure under projective tranformations
Propositions • Proposition 1.
– A rotation (about the origin) and a uniform scaling are the only linear transformations that preserve isotropic pairs with respect to two or more lighting directions that are non-coplanar with the view direction.
• Proposition 2. – If the principal meridian vs is known, a classic bas-relief transformation is the only linear
transformation that preserves isotropic pairs with respect to two or more sources that are non-collinear with the view direction
• Proposition 3.
– If the lighting directions are known, the identity transformation is the only linear transformation that preserves isotropic-reciprocal quadrilaterals with respect to
– two or more lighting directions that are non-collinear with the view direction.
Application : uncalibrated photometric stereo• GBR transformation affects the normals and source directions
• A GBR transformation scales and translates the quadrilateral to
and moves the source to a different point ¯s on the principal meridian
• To resolve the GBR ambiguity, we must find the transformation
that maps back to its canonical position.
• Proposition 4. – The GBR ambiguity is resolved by the isotropy and reciprocity constraints
in a single image.
• By isotropy and reciprocity, an isotropic-reciprocal quadrilateral (n,m, n′,m′) with respect to s, satisfies:
• Using (6)(7) A hypothesis (′ v1, λ1) yields hypotheses for the point h × (v × s) and BRDF value at each point. These in turn induce a hypothesis for the reciprocal match ¯m as the intersection of the iso-BRDF curve and the join of n and the hypothesized h×(v×s)
provides a measure of inconsistency, and the exhaustive 2D search is
used to minimize this inconsistency.
calibrated photometric stereo • Given a set of images I(x, y, t) captured using a cone of known source directions s(t),
t [0, 2) centered ∈ about view direction v, this method yields one component of the normal at every image point (x, y).
• if the surface is differentiable, the surface gradient direction can be recovered
at each point, but the gradient magnitude is unknown. This
means that one can recover the ‘iso-depth contours’ of the
surface, but that these curves cannot be ordered
• Consider a surface S = {x, y, z(x, y)} that is described by a
height field z(x, y) on the image plane. A surface point with
gradient zx, zy is mapped via the Gaussian sphere to point
n (zx, zy,−1≃ ) in the projective plane, and the ambiguity
in gradient magnitude from[1] corresponds to a transformation
of normal field ¯n(x, y) diag(1, 1, (x, y))n(x, y), ≃ where the per-pixel scaling λ (x, y) is unknown
• Proposition 5. • In the general case, if differentiable height fields z1(x, y) and z2 = h(z1) are
related by a differentiable function h and possess equivalent sets of iso-slope contours, the function h is linear
• If the surface has uniform reflectance (or has a uniform separable component), the match ¯n′ can be located by intersecting this line with the iso-intensity contour passing through ¯n. Such isotropic matches ¯n′(t) under all light
• directions s(t), t [0, 2) ∈ define the iso-slope contour
• if the spatially varying BRDF is of the form in Eq. (8), a necessary condition• for two points (x1, y1) and (x2, y2) to have normal directions forming an
isotropic pair is In(x1, y1, t) = In(x2, y2, t) t [0, 2).∀ ∈
• This is because normalizing the temporal radiance at each pixel to [0, 1] removes the effects of the spatially-varying reflectance terms f1 and f2.