29
1 Photometric Stereo Photometric Stereo Reconstruction Reconstruction Dr. Maria E. Dr. Maria E. Angelopoulou Angelopoulou

1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

Embed Size (px)

Citation preview

Page 1: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

1

Photometric Stereo Photometric Stereo ReconstructionReconstruction

Dr. Maria E. AngelopoulouDr. Maria E. Angelopoulou

Page 2: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

22

Photometric Stereo (PS) Basics

Input images: same viewpoint / different illumination directions

Varying albedo values → minimum of 3 illumination directions

The measured pixel intensities and actual albedo for a given patch o are related as follows

Outputs: orientation & albedo of each surface facet Height map produced after integration of the surface

normals.

(1)

Page 3: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

33

Pros & Cons of PS Compared to Conventional Stereo

Pros+ ability to operate on featureless objects+ absence of feature matching errors+ computational simplicity

Cons- controlled imaging conditions- inaccurate simplifying assumptions- need for off-line calibration sessions & exclusive

access to the imaging system

Page 4: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

44

Objective

Provide an autonomous, purely data-driven PS reconstruction system that is suitable for real-life applications.

Focus on: uncalibrated flatfielding & uncalibrated light estimation.

Diffuse light component as a degradation factor for intended directional lighting. Indicate up to which ratio of ambient to directional light component photometric stereo gives useful reconstruction outputs.

Page 5: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

5

The Inverse-Square Law of Light Propagation

Fundamental assumption of PS: The variation in brightness for a given pixel is solely dependent on the angle between the illumination vector and the surface normal at the corresponding real-world surface facet.

In practice the inverse-square law of light propagation renders the above assumption inaccurate.

To correct the input data appropriately, flatfielding may be employed. Flatfielding employs a set of reference images captured at a dedicated imaging session under the same imaging conditions as the main session. The illumination variations of the reference images is solely due to inhomogeneities of the system.

Page 6: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

66

Standard Calibrated Flatfielding Technique

Use a grey piece of card as calibrating device. Photograph the card multiple times under the same

illumination as the main imaging session. Perform 2D 2nd order polynomial fitting on the flatfielding

reference images to smooth out high frequency noise. This gives the illumination fields .

For every point of the image plane, the new pixel value is computed as

(2)

Page 7: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

77

Standard Calibrated Flatfielding Technique

Page 8: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

88

Uncalibrated Flatfielding

Intensity values decrease across the kth illumination field from the brightest point to the darkest point. Due to the inverse-square law, the 2D illumination field can be approximated with

Only radial distance matters, and thus

Placing the origin of the Cartesian coordinate system at the brightest point:

(3)

(4)

(5)

Page 9: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

9

(6)

(7)

Employing constraints (4) and (5), we get:

In (6) the origin of the coordinate system moves to the brightest point of each kth image. Instead of moving the origin for each k, it is more convenient to express (6) on the fixed image coordinate system as:

Uncalibrated Flatfielding

Page 10: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

10

Uncalibrated Flatfielding

(8)

(10)(9)

(11)

Find the values of the brightest and darkest points:

Consider the location of the brightest and darkest points:

Page 11: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

11

Uncalibrated Flatfielding

Page 12: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

1212

Uncalibrated Flatfielding

Page 13: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

1313

Uncalibrated Flatfielding

Page 14: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

14

PS Reconstruction

(a) no flatfielding, (b) calibrated flatfielding, (c) uncalibrated flatfielding

Page 15: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

15

PS Reconstruction

Page 16: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

16

- The relationship between surface gradients and brightness is captured by the reflectance map of the surface.

- A Lambertian sphere illuminated by a point source in direction

16

Standard Calibrated Estimation of Illumination Directions

- Let denote the global maximum of the kth

reflectance map. It is

has a reflectance map of the form:

(12)

(13)

Page 17: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

17

- If patch o is projected at pixel ,

- Let the gradient at surface patch o be .

17

Standard Calibrated Estimation of Illumination Directions

then the measured image intensity at that pixel is given by the image irradiance equation as

. (14)

- Thus the surface orientation that maximizes the Lambertian reflection component is the one for which the normal vector points to the illumination source!

Page 18: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

18

the global maximum of the Lambertian reflectance mapcorresponds to the maximum intensity measurement , theluminous dot. If the latter resides at and is the projection of the real-world patch m then

18

Standard Calibrated Estimation of Illumination Directions

(13) and (14)- Due to equations

- If a Lambertian sphere is photographed, the normal vector canbe recovered at any point and can be estimated once theluminous dot is identified.

(15)

Page 19: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

1919

Uncalibrated Estimation of Illumination Directions

The proposed uncalibrated illumination vector technique targets the human face class.

A human face can be realistically approximated with a 3D ovoid that is reconstructed on top of the face area. A 3D ovoid is given in the xyz Cartesian system by

where

Page 20: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

2020

Uncalibrated Estimation of Illumination Directions

Page 21: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

2121

Uncalibrated Estimation of Illumination Directions

Page 22: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

22

PS Reconstruction

Page 23: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

23

PS Reconstruction

Page 24: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

24

PS Reconstruction

Page 25: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

25

The Effect of Diffuse Light on PS Reconstruction

PS requires the subject to be illuminated in turn by directional light sources.

The ambient light component that is present constitutes a degrading factor that reduces the directionality of the intended directional component.

Objectives:– Assess the PS robustness with respect to the ratio of

ambient to directional illuminance. – Find the illuminance ratio where reconstruction is no longer

informative of the actual surface.

Page 26: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

26

The Effect of Diffuse Light on PS Reconstruction: Experimental Setup

Floodlight connected to a dimmer provides uniform ambient illumination of varying illuminance.

Light-/Flash-meter used to measure both ambient and directional components.

Page 27: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

27

The Effect of Diffuse Light on PS Reconstruction

λ=9

λ=11

Page 28: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

28

Conclusions

The proposed uncalibrated flatfielding technique is general-purpose and renders similar reconstruction results as its calibrated counterpart.

The proposed uncalibrated light estimation technique is a practical approach that targets the human face class.

The above techniques enable autonomous and reliable surface reconstruction for challenging real-world applications, such as the on-line capturing of human faces.

In the presence of ambient light, PS reconstruction quality decreases linearly as the illuminance ratio of the diffuse to the directional light component increases. PS provides informative outputs for illuminance ratios as high as λ=9.

Page 29: 1 Photometric Stereo Reconstruction Dr. Maria E. Angelopoulou

2929

Thanks for your attention!