View
215
Download
0
Category
Preview:
Citation preview
Relies on:
• Marshall F. Tappen, William T. Freeman and Edward H. Adelson.“Recovering Intrinsic Images from a Single Image.” IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 27, no. 9, pp. 1459-1472, September, 2005 • Matt Bell and William T. Freeman.
“Learning Local Evidence for Shading and Reflectance”Proc. Int’l Conf. Computer Vision, 2001.
Motivation
• Interpreting real-world images
• Distinguish the different characteristics of the scene
• Shading and reflectance – two of the most important characteristics
Our Goal
• Decompose an Input Image into its Intrinsics
• Simple Approaches like Band Filtering won't help us.
for example:
Our Approach
• Recovering the images using multiple cues
• Implicit assumption – surfaces are lambertian (a good starting point…)
• Classify Image Derivatives
Separating Shadows and Reflectance
• As shown in the preceding talk:
• Recovering S and R using derivatives of the input image I
),(),(),( yxRyxSyxI
Creating The Intrinsic Image
• Building S and R is performed in the same manner as shown in the last talk (Weiss)
- convolution operator
imgX - S or R
F – estimated derivative
f – derivative filter – ([-1 1] in out case)
f(-x,-y) – reverse copy of f(x,y)
),(),(),(),( yxfyxfyxfyxfg yyxx
yyxx FyxfFyxfgyximgX ),(),(),(
Binary Classification
• Assumption – each derivative is caused either by Shadings or Reflectance
• This reduces our problem into a binary classification problem
Classifying Derivatives
• 3 Basic phases:1. Compute image derivatives2. Classify each derivative as caused by shading or reflectance3. Invert derivatives classified as shading to find shading images. The reflectance image is found in the same way.
Classifying Derivatives
• The Classifying stage is achieved using two forms of local evidence:
1. color information
2. statistical regularities of surfaces(Gray-scale information)
Color Information
• When speaking of defusive surfaces :
And lights have the same color,changes due to shading should affectR,G and B proportionally
Color Information
• Let and be RGB vectors of two adjacent pixels.
• A change due to shading can be represent as:
α is scalar
(intensity change)
21 CC
1C 2C
Color Information
• If the changes are caused by a reflectance change
• After normalizing and , the dot product will result 1 if the changes are due to shadings ( )
• Practically a threshold is chosen manuallyso:
21 CC
1C 2C
TCC 21ˆˆIf - shading
Else - reflectance
0cos
Color Information
• The threshold eliminates chromatics artifacts caused by JPEG compression for example
• The chosen threshold: cos 0.01
• When speaking of non-lambertian surfaces:
the results are less satisfying
Color Information - examples
Black on white may be interpreted as intensity change.
Resulting in misclassification
Color Information - examples
As before - the face is incorrectly placed in the shadings image
The hat specularity is added to the reflectance image
Gray-scale Information
• Shading patterns have a unique appearance
• We will examine ROIs wrapping each derivative in a gray-scale image to find shadow patterns
Gray-scale classifier
• The Basic Feature
where I is the ROI (patch) surrounding a derivative and w is a linear filter
• the non-linear F is the result in the center of the ROI
wIIF pp )(
Training the classifier
• Two tasks are involved:
1. choosing the filters set – which will build the features
2. training the classifier on the features
AdaBoost (in general)
• Both Tasks are achieved by the chosen classifier – AdaBoost
• First introduced in 1995 by Freund and Schapire
• The main idea is to boost a “weak classifier” – a classifier with error slightly less than 0.5
AdaBoost• The classifier is trained by giving it a training set
• is a binary mapping from the X domain to the Y domain – {-1,+1}
• In our case X is a set of synthetic images of shadings and reflectance,-1 is for reflectance and +1 is for shading
),(),...,,( 11 mm yxyx
),( ii yx
AdaBoost
• AdaBoost is also gets the weak classifier as an input
• The learning stage is iterative• At each round t, AdaBoost weights the training
set and run the weak classifier• The weak classifier job is to find an hypothesis h
such that:
5.0])(Pr[ iit yxh
AdaBoost
• Elements that were misclassified will get a higher weight for the next iteration
• AdaBoost also weights the classifier votes
• At the end – once the desired number of rounds has run, all the weighted votes is gathered to compute the final strong classification H.
T
ttt xhwcWeightsignxH
1
)()(
Our AdaBoost
• The Weak Classifier
where and
recall that
)()( pp IhIh
1,1
1
1)( pIh
If
otherwise
ThresholdIF p )(
wIIF pp )(
• So AdaBoost needs to choosews, thresholds and s
• w – a set of patches constructed from 1st and 2nd derivatives of Gaussian filters
• The training set (which the I’s patches is derived from) is a set of synthetic images
Our AdaBoost
• The shading images were lit from the same direction
• An assumption – when an input image is given, the light direction is known
• Preprocess - rotate the input image so the light will match the light in the training set
Our AdaBoost
GrayScale Information - examples
The shading image is missing some edges
These edges didn”t appear in the training set
Combing Informations
• The final result is based on a statistical calculation of conditional probability
• Assumption: both classifiers (color and gray-scale) are statistically independent
• Bayes rule:
• Each Pr is computed with some modifications on the classifiers
CDGDCDGD
CDGDCGD
rrss
sss /Pr/Pr/Pr/Pr
/Pr/Pr,/Pr
Handling Ambiguities
• Ambiguities - In the former slide for example – the center of the mouse
Shading example Input image Reflectance example
• Derivatives that lie on the same contour should have the same classification
• The mouth corners are well classified as reflectance
Handling Ambiguities
• Areas where the classification is clear are to propagate their classification to disambiguate other areas
• Achieved by a Markov Random Field – which generalize Markov Chains
Handling Ambiguities
• First a potential function is applied on the image finding the “most interesting” gradients
• Then the propagation starts from points having both strong derivatives and no ambiguities
Handling Ambiguities
Recommended