Upload
conrad-triquell
View
188
Download
1
Embed Size (px)
Citation preview
Image Matting Using Linear Optimization
Image Analysis and Interpretation
Conrad Triquell
Reminder: What is image matting ?
• A composite image is treated as a linear combination of foreground and background images.
• Compositing equation:
• Where:
The alpha matte of a pixel defines the contributions from foreground/background
User interaction for Matting
1 equation 3 unknowns (grayscale images) 3 equations 7 unknowns (color images)
The image matting problem is severely under-constrained User interaction helps embed constrains into the image
Idea and how we handle the problem?
• Novel matting approach that converts the image matting problem into a simple linear optimization problem.
• 1st) F, B, X, X’ are estimated simultaneously by solving a
system of linear equations using the least square error (LSE) .
• 2nd) Alpha is obtained with the estimated F, B, X, X’ by solving another system of linear equations using LSE estimation.
Note that X, X’ are medium variables.
Formulation
• Define medium variables:
• Compositing equation becomes:
• From I, find X,X’,F and B. From X calculate alpha.
Calculating X, X’, F and B
• P denote the set of pixels of the unknown region and W be a window of size n x n centred at pixel i.
• We enforce a smoothness constraint on F and B for each color channel c= r, g or b. Our goal is to minimize the cost function:
where
• First terms of J1 and J2 are from compositing equations
• Second term are the smoothness constrains.
• Every term in J1 can be rewritten in the form:
where :
represents the values of Xc (or Bc) in the unknown region in vector form.
a=
• Then J1(Xc, Bc)
J2(X’c,Fc)
• Minimizing J1, derivative of J1 with respect to z and set derivative to 0
Therefore we have
For J2 :
• Since J1 and J2 are independent,
are also the closed form solution to minimizing
• By examining J1 and J2 we can see that the solution can also be obtained by solving the following system of over-determined linear equations using LSE estimation:
Calculating α (alpha)
• Given X, X’, F and B:
(Considered medium variable)
from which with additional constraints, the same alpha value for pixel i can be achieved.
• Combining the two equations
• Compute the mean and the variance of the components
• Now our goal is to minimize
where μ are two factors used to balance
In the first term the variance in the exponential allows
more or less deviation of from Second term enforces a smoothness contraint on alpha • J’ is similar to J1 so it can also be rewritten in matrix form
alpha are the values in the unknown region
• The derivation of D and m is similar to A and y we have done before. By minimizing we have:
• Again this result is the same as the result of the LSE estimation of the following over-determined linear equations:
• Note: matrices A and D are large size, which may cause the out-of-memory problem implementation when P is large. Fortunately, these matrices are sparse.
Experimental Result
Comparative results of alpha mattes and composed images with blue background
a) Simulated alpha b) Composed image of a fire foreground, simulated alpha and blue background c) Composed image with a background taken from e) d) Trimap f) Total errors after 2000 experiments
Quantitative evaluation on synthesized images.
Alpha matte and new composed images with new background images
Conclusion
• Novel algorithm to transform the ill-posed image matting problem into two over-determinated optimization problems.
• Closed-form solutions by enforcing the smoothless constraints and introducing two medium variables.
• Steps algorithm: 1) Foreground and background are estimated. 2) The matte is pulled with the estimated foreground and
background. Each step can be solved using linear optimitzation. • High quality image matting
Thanks!
http://www.ee.columbia.edu/~zgli/papers/MM07_Image_Matting.pdf
Reference