Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Image Warping and Morphing
Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition
CS4243 Image Warping and Morphing 1
Morphing is artistic
CS4243 Image Warping and Morphing 2
Morphing is captivating!
CS4243 Image Warping and Morphing 3
Morphing is dynamic!
CS4243 Image Warping and Morphing 4
Morphing is fun!
CS4243 Image Warping and Morphing 5
Morphing is fun!
CS4243 Image Warping and Morphing 6
OK, that was not really morphing.
But, you get the point.
Two kinds of morphing Image morphing
2D
Object morphing 2D: like image morphing 3D: more complex
CS4243 Image Warping and Morphing 7
our focus
Image Warping & Morphing Involves 3 things
Spatial transformation Colour transfer Continuous transition
CS4243 Image Warping and Morphing 8
image warping
Image Warping & Morphing Involves 3 things
Spatial transformation Colour change Continuous transition
CS4243 Image Warping and Morphing 9
Spatial Transformation Simplest form: affine transformation
CS4243 Image Warping and Morphing 10
Affine Transformation
CS4243 Image Warping and Morphing 11
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
−
1005.010
8.001
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
translation
Affine Transformation
CS4243 Image Warping and Morphing 12
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
10002.10008.0
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
scaling
Affine Transformation
CS4243 Image Warping and Morphing 13
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
100011.002.01
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
shearing
parallel lines remain parallel
Affine Transformation
CS4243 Image Warping and Morphing 14
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
−
1000cossin0sincos
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
rotation
parallel lines remain parallel
Affine Transformation To solve for affine matrix:
For i = 1,…, n, arrange into two matrix equations:
Then, solve each equation using linear least square.
CS4243 Image Warping and Morphing 15
Perspective Transformation Generalisation of affine transformation:
Looks like a linear equation. But, perspective transformation is nonlinear. Will discuss further later.
CS4243 Image Warping and Morphing 16
Polynomial Transformation Described by polynomial equations
Example: 2nd-order polynomial, e.g., quadratic
CS4243 Image Warping and Morphing 17
Quadratic Transformation In matrix form
Solving the parameters is easy! Just like for the affine case (exercise).
CS4243 Image Warping and Morphing 18
If these are zero, what is the matrix?
Quadratic Transformation
CS4243 Image Warping and Morphing 19
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
0100000011.000
tapering / perspective distortion
Quadratic Transformation
CS4243 Image Warping and Morphing 20
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
− 0101.000
001000
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
tapering / perspective distortion
Quadratic Transformation
CS4243 Image Warping and Morphing 21
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
− 01001.00
001001.0
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
expansion / compression
Quadratic Transformation
CS4243 Image Warping and Morphing 22
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
− 010001.0
00101.00
Source (x, y) Target (u, v)
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
warping
General Transformation General form of u and v
Possible kernel functions: Affine Polynomial Splines: B-splines, cubic splines, etc. Thin-plate spline: model physical bending energy
CS4243 Image Warping and Morphing 23
),(1
yxhau k
K
kk∑
=
=
parameters kernel function
Which kernel to use? General guidelines
Sufficient to capture transformation Not overly complex
CS4243 Image Warping and Morphing 24
Example
Fit affine transformation. Correct transformation, small error.
CS4243 Image Warping and Morphing 25
source target mapped source
Example
Fit affine transformation. Incorrect transformation, large error.
CS4243 Image Warping and Morphing 26
source target mapped source
Example
Fit quadratic transformation. Looks correct, small error; but can over fit.
CS4243 Image Warping and Morphing 27
source target mapped source
Example
Fit quadratic transformation. Correct transformation, small error.
CS4243 Image Warping and Morphing 28
source target mapped source
Image Warping & Morphing Involves 3 things
Spatial transformation Colour transfer Continuous transition
CS4243 Image Warping and Morphing 29
Colour Transfer Determine colour in transformed image. Remember to use backward mapping plus
bilinear interpolation
CS4243 Image Warping and Morphing 30
Colour Transfer Forward mapping
Copy colour of p in I to q in I’. But q is at real-valued coordinates; problem.
CS4243 Image Warping and Morphing 31
source target
Colour Transfer Backward mapping
Map q in I’ to p in I. Use bilinear interpolation to determine colour of p. Copy colour of p to q.
CS4243 Image Warping and Morphing 32
source target
Image Warping Example With affine mapping
CS4243 Image Warping and Morphing 33
Image Warping Example With affine mapping
CS4243 Image Warping and Morphing 34
Image Warping Example With affine mapping
CS4243 Image Warping and Morphing 35
Image Warping Example With affine mapping
CS4243 Image Warping and Morphing 36
Image Warping Example With quadratic mapping
CS4243 Image Warping and Morphing 37
Why so much distortion?
Image Warping Example With quadratic mapping
CS4243 Image Warping and Morphing 38
Image Warping Example With quadratic mapping
CS4243 Image Warping and Morphing 39
Image Warping Example With quadratic mapping
CS4243 Image Warping and Morphing 40
Local Transformation Divide image into regions. Warp each region by a different transform.
Ensure changes over boundaries are smooth.
Achieve finer control.
CS4243 Image Warping and Morphing 41
Summary Need to mark good corresponding points. Lower-order function may not warp enough. Higher-order function can lead to distortions. Local transformations give finer control.
CS4243 Image Warping and Morphing 42
Image Warping & Morphing Involves 3 things
Spatial transformation Colour transfer Continuous transition
CS4243 Image Warping and Morphing 43
Image Morphing Basic ideas:
Want positions to change smoothly. Want colours to change smoothly.
CS4243 Image Warping and Morphing 44
Image Morphing Simplest transition: linear
CS4243 Image Warping and Morphing 45
pi qi
ri
I J
morphing path
CS4243 Image Warping and Morphing 46
Morphing path is actually like this:
Image Morphing Set time step ∆t = 1 / number of frames. Repeat for 0 ≤ t ≤ 1
Warp I to I(t) by mapping pi to ri(t). Warp J to J(t) by mapping qi to ri(t). Blend I(t) and J(t) into M(t)
Save M(t) into a video.
CS4243 Image Warping and Morphing 47
Example
CS4243 Image Warping and Morphing 48
Anything peculiar?
CS4243 Image Warping and Morphing 49
Sample frames
Summary For seamless morphing
Match all corresponding features. Need accurate transformation.
For smooth morphing Use smaller time step.
CS4243 Image Warping and Morphing 50
Further Reading More sophisticated image warping: [Wolberg90]. More sophisticated image morphing: [Lee96]
CS4243 Image Warping and Morphing 51
References S.-Y. Lee and S. Y. Shin. Warp generation and transition control in
image morphing. In Interactive Computer Animation. Prentice Hall, 1996.
G. Wolberg. Digital Image Warping. IEEE Computer Society Press, 1990.
CS4243 Image Warping and Morphing 52