Introduction to Computer

Graphics12. Warping and Morphing

National Chiao Tung Univ, Taiwan

By: I-Chen Lin, Assistant Professor

Outline

Image Warping

Image Morphing

Warping of 3D model

Reference:

• T. Beier, S. Neely, "Feature Based Image Metamorphosis," Proc. SIGGRAPH'92, pp. 35-42, 1992.

• A.W.F. Lee, D.Dobkin, W.Sweldens, P. Schroder, “Multiresolution Mesh Morphing, Proc. ACM SIGGRAPH’99. pp.343-350.

• Related articles in SIGGRAPH, Eurographics, ACM Trans. Graphics, IEEE Trans. Visualization and Computer Graphics, and other graphics related publications.

Image Warping

Rearranging pixels of a picture.

It’s useful for both image processing and for computer graphics (namely, for texture mapping).

Finding corresponding points in the source and destination images.

This function is called the “mapping” or “transformation”.

Image Warping (cont.)

u

v source

x

y destination

Forward mapping : (x, y) = f (u, v)

Inverse mapping : (u, v) = f’ (x, y)

Mapping types

How to create the mapping systematically?

Simple mappings

affine mapping

projective mapping

bilinear mapping

………

Affine Mappings

u = ax+by+c

v = dx+ey+f

A combination of 2-D scale, rotation, and translation transformations.

Allows a square to be distorted into any parallelogram.

6 degrees of freedom.

Inverse is of same form (is also affine).

Projective Mappings

u = (ax+by+c)/(gx+hy+i)

v = (dx+ey+f)/(gx+hy+i)

Linear numerator and denominator.

If g=h=0 then you get affine as a special case.

Allows a square to be distorted into any quadrilateral.

8 degrees of freedom.

Inverse is of same form (is also projective)

u = uq/q , v = vq/q

Bilinear Mappings

u=axy+bx+cy+d

v=exy+fx+gy+h

If a=e=0 then you get affine as a special case.

Allows a square to be distorted into any quadrilateral.

8 degrees of freedom.

Not recommended for if you need the inverse.

Bilinear Interpolation

An inexpensive, continuous function that interpolates data on a square grid.

At (0,0), (1,0), (0,1), and (1,1), the corner values are p00, p10, p01, p11.

pxy = (1-x)*(1-y)*p00 + x*(1-y)*p10 + (1-x)*y*p01 + x*y*p11

Another form,

px0 = p00 + x*(p10-p00)

px1 = p01 + x*(p11-p01)

pxy = px0 + y*(px1-px0)

What’s the different between these two forms?

Performing an Image Warp Source scanning:

for v = vmin to vmax

for u = umin to umax

x = x(u,v)

y = y(u,v)

copy pixel at source[u,v] to dest[x,y]

Destination scanning:for y = ymin to ymax

for x = xmin to xmax

u = u(x,y)

v = v(x,y)

copy pixel at source[u,v] to dest[x,y]Is there any problem?

More Complex Situations

How to warp an image with a more complex function?

If the source and destination is of structural similarity

piecewise affine over a triangulation.

piecewise projective over a quadrilaterization.

piecewise bilinear over a rectangular grid.

More Complex Situations …

Arbitrary functions can be used.

Store u[x,y] and v[x,y] in large arrays

Learning techniques: e.g. neural network

………

Popular methods in computer graphics:

Beier-Neely warp (popular for image morphing)

Radial-basis-function (RBF) based data scattering.

………

Morphing

Short for “metamorphosis”.

Early films used cross-dissolving only, but that looks artificial, non-physical.

Morph = warp the shape + cross-dissolve the colors.

If the mappings are defined, cross-dissolving can simply be performed by interpolation.

Feature-Based Image

Metamorphosis

Proposed by Thaddeus Beier, Shawn Neely

Prof. ACM SIGGRAPH’92, pp. 35-42.

Motivation

To transform one image smoothly into another image

Overview

Image

warping

Image

warping

Color

Blending

Concepts (Beier-Neely warp )

How to find a controllable mapping function with little manual assistance?

What’s the problem of mesh-based warping tech.?

How about the concept of “field”?

How to control the mapping? Features

Points

Lines

Curves

……

Fields

Linear

quadratic

……

Source destination

Using control lines

With only one pair of lines

To map the pixel X from pixel X' by corresponding lines PQ and P'Q'.

With only one pair of lines Scan every pixels on the destination image and compute

the corresponding pixels on the source image.

An example (only one pair of lines)

The properties of this approach

Transforms each pixel coordinate by a rotation, translation ,and/or scale.

The image is scaled along the direction of lines by the ratio of the length of the line.

Pure rotation and translation.

Affine transformations. (uniform scales and shear are not possible to specify.)

With multiple pairs of lines Specifying more complex transformations.

The displacement Di = Xi’-X.

A weighted average of these displacements is calculated.

With multiple pairs of lines (cont.)

With multiple pairs of lines (cont.)For each pixel X in the destination

DSUM = (0,0)

weightsum = 0

For each line Pi Qi

calculate u,v based on Pi Qi

calculate X'i based on u,v and Pi'Qi'

calculate displacement Di = Xi' - Xi for this line

dist = shortest distance from X to Pi Qi

weight = (length^p / (a + dist))^b

DSUM += Di * weight

weightsum += weight

X' = X + DSUM / weightsum

destinationImage(X) = sourceImage(X')

An example with two line pairs

Morphing between two images

1. Assign the control lines.

2. CL(t) = (1-t)CL(0)+t CL(1)

3. Generate I0(t), I1(t) according to CL(t)

4. I(t) = (1-t)I0(t) + t I1(t)

Source

CL(0)

I(0)

Destination

CL(1)

I(1)

CL(t)

I0(t)

CL(t)

I1(t)

Performance

Proportional to the number of lines times the number of pixel.

Design time of metamorphosis is the dominant time in interactive design.

Advantage and disadvantage Advantages

It’s much more expressive.

The animator simply has to describe how lines in a source image are mapped into lines in a destination image.

The mesh warping is much less intuitive.

Disadvantages This algorithm must compute the effect of every feature

line on the movement of that pixel. -> slow!

In contrast, mesh warping has only local control .

Sometimes unexpected interpolations are generated.

pixel may be far away from all lines.

Disadvantage (cont.) The algorithm might make a mistake in computing how

that pixel should move during morphing.

Animated sequences Need a set of line segments at key frames for each

sequence.

A sequence from Michael Jackson’s MTV “Black or White”

View morphing Is this warping/morphing tech. applicable to all kinds of

image transitions?

2D image

morphing

What we want :

Out of the scope of an undergraduate CG course

Warping 3D model

For instance, head model adjustment…

Warping 3D model

You can drag all vertices (more than 6000) or drag

feature samples…

Drag by users

What are the

displacement ?

?

?

?

Warping 3D model

There are hundreds of papers about mesh morphing.

You may extend the Beier-Neely warp/morphing from images to 3d models.

Besides control lines, you may also use control points or other control primitives.

An mesh morphing algorithm

A.W.F. Lee, D.Dobkin, W.Sweldens, P. Schroder, “Multiresolution Mesh Morphing, Proc. ACM SIGGRAPH’99. pp.343-350.

An mesh morphing algorithm (cont.)

Assign feature points.

Finding the corresponding points.

Interpolation