Vis Comput (2010) 26: 1229–1239DOI 10.1007/s00371-009-0404-7

O R I G I NA L A RT I C L E

An MLS-based cartoon deformation

Yang Shen · Lizhuang Ma · Hai Liu

Published online: 17 December 2009© Springer-Verlag 2009

Abstract We present an image deformation method drivenby skeleton; it is based on MLS deformation algorithm(Schaefer et al. in SIGGRAPH, vol. 25, pp. 533–540, 2006).We improve the MLS deformation by defining a new weightfunction based on skeleton. Being different from the weightfunction based on control points, our weight function hasbenefited from the shape information of undeformed ob-ject and keeps deformation local, therefore our method canachieve a realistic effect. In cartoon video, we propose anew method to track the skeleton in the video, to build neworigin skeleton and new target skeleton on each frame, andto apply our image deformation method to each frame andmaintain spatiotemporal consistency. Results demonstratethat our method is able to decrease the effect of squeeze anduse less control points.

Keywords Deformation · MLS · Cartoon

1 Introduction

Image deformation has been a prosperous research area inthe field of computer graphics for many years. It has a num-ber of useful applications; for example, a user can changethe pose of a person, or shape of object in the photograph bydragging the mouse in real-time.

Image deformation can also be used in industry; for ex-ample, we can change the position of cartoon man’s foot or

Electronic supplementary material The online version of this article(http://dx.doi.org/10.1007/s00371-009-0404-7) containssupplementary material, which is available to authorized users.

Y. Shen (�) · L. Ma · H. LiuShanghai Jiaotong University, Shanghai, Chinae-mail: sheny@sjtu.edu.cn

hand, the cartoon man can move under the user’s control, sothat we can make a video clip by deforming a cartoon image.These applications demand the intuitive user interaction, re-markable performance and manageable result.

Our method uses skeleton to deform the image. Theskeleton-based approach uses a skeleton, where two or morebones meet at articulating joints to control shape deforma-tion. This allows intuitive control as it naturally describes theway in which many objects (such as animals) deform eachpart of the object that follows motions of underlying bonesin the skeleton. Such methods are usually controlled by auser-chosen skeleton rather than grid line of control points.Compared with other methods, our method is more intuitive.

Our method is based on MLS deformation method(Schaefer et al. [14]).

Let p be a set of control points and q the deformed po-sitions of the control points p. Given a point v in the im-age, we solve for the affine transformation lv(x) that mini-mizes

∑wi |lv(pi) − qi |2 where pi and qi are row vectors

and the weights wi have the form wi = 1‖pi−v‖2a . Because

the weights wi in this least squares problem are dependenton the point of evaluation v, we call this a Moving LeastSquares minimization. Therefore, for each v we obtain a dif-ferent transformation lv(x).

Traditional weight function wi in MLS methods is basedon Euclidean distance. In this paper, we use interior dis-tance instead of Euclidean one. Compared with Euclideandistance, interior distance is more reasonable in MLS de-formation; we will discuss it in the subsequent section. Ourweight function is also based on skeleton. It can keep defor-mation local and get a manageable result in most times.

In this paper, we also apply our deformation method tocartoon video. We specify the original skeleton and targetskeleton on the first frame of video, and locate the new orig-inal skeleton and target skeleton on the next frame by a new

1230 Y. Shen et al.

point trajectory method based on triangle mesh, so that wecan deform the next frame and maintain spatiotemporal con-sistency.

2 Related work

In recent years, more and more people are interested in car-toon film; however, the production of cartoon film is verylimited because of the trivial work on painting in each frame.Using the computer graphics to achieve the trivial work is afeasible way in terms of cartoon animation. An artist canpaint the object in the first frame, and use image/video de-formation technique to change the shape of the object in thefollowing frame, so that a video clip can be made by imagedeformation technique.

Early techniques such as free-form deformation (Seder-berg et al. [16], Lee et al. [8]) use grids as deformation han-dles, which require aligning grid lines corresponding to thecontrol points of the spline to features in the image, and arecumbersome for the user. In the recent applications, the useroften selects such handles as points and lines to control thedeformation. Bookstein et al. [1] present a deformation al-gorithm using control point and use radial-basis functionswith thin-plate splines, but their method cannot keep the de-formation as rigid as possible. Igarashi et al. [5] proposea point-based image deformation technique for cartoon-likeimages which keeps the transformations as rigid as possible.The shape of object is represented by a triangle mesh andthe user moves several vertices (control points) of the meshas constrained handles; the system then computerizes thepositions of the remaining free vertices by minimizing thedistortion of each triangle. Their method can keep the defor-mation as rigid as possible, but it cannot keep deformationlocal: changing each control point can change all pixels inthe image. Weng et al. [18] present a nonlinear optimizationapproach image deformation. Their method aims to preservethe Laplacian coordinates of the shape boundary and theshape of local area. However, solving this system requiresminimizing a nonlinear energy through an iterative method,which is time-consuming. Chiwietz et al. [15] use physics-based models to enhance as-rigid-as-possible deformations;their method is based on a linear elasticity model, and canproduce good effects in many aspects. However, 2D imagedeformation cannot be understood as an elastic transforma-tion in such condition as projection transformation. Schae-fer et al. [14] use moving least squares (MLS) deformationto implement a smooth deformation system. The MLS de-formation technique tries to minimize the local scaling andshearing by solving a small linear system (2 ∗ 2). Deforma-tion based on the MLS can be seen as a least squares prob-lem. By defining a least squares problem on each grid, theMLS does not have to solve a global optimization such as in

Igarashi et al. [5], so it can achieve high performance. Withthis technique, the user can deform images by specifyingcontrol points, which is easy to operate. However, we findmany problems in the practical use of the MLS deformationtechnique. The deformation method in Schaefer et al. [14] isbased on a whole image: foreground and background objectsare not separated. It leads to unexpected squeeze betweenforeground and background objects. The weight function de-fined in Schaefer et al. [14] does not consider the shape in-formation and cannot keep deformation local. In some cases,one cannot get a manageable result.

Skeleton-based deformation has been discussed in manypapers, such as Mohr et al. [10] and Yan et al. [20]. A se-rious problem, however, of traditional skeleton-based meth-ods is that they require a tedious process of weight selectionto obtain satisfactory results. Mohr et al. [10] give reasonswhy the traditional skeleton-based methods are incapable ofrepresenting complex deformations. Yan et al. [20] presenta mesh deformation method that combines the skeleton-based method and the simplex transformation method: theirmethod can avoid the weight adjustment, but it needs trian-gle mesh segmentation and global optimization. Our methoduses the skeleton to deform the image, and succeeds inavoiding the weight adjustment and global optimization.

In this paper, we propose a new weight function to im-prove the MLS image deformation. At first, we segment theimage, extract the foreground from the image, so we cancalculate the new weight function based on skeleton; ourweight function can keep deformation local, which makesdeformation result more realistic.

In cartoon video deformation, we specify the originalskeleton and target skeleton on the first frame of video, buildthe new original skeleton and target skeleton on the nextframe, so that we can deform each frame respectively andmaintain the spatiotemporal consistency.

3 New weight function based on skeleton in MLSdeformation

In MLS deformation method, the image deformation func-tion f is a function that maps the points in the undeformedimage with the points in the deformed image. f should mapthe control points pi directly to qi , and produce smoothdeformation to the other points. By moving least squares,deformation can be defined as the following minimizationproblem:

f = lv(v)E =∑

wi

∣∣lv(pi) − qi

∣∣2 (1)

where lv(x) is an affine transformation that minimizes E,

lv(x) = xM + T . (2)

An MLS-based cartoon deformation 1231

If we expect the deformation to be as rigid as possible, theconstraint MT M = I should be satisfied [14].

In this paper, we are interested in the weight wi in (1).The weight wi plays an important role in the computationof E: wi is large when the point v is near the control pointpi , small when it is far away from pi . When the point v isnear pi and far away from pj , the deformation of point v

will follow the deformation of control point pi rather thancontrol point pj .

In the paper of Schaefer et al. [14], wi is definedby the following formula which uses the Euclidean dis-tance:

wi = 1

(|pi − v|2)2. (3)

The method in Schaefer et al. [14] sometimes can geta perfect result; however, it also produces some artifactswhich we show in Figs. 2 (original image) and 3 (deformedimage). We sway the cartoon man’s hand because the handis near the head in Euclidean distance; the deformation ofhand distorts the cartoon man’s head unexpectedly. A so-lution to this problem is to use the inner distance insteadof the Euclidean distance. Some similar methods have beendiscussed in the previous works, such as Pushkar Joshiet al. [6], Alvaro Cuno et al. [4]: they use the geodesic dis-tance instead of the Euclidean one, in 3D deformation. Inthis paper we use the interior distance instead of the Euclid-ean one in image deformation; it is different from the geo-desic distance.

We segment the undeformed object in the image be-fore deformation. Segmentation plays an important role inthe image deformation application. At first, the user onlywants to deform the specified foreground object in the im-age, and the background object should not be deformed,so we segment and deform the specified object in the im-age, while other parts of the image need not be changed.Secondly, segmentation provides the shape information ofthe object which can be used to estimate the interior dis-tance.

In Fig. 1, the distance from p1 to p2 is defined by theEuclidean distance; a minimal interior distance is basedon l2 norm and a minimal interior distance is based on l1

norm, respectively. The minimal interior distance based onl1 norm is the Manhattan distance. The Manhattan distancecan be simplified and solved by region-growing algorithm:the region grows from control point; when the region growsto next points in surrounding area, the Manhattan distancebetween the control point and next points increases by one.By this method, we can calculate the Manhattan distancebetween control points and each point in image efficiently.Complexity of this algorithm is O(C∗N), N being the num-ber of points in image, C the number of control points in

Fig. 1 For three types of distance, the first one is the Euclidean dis-tance, the second one is the inner distance based on l2-norm, and thelast one is the inner distance based on l1-norm

image.

dis(p1,p2) = min(|p1,pa| + |pa,pb| + |pb,pc|

+ · · · + |pn,p2|)

(p1,pa), (pa,pb), (pb,pc), . . . , (pn,p2)

∈ foreground object, (4)

|pa − pb|1 = |pax − pbx | + |pay − pay |. (5)

We use the Manhattan distance instead of the Euclideanone: it is effective most of the time; however, derivative ofthe Manhattan distance is discontinual and not smooth, sowe use the interior distance introduced in Raif Rustamovet al. [13] instead of the Manhattan distance.

In Raif Rustamov et al. [13], the 2D image object can beembedded into high dimensional space by barycentric coor-dinates, therefore the interior distance between two points issimply defined as their Euclidean distance in the embeddingspace.

p → p∗ = ∑i wi(p)v∗

i , wi(p) is the barycentric coordi-nates, the interior distance of p and q can be represented bythe following equation:

dis(p, q) = ‖p∗ − q∗‖Rm (6)

dis2(p, q) = 〈p∗ − q∗,p∗ − q∗〉Rm

=∑(

wi(p) − wi(q))

× (wj(p) − wj(q)

)〈v∗i , v∗

j 〉Rm

= (−−→w(p) − −−→

w(q))T

A(−−→w(p) − −−→

w(q)). (7)

−−→w(p) = (w1(p),w2(p), . . . ,wn(p))T is the vector of

barycentric coordinate for point p, A is the Grammatrix ofthe dot-products between the embedded vertices v∗

i , Ai,j =〈v∗

i , v∗j 〉Rm .

1232 Y. Shen et al.

Fig. 2 The cartoon man sway his hat

Fig. 3 The old method distorts the cartoon’s head, while we deformthe hand

The matrix A can be solved by the following equation[13]:

J = I − 1

n

−→1

−→1 T , Di,j = d2(vi, vj ),

(8)A = −1

2JDJ,

so we define the new weight function as

wi = 1

dis(pi, v)2. (9)

In formulas (3), (9), the weight wi could not maintaincompactness, and the deformation could not keep local, soit would produce many artifacts in some cases. For example,we expect that the movement of the cartoon person’s handdoes not change the shape of cartoon’s head, but in Fig. 4 wefind that the head is distorted with the movement of hand; toprevent this artifact we expect wi to decrease to zero con-tinuously in a finite range. Let controlrangei be the controlrange of control point pi . When dis(pi, v) ≥ controlrangei ,then wi = 0, the energy wi |lv(pi)−qi |2 is zero in (1), so thecontrol point i could not process point p when dis(p, i) >

controlrangei .

Fig. 4 The control range is too large on the hand

Fig. 5 The control range is too small on the hand

In (1), in order to get an effective solution of least squaresproblem, each point in the image should be controlled by atleast two control points. If control range is too small, defor-mation algorithm produces discontinuity in the area withouteffective control points. In Fig. 5, the control range of con-trol point on the hand is too small, therefore we find discon-tinuous area on the arm after deformation.

For each control point, the control range should be withina reasonable range. If it were too large, the weight functionwould not keep compact and deformation could not keeplocal; if it were too small, it leads to discontinuity. To solvethis problem, we use the skeleton instead of control pointsto handle the deformation.

A skeleton of image is shown in Fig. 6: each control pointis an articulating joint. We can define the control range ofeach control point in a reasonable range by the skeleton. Forexample, the degree of the control points e, c, d is one, so wedefine the control range of control point e between dis(e, a)

and min(dis(e, a)+dis(a, d),dis(e, a)+dis(a, b)); they can

An MLS-based cartoon deformation 1233

Fig. 6 The skeleton of animage

be denoted by the following equation:

dis(e, a) < controlrange(e)

< min(dis(e, a) + dis(a, d),dis(e, a) + dis(a, b)

),

dis(c, b) < controlrange(c) < dis(c, b) + dis(b, a), (10)

dis(d, a) < controlrange(d)

< min(dis(d, a) + dis(a, b),dis(d, a) + dis(a, e)

);

controlrange(e) = dis(e, a) + min(dis(a, b),dis(a, d))

2,

controlrange(c) = dis(c, b) + dis(b, a)

2, (11)

controlrange(d) = dis(d, a) + min(dis(a, b),dis(a, e))

2.

However, the wi in (13) is isotropy, the control range ofpi is in a circle whose center is pi . For the control pointsa, b, the control range should be anisotropic along the dif-ferent directions.

In Fig. 6, the control range of control point a is betweendis(a, b) and dis(a, b) + dis(b, c) along the direction of

−→ab,

is dis(a, e) along the direction of −→ae , is dis(a, d) along the

direction−→ad ; the weight function is anisotropic, so we use

the ellipse in parameter type to define the control range, in-stead of a circle.

Let ∠dab = θ1, ∠dae = θ2, ∠bae = θ3, p be the pointin image; we define the control range by the following equa-tion:

controlrange(a, θ)

= dis(d, a) ∗ cos(πθ/2θ1) + (dis(b, a) + dis(b, c)/2

)

∗ sin(πθ/2θ1), θ = ∠pad, θ ∈ dab,

Fig. 7 Deformation method based on Schaefer et al. [14] with threecontrol points

Fig. 8 Method based on Schaefer et al. [14] with four control pointsand our method with three control points

controlrange(a, θ)

= dis(d, a) ∗ cos(πθ/2θ2) + dis(e, a) (12)

∗ sin(πθ/2θ2), θ = ∠pad, θ ∈ dae,

controlrange(a, θ)

= dis(e, a) ∗ cos(πθ/2θ3) + (dis(b, a) + dis(b, c)/2

)

∗ sin(πθ/2θ3), θ = ∠pae, θ ∈ bae.

We define the weight function by the following formula:

wi = 1

(dis(pi, v))2− 1

(k ∗ controlrange(pi, v))2,

dis(pi, v) < k ∗ controlrange(pi, v), (13)

wi = 0, dis(pi, v) ≥ k ∗ controlrange(pi, v).

Here k is a threshold which prevents control range to betoo small; in this paper, k = 1.2. Our result (Figs. 9, 10, 11)shows that our method achieves an effective result.

We minimize (1) by the method in Schaefer et al. [14];the derivatives with respect to each of the free variables inlv(x) are zero. p∗ and q∗ are weighted centroids, so we can

1234 Y. Shen et al.

Fig. 9 Image deformationbased on Schaefer et al. [14]

Fig. 10 The image deformationbased on our method

Fig. 11 Deformation based onIgarashi et al. [5], the head in thefirst image is distorted, the headin the second image is squeezed

solve the T in (2) by the following equations from Schaeferet al. [14]:

p∗ =∑

i wipi∑

i wi

, p̂i = pi − p∗,

q∗ =∑

i wiqi∑

i wi

, q̂i = qi − q∗,

T = q∗ − p∗M.

So the minimization problem in formula (1) can be solvedby the following formula from Schaefer et al. [14]:

Ai ={

wi

(p̂i

−p̂i⊥)(

v − p∗−(v − p∗)⊥

)T}

, (14)

An MLS-based cartoon deformation 1235

−−→fr(v) =

∑q̂iAi, (15)

fr(v) = |v − p∗|−−→fr(v)

|−−→fr(v)|

+ q∗, (16)

where fr(v) is the deformation map that implements therigid image deformation.

4 Video deformation

In this section, we discuss how to deform a clip of video.We define the original skeleton on the first frame, and definethe target skeleton by dragging the mouse. We record thetarget skeleton by the relative position form; for example, ifthe original bone b1 is rotated by an angle θ scaled by thelength of α ∗ length(b1), the relative position of deformedbone can be represented by (α, θ).

We track each original control point (articulating joint)into the next frame of video, build the new original skele-ton by tracking articulating joints (control points), then webuild the new target skeleton by relative position, so we candeform the subsequent frame by image deformation methodand maintain the spatiotemporal consistency.

Our method is based on tracking the control points. In re-cent years, many methods were proposed in optical flow andpoint trajectory, such as J. Xiao et al. [7], T. Brox et al. [2]and P. Sand et al. [12].

Traditional optical flow algorithm estimates the move-ment of object by the derived grayscale on x-coordinate, y-coordinate and t-coordinate. Optical flow algorithm in lowtextured areas or quickly moving areas often achieves a pooreffect. In cartoon video, there are many areas with low tex-ture and fast moving objects, so the optical flow cannotachieve a perfect effect.

In this paper, we use the SIFT algorithm to match the fea-ture points between two frames instead of particle trajectory.The SIFT algorithm [9] can extract distinctive invariant fea-tures from images and perform reliable matching betweendifferent views of an object or scenes; the features are in-variant to image scale and rotation, a substantial range ofaffine distortion, and change in 3D viewpoint which oftenappears in cartoon.

For example, two photographs record the same person indifferent positions, so we use the SIFT algorithm to find thefeature points such as nose and mouth from two photographsand match those feature points in two photographs respec-tively.

By SIFT matching algorithm, we can match many pointsbetween neighboring frames. The points with high ratiovalue in SIFT matching are called key-points, they are thematching points with high matching accuracy; we use key-points to construct the triangle mesh in video.

We read 8–10 frames and match the key-points in eachframe by the above method, calculate the Voronoi diagramof key points in the first frame, then we get the Delaunay tri-angle mesh by Voronoi diagram. We build the triangle meshon the first frame, then build the triangle mesh on the sub-sequent frames by matching key-points with the previousframe.

For a given triangle T f −1 in the frame f − 1, we get thetriangle T f by matching vertex with T f −1 in the frame f .Since movement of the object in the cartoon video is a par-allel move, rotation and projection in most cases, we find alinear transformation that maps the triangle T f −1 to T f by

T f = BT f −1 + T ∗. (17)

Triangle T f is represented by (Tf

1 , Tf

2 , Tf

3 ), triangle

T f −1 is represented by (Tf −11 , T

f −12 , T

f −13 ), T

fn ,T

f −1n is

the vertex of triangle, B is the transform matrix, T ∗ is thetranslation vector; they can be solved by the following equa-tion:⎧⎪⎪⎨

⎪⎪⎩

U = [T f

1 − Tf

3 , Tf

2 − Tf

3 ],V = [T f −1

1 − Tf −13 , T

f −12 − T

f −13 ],

B = V −1U.

(18)

Having found B , it can be substituted into (17) to ob-tain T ∗.

For a given original control point P f −1 in frame f − 1,we get the triangle T f −1 surrounding the point P f −1. Inthe next frame f , we get the corresponding triangle T f bymatching with the vertex of T f −1 in frame f − 1. We cancalculate the position of P f in frame f by the followingequation:

P f = B(P f −1 − T

f −13

) + T ∗. (19)

By linear transformation, we can track original skeletonin the next frame and calculate the target skeleton in eachframe, so we can apply our image deformation method toeach frame.

Our method is effective when the movement of objectin cartoon video is parallel to the drawing plane. Whenthe 3D movements such as rotations outside the drawingplane are involved, for instance, a character turning its head,we need to improve our method. Because some key-pointsand control points would be invisible with the rotation ofthe object in 3D space, some control points could not betraced, 2D skeleton is difficult to represent by the 3D move-ments of object in 3D space. To solve this problem, we de-compose the 3D movement into a series of 2D movementsapproximately. In each 2D movement, no control point isoccluded and movement can be described by a 2D skele-ton, so we can deform the whole video clip by a series

1236 Y. Shen et al.

of 2D deformations. However, our method will not workwhen skeleton is hard to be tracked. For example, key-points are sparse between neighboring frames and objectchanges fast in the video; in this case, we have to rede-fine the new original skeleton and target skeleton in the newframe.

5 Experimental results

We implemented the described algorithm on a 3.7 Ghz PCwith 2 GB of memory. Compared with old methods suchas Schaefer et al. [14] and Igarashi et al. [5], our methodachieves better results.

In Fig. 9 we show the results of Schaefer et al. [14]. Be-cause the Euclidean distance is used in their weight function,cartoon guy’s head is squeezed by the hand while the handis near the head in the Euclidean distance.

The results of Igarashi et al. [5] are shown in Fig. 11.Their method could not keep deformation local. We movethe left hand of cartoon guy, the head of cartoon guy isdistorted, unexpected squeeze also happens in the exam-ple.

Our results are shown in Fig. 10: a head is not distortedby right hand in the first image by defining control range ofhands. Our result shows that the weight function in (13) canmaintain the shape of the cartoon guy’s head.

In Fig. 7 we use three control points and get a bad resultwhile using the weight function from Schaefer et al. [14].Because the Euclidean distance is used in weight function,the left foot squeezes the other part of the image and pro-duces a rotation. To prevent this effect, in Fig. 8 we define astill control point on the right foot in the left image, but theleft foot squeezes the right leg and twists it. Skeleton withthree control points is applied to a superman in our method.Because our method can keep deformation local, we findthat no squeeze happens, and we need not define a still con-trol point on the right foot. Results show that our methodachieves better results while using less control points.

In Figs. 12, 13, we change the shape of object gradu-ally. In Fig. 14 we raise the woman’s hand by the method ofSchaefer et al. [14] and our method, respectively. In Fig. 13,because the Euclidean distance is used in weight function,we find that the deformation distorts the shape of a woman.

Fig. 12 We sway the cartoon man’s hat by image deformation

Fig. 13 We sway the animal’sear by our method

Fig. 14 We raise the woman’shand by Schaefer et al. [14] andby our method, respectively

An MLS-based cartoon deformation 1237

Fig. 15 Triangle mesh based on SIFT feature in cartoon frames

Fig. 16 Original user skeletonand target user skeleton in thefirst frame

Fig. 17 A result of video deformation: we reduced the size of monster’s mouth

Fig. 18 We create a video clipby deforming a cartoon image

In our method (Fig. 14), we do not discover any unreason-able distortion in the image.

Figure 15 shows the triangle mesh based on SIFT fea-tures. Figure 16 shows the original skeleton and target skele-ton in the first frame. In Fig. 17, the size of the monster’smouth is decreased by our method.

Results in Tables 1 and 2 show performance of ourmethod and methods in Schaefer et al. [14] and Igarashiet al. [5]. Compared with method in Igarashi et al. [5], our

method can achieve a high performance; our method neednot solve large equation group, which is time-consumingwhen the deformed object is made up of large number ofvertices. Compared with the method in Schaefer et al. [14],though our method takes more time in estimating the inte-rior distance which can be pre-computed, it is more effectivein maintaining the shape of deformed object and requires asmaller number of control points. Our method is easy to im-plement on the GPU (Weng et al. [17]). As demonstrated in

1238 Y. Shen et al.

Table 1 The performance statistics of our method and that of Schaeferet al. [14]

Image grid Control points Time

(our method/Schaefer) (our method/Schaefer)

60 ∗ 60 10/10 31 ms/23 ms

440 ∗ 470 3/4 72 ms/57 ms

512 ∗ 512 10/10 95 ms/74 ms

Table 2 The performance statistics of our method and that of Igarashiet al. [5]

Vertices Control points Time

93 3 34 ms

150 3 80 ms

287 3 267 ms

the accompanying video, the cartoon man’s hat is draggedby the mouse in real-time.

6 Conclusions

We have improved the MLS image deformation techniqueby proposing a new weight function in the MLS. Comparedwith older method of Schaefer et al. [14] and Igarashi et al.[5], our method is more effective. We also apply our methodto cartoon video.

Differently from other methods, our method is very prac-tical and can achieve manageable result in most cases. InFigs. 12, 13 and 18, we deform the cartoon object and turnit into an animation sequence. Our method can be applied tocartoon design, it will improve its efficiency.

However, there are still many problems which need tobe solved. In Fig. 10, we find that the texture of the guy’shand is twisted and not ideal. In the future we will focus onhow to deform the image and maintain the important fea-tures. Cartoon vectorizing in Song-Hai Zhang et al. [22] andPrice et al. [11] is easy to control and results in smallerfile sizes. In the future, we will investigate how to deformvectorized images efficiently and apply them in other areas[3, 19, 21].

Acknowledgements This work is supported by National Basic Re-search Program of China (Program 973) under Grant No. 2006CB-303105, Fund for National High Technology Research and De-velopment Program of China (Program 863 No. 2009AA01Z334),Natural Science Foundation of China (No. 60873136). We would liketo thank Scott Schaefer for providing us with the image of the woodenguy.

References

1. Bookstein, F.L.: Principal warps: Thin-plate splines and the de-composition of deformations. PAMI 22, 567–585 (1989)

2. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracyoptical flow estimation based on a theory for warping. ECCV12(1), 25–36 (2004)

3. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., Hu, S.-M.:Sketch2photo: Internet image montage. ACM Trans. Graph. (toappear)

4. Cuno, A., Esperanca, C., Oliveira, A., Cavalcanti, P.R.: 3D as-rigid-as-possible deformations using MLS (2008)

5. Igarashi, T., Moscovich, T., Hughes, J.: As-rigid-as-possible shapemanipulation. In: SIGGRAPH, vol. 24, pp. 1134–1141 (2005)

6. Joshi, P., Meyer, M., DeRose, T., Green, B., Sanocki, T.: Harmoniccoordinates for character articulation. ACM Trans. Graph. 26(3),71 (2007)

7. Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M.: Bilateralfiltering-based optical flow estimation with occlusion detection.ECCV 9(4), 211–224 (2006)

8. Lee, S.Y., Chwa, K.Y., Shin, S.Y.: Image metamorphosis usingsnakes and free-form deformations. Comput. Graph. (Ann. Conf.Ser.) 29, 439–448 (1995)

9. Lowe, D.G.: Distinctive image features from scale-invariant key-points. Int. J. Comput. Vis. 60, 91–110 (2004)

10. Mohr, A., Tokheim, L., Gleicher, M.: Direct manipulation of in-teractive character skins. In: Proceedings of the 2003 symposiumon Interactive 3D Graphics, pp. 27–30. ACM, New York (2003)

11. Price, B., Barrett, W.: Object-based vectorization for interactiveimage editing. Vis. Comput. 22(9), 661–670 (2006)

12. Sand, P.: Particle video: Long-range motion estimation using pointtrajectories. In: CVPR, vol. 24, pp. 2195–2202 (2006)

13. Rustamov, R., Lipman, Y., Funkhouser, T.: Interior distance usingbarycentric coordinates. In: Computer Graphics Forum (Sympo-sium on Geometry Processing), 28(5), July 2009

14. Schaefer, S., Mcphail, T., Warren, J.D.: Image deformation us-ing moving least squares. In: SIGGRAPH, vol. 25, pp. 533–540(2006)

15. Schiwietz, T., Georgii, J., Westermann, R.: Free-form image,pp. 27–36 (2007)

16. Sederberg, T.W., Parry, S.R.: Free-form deformation of solid geo-metric models. In: SIGGRAPH ’86: Proceedings of the 13th An-nual Conference on Computer Graphics and Interactive Tech-niques, pp. 151–160. ACM, New York (1986)

17. Weng, Y.-L., Shi, X.-H., Bao, H.-J.: Sketching mls image defor-mation on the GPU. Comput. Graph. Forum 27(7), 1789–1796(2008)

18. Weng, Y., Xu, W., Wu, Y., Zhou, K., Guo, B.: 2D shape defor-mation using nonlinear least squares optimization. Vis. Comput.22(9), 653–660 (2006)

19. Xu, K., Li, Y., Ju, T., Hu, S.-M., Liu, T.-Q.: Efficient affinity-basededit propagation using k-d tree. ACM Trans. Graph. (to appear)

20. Yan, H.-B., Hu, S., Martin, R.R., Yang, Y.-L.: Shape deformationusing a skeleton to drive simplex transformations. IEEE Trans.Vis. Comput. Graph. 14(3), 693–706 (2008)

21. Zhang, G.-X., Cheng, M.-M., Hu, S.-M., Liu, R.R.M.: A shape-preserving approach to image resizing. Comput. Graph. Forum 28,2009 (1897–1906)

22. Zhang, S.-H., Chen, T., Zhang, Y.-F., Hu, S.-M.: Vectorizing car-toon animations. TVCG 99(2), 1077–2626 (2009)

An MLS-based cartoon deformation 1239

Yang Shen received the Master’sdegree from Department of Com-puter Science and Technology, HeFeiUniversity of Technology, China in2004. He is currently working to-ward the Ph.D. degree in the De-partment of Computer Science andTechnology, Shanghai Jiao TongUniversity. His research interests in-clude computer graphics and imageprocessing.

Lizhuang Ma received his B.Sc.and Ph.D. degrees at Zhejiang Uni-versity, China in 1985 and 1991, re-spectively. He was a Post-doctor atthe Department of Computer Sci-ence of Zhejiang University from1991 to 1993. Dr. Ma was promotedas an Associative Professor and Pro-fessor in 1993 and 1995, respec-tively. Dr. Ma stayed at ZhejiangUniversity from 1991 to 2002. Heis now a Full Professor, Ph.D. Tu-tor, and the Head of Digital MediaTechnology and Data Reconstruc-tion Lab at the Department of Com-

puter Science and Engineering, Shanghai Jiao Tong University, Chinasince 2002. He is also the Chairman of the Center of Information Sci-ence and Technology for Traditional Chinese Medicine at ShanghaiTraditional Chinese Medicine University. Dr. Ma has published morethan 100 academic research papers in both domestic and internationaljournals. His research interests include computer-aided geometric de-sign, computer graphics, scientific data visualization, computer anima-tion, digital media technology, and theory and applications for com-puter graphics, CAD/CAM.

Hai Liu received the M.Sc. degreein Department of Computer Scienceand Engineering of Taiyuan Uni-versity of Technology. Now he isa Ph.D. candidate in Department ofComputer Science and Engineering,Shanghai Jiao Tong University. Hiscurrent research interests includeimage/video editing, computer vi-sion, digital media technology, andvisualization in scientific comput-ing.