CPSC 524 - Geometric Modeling - Course Project ReportSubspace Mesh Deformation with

Constrained Nonlinear Least Squares Energy

Minchen Li∗

University of British Columbia

Figure 1: Mesh deformation using ARAP (middle) and our subspace mesh deformation method with constrained nonlinear least squaresenergy (right). The blue vertices are fixed, and the orange vertices are moving through the red arrow downwards and backwards as shownin the left picture. It’s obvious that ARAP generates unnatural neck shape in this extreme deformation case, while our method gives a bettersolution although the head is slightly elongated.

Abstract

Our project aims at implementing a basic version of subspace meshdeformation method with constrained nonlinear least squares en-ergy following [Huang et al. 2006]. This report will cover the keypoints in [Huang et al. 2006] with many untouched mathematicalderivations and implementation details, which could be served asa supplemental material to the original paper for potential readers.Besides, some experiments will be conducted and the results withcomparison and analysis will be demonstrated.

1 Specification

The input meshes will be a refined closed-manifold mesh with arelativly large number, n, of vertices and a closed-manifold coarsecontrol mesh1 with a much smaller number, m, of vertices. It isrequired that the refined mesh needs to lie exactly inside the interiorof the coarse control mesh.

Given the input meshes, a mapping W ∈ R3n×3m between thecoarse control mesh and the refined mesh will be constructed usingthe mean value interpolation method [Ju et al. 2005; Floater 2003].Then W will be used to map the energy function defined on therefined mesh to the subspace of the coarse control mesh.

The energy function works on preserving surface details, mesh vol-ume, and some objective vertex positions, where the objective ver-tex positions are defined via user interactions. Note that here thisscheme is different from free-form deformation (FFD) [Sederbergand Parry 1986] where energy terms are directly defined on thecoarse control mesh. They have different mathematical forms.

∗e-mail:minchenl@cs.ubc.ca1The coarse control mesh could be constructed by applying the progres-

sive convex hull construction algorithm in [Sander et al. 2000] on the refinedmesh, or by using a modeling software. This will not be a part of the imple-mentation in this project

Finally, to solve for vertex coordinates for the deformed mesh, theenergy function will be minimized in the subspace, which will pro-vide better numerical stability, faster convergence, and less runtimeand memory cost than that in the original space.

Through this project, we can learn the space reduction idea, whichis very useful in many engineering scenarios. Since the mesh de-formation problem here is formulated as a nonlinear least squaresproblem, getting experience on applying Gauss-Newton methodwith backtracking line search is also a big benefit.

The rest of the report is organized as follows: §2 formulate the meshdeformation problem as a constrained nonlinear least squares prob-lem (§2.1), which will be solved by Gauss-Newton iterations (§2.2)with backtracking line search (§2.3). Then the subspace solve willbe discussed in §2.4, so does the space reduction idea (§2.5). In or-der to provide more useful details on implementation, the computa-tion and verification of derivatives is presented in (§2.6) at length.The experiments and results will be demonstrated in §3 where oursubspace solve will be compared with the full space solve (§3.1)and also the ARAP [Sorkine and Alexa 2007] (§3.2, §3.3). Finally,we will conclude our work and provide possible future directions in§4.

2 Formulation and Implementation

2.1 A Constrained Nonlinear Least Squares Problem

With the rotation invariant nonlinear Laplacian constraint LX =

δ(X) and position constraint ΦX = V being the soft constraints,and with volume constraint ψ(X) = v being the hard constraint,we can formulate our mesh deformation problem as a constrainednonlinear least squares problem [Huang et al. 2006]

minX

1

2||f(X)||2 subject to g(X) = 0 (1)

1

where

f(X) =

[LΦ

]X −

[δ(X)

V

]g(X) = ψ(X)− v

(2)

Here X ∈ R3n is the vertex coordinates of the refined meshes,L is the laplacian matrix constructed using the cotangent weights[Desbrun et al. 1999] of the undeformed refined mesh, δ(X) is therotation invariant expression of the refined meshes, and ψ(X) =16

∑Tijk

(xi × xj) · xk is the volume of the refined meshes. δ(X)

and ψ(X) are nonlinear functions of X . The position constraintwill be defined by user interaction, and the solved X which givesthe minimal objective function value will be taken as the vertexcoordinates of the deformed refined mesh.

2.2 Gauss-Newton Iterations

To solve the problem, Gauss-Newton iterations [Wedderburn 1974]are applied. We linearly approximate f(X+h) ≈ l(h) ≡ f(X) +Jf (X)h at each iteration and solve

minh

1

2||l(h)||2 subject to g(X + h) = 0 (3)

By locally linearizing g(X + h) ≈ g(X) + Jg(X)h and applyingLagrange multipliers [Bellman 1956] with Newton’s method [Kel-ley 2003], we can express the local update that minimizes Eq.3 as:

h = −(JTf Jf )−1(JTf f + JTg λ)

λ = −(Jg(JTf Jf )−1JTg )−1(g − Jg(JTf Jf )−1JTf f)

(4)

where Jf ≡ Jf (X) =

[L − Jδ(X)

Φ

]and Jg ≡ Jg(X) =

∇Xψ(X). Thus starting from an initial X0, we can solve Eq.1iteratively by computing the update hk from Eq.4 (assuming X =Xk−1) and then setting Xk = Xk−1 + αhk, where α is a smallconstant that can be found by line search.

2.3 Backtracking Line Search with Armijo Condition

In each iteration k after we solve for hk, we set pk = hk/||hk||,αk = ||hk|| and test whether αk satisfy

1

2||f(Xs)||2+λg(Xs) ≤

1

2||f(Xk−1)||2+λg(Xk−1)+c1m (5)

where

Xs = Xk−1 + αkpk

m = αkpTk ((Jf (Xk−1))T f(Xk−1) + λ(Jg(Xk−1))T )

(6)

If the above Armijo-Goldstein condition [Armijo 1966] is not satis-fied, we set αk to c2αk, and test the condition again iteratively untilit is satisfied or αkpk becomes too tiny with respect to the boundingvolume size of Xk−1. Here c1 and c2 are two constants usually setto c1 = c2 = 0.5 when Armijo first publish it, or c1 = 1.0×10−4,c2 = 0.9 in Wolfe condition [Wolfe 1969].

Note that in Eq.5,m = αkpTk (∂( 1

2||f(X)||2+λg(X))/∂X). This

backtracking line search method ensures that in each iteration, theLagrange function value decreases sufficiently, which is necessaryin most of the Newton-like methods since the linearly approximatedlocal minimum may even not be a descent update of the originalnonlinear objective.

2.4 Solve in Subspace

Solving Eq.1 with iterative methods will run into serious problemswith slow convergence and numerical instability. Often the stabilityproblem is so severe that the iterations do not converge. The twodominating causes for the instability are the large condition numberκ(JTf Jf ) of the matrix JTf Jf and the nonlinearity of δ(X). Thesubspace deformation technique is designed to address these issues.

(a) Horse (b) Cow head

Figure 2: The coarse control meshes used in this report constructedusing 3ds Max.

With a coarse control mesh around the original mesh (e.g., Fig.2),the deformation energy and the hard constraints are then projectedonto the control mesh vertices using mean value interpolation [Juet al. 2005; Floater 2003]. Let the control mesh vertices P ∈ R3m

be related to original mesh vertices X ∈ R3n through X = WP .After projection we perform energy minimization in the controlmesh subspace as follows:

minP

1

2||f(WP )||2 subject to g(WP ) = 0 (7)

which will change the Gauss-Newton iteration update to

hP = −(WTJTf JfW )−1(WTJTf f +WTJTg λ)

λP = −(JgW (WTJTf JfW )−1WTJTg )−1η

η = (g − JgW (WTJTf JfW )−1WTJTf f)

(8)

This can just be derived by substituting Jf and Jg with their sub-space projections JfW and JgW into the full space update Eq.4.The line search will also need to be conducted in the subspace.

In fact, this subspace solve is just equivalent to adding one morehard equality constraint X = WP to the original problem, so thatthe degree of freedom decreases from |X| to |P |. Since |P | is muchsmaller than |X|, the linear systems to be solved at each iterationare relatively small. Furthermore, using the smoothness of the meanvalue coordinates we can show that, for a properly constructed con-trol mesh, κ(WTJTf JfW ) has magnitudes smaller than κ(JTf Jf )

and the nonlinearity of δ(WP ) with respect to P is significantlyreduced from that of δ(X) with respect to X .

Most importantly, this technique does not simply apply constraintsand solve the deformation on the coarse mesh P and interpolateback the results to the original mesh X; this naive approach wouldcertainly not preserve mesh properties on the original mesh.

One thing worth noticing is that the time and memory complexityreduction of subspace solve compared to the original solve dependson a more strict dimensionality condition, just |P | < |X| is notenough since W is dense and so does all the subspace systems,while in original space the systems are sparse.

2

2.5 The Space Reduction Idea

The design of the subspace deformation solver is based on two ob-servations: (1) the key in gradient domain deformation is to de-form the low frequency coarse shape while maximally preservingthe high frequency features such as surface and skeleton details.Thus, in the view of spectral analysis via singular value decompo-sition, the deformation is mostly performed in a subspace definedby low frequency features. (2) If the subspace deformation for-mulation is robust and only involves a small number of variables,then the (inexact) Gauss-Newton method can converge rapidly andhence we can meet the interactive deformation requirements.

To determine a quality subspace and its parameterization, ideally,one can use spectral analysis to capture the subspace of low fre-quency features: Consider a deformation X = X0 +D, where X0

denotes the original mesh position and D is the desired displace-ment of the deformation. Let L be the Laplacian matrix. Then thechanges in the differential coordinates is LX − LX0 = LD. LetL = USV T be the SVD of L, and D =

∑j djVj be an expansion

using the singular vectors Vj in V . Then ||LD||2 =∑j(djsj)

2,where sj are the j-th singular values. In order to preserve the highfrequency surface details, D should lie in a subspace formed by theset of singular vectors with small singular values. So one can form areduced subspace in which energy minimization is performed in thesubspace formed by the singular vectors in V with small singularvalues.

In practice, it could be expensive to compute the SVD ofL for largemeshes. So the alternatives would be to apply mesh simplificationtechniques [Sander et al. 2000] or to use modeling software.

2.6 Computation and Verification of Derivatives

Computing Jδ(X): We will evaluate Jδ(X) analytically by ap-plying chain rule:

Jδ(X) =∂δ(X)

∂d(X)

∂d(X)

∂X

Since δ:i(X) = (γi/||d:i(X)||2)d:i(X), we have

∂δ(X)

∂d(X)= blockDiag(

∂δ:i(X)

∂d:i(X)) , i = 1, 2, ..., n

(Since notation Xk has been used to refer to vector X in k-thGauss-Newton iteration, we use X:i in this report to refer to thei-th element/dimension of a vector.)

Let d:i(X) ≡ (x, y, z), we have

∂δ:i(X)

∂d:i(X)=

γi

(x2 + y2 + z2)32

y2 + z2 −xy −xz−xy x2 + z2 −yz−xz −yz x2 + y2

For ∂d(X)/∂X , we also consider per vertex ∂d:i(X)/∂X:

∂d:i(X)

∂X=

N (i)∑j

µij∂((xj−1 − xi)× (xj − xi))

∂X∈ R3×3n

(In this report, we assume gradients to be row vectors, while all theother vectors to be column vectors.) and we can construct it as:

∂d(X)

∂X= [(

∂d:1(X)

∂X)T , (

∂d:2(X)

∂X)T , ..., (

∂d:n(X)

∂X)T ]T

As we know, cross product can be written in matrix-vector productform:

a× b = [a]×b =

0 −az ayaz 0 −ax−ay ax 0

bthus we can compute the derivative in a more concise form:

∂((xj−1 − xi)× (xj − xi))∂X

=∂(xj−1 × xj + xi × xj−1 + xj × xi)

∂X=[...,03,3, [xj ]× − [xj−1]×,03,3,

...,03,3, [xi]× − [xj ]×,03,3,

...,03,3, [xj−1]× − [xi]×,03,3, ...]

(9)

Computing Jg(X): For Jg(X), we have:

Jg(X) = ∇Xψ(X) =1

6

∑Tijk

∂(xi × xj) · xk∂X

where we first calculate the gradient per triangle as follows and addthem up:

∂(xi × xj) · xk∂X

=[..., 0, xTk [−xj ]×, 0,

..., 0, xTk [xi]×, 0,

..., 0, (xi × xj)T , 0, ...]

(10)

Verifying Jδ(X) and Jg(X): Since the above analytic deriva-tions bear certain complexity, it’s necessary to ensure that they arecoded correctly. We verify the computation by comparing them totheir corresponding finite difference results.

According to finite difference methods [Ascher and Greif 2011], wecan compute Jδ(X) and∇Xψ(X) as

Jδ(X) = [(∇X:0 δ(X))T , (∇X:1 δ(X))T , ..., (∇X:3n δ(X))T ]T

∇Xψ(X) = [∂ψ(X)

∂X:0,∂ψ(X)

∂X:1, ...,

∂ψ(X)

∂X:3n]

(11)where

(∇X:t δ(X))T =δ(X + εe:t)− δ(X)

ε∂ψ(X)

∂X:t=ψ(X + εe:t)− ψ(X)

ε

(12)

and we use ε = 1.0× 10−5 in our program.

Although finite difference method is much more straight forwardand easy to implement, it usually costs more running time than an-alytic computation. Besides, the precision of the results given byfinite difference is not easy to control sometimes due to the tradeoff between truncation error and round-off error, which is also de-pendent on the properties of the objective function. Thus, we useanalytic computation in our Gauss-Newton solver.

3 Experiments and Results

3.1 Full space v.s. Subspace

We first compare the full space solve and the subspace solve of ournonlinear least squares based mesh deformation method.

3

(a) User interaction (b) Full space solve (c) Subspace solve

(d) Objective function values in each Gauss-Newton iteration.

Figure 3: Comparison of full space solve and subspace solve. Theposition constraint is given by fixing the left horn singularity anddisplace the right horn singularity of the cow head model towardsleft as shown in (a). From (b) we can see that full space solve iseasily trapped in bad local minimum, while from (c) we see thatsubspace solve gives a much better solution. (d) shows that sub-space solve is converging faster and better than full space solve.

Fig.3 shows the results and convergence curve of a very simple ex-periment, where there are only two vertices whose positions areconstrained. With full degree of freedom in the shape space, the fullspace solve is just trapped in a local minimum where the positionconstraints are not well satisfied. However, the subspace solve pro-vides a very meaningful result with much smaller objective functionvalue. Both of the two iterations stopped when there are no signif-icant changes of X allowed during the line search. In this sense,we can say that solving in subspace provide better search directionsfor the nonlinear minimization problem. This is because the searchspace of the subspace solve only contains low frequency features,i.e. the coarse structure, of the mesh, where high frequency featureswon’t interfere the search, thus directing the search to those localminimums that are more likely to be expected.

3.2 Subspace v.s. ARAP: Large Rotations

Next, we compare our subspace solve with as-rigid-as-possible(ARAP) mesh deformation solved by a local-global scheme[Sorkine and Alexa 2007]. ARAP is also a kind of iterative method,it conquers the local rotation variance obstacle of the basic laplaciancoordinates by approximating better pre-scripted rotation iterativelyvia a local stage applying SVD to the deformation gradient at eachvertex, and then a global stage solves a laplacian surface editingproblem [Sorkine et al. 2004] with the newly pre-scripted rotationsapplied on the original laplacian coordinates.

However, if the given position constraint moves some vertices faraway from their original position, which means large rotations areexpected in the final deformed shape, it is also not easy for ARAP to

(a) User interaction (b) ARAP iter100 (c) Subspace solve iter10

Figure 4: Comparison of ARAP and our subspace solve. The po-sition constraint is given by fixing the left horn singularity and dis-place the right horn singularity of the cow head model upwardsand backwards as shown in (a). From (b) we can see that althoughARAP is able to introduce local rotations to preserve surface de-tails, it is still not able to introduce large global rotations to reachbetter surface preservation. But from (c) we see that our subspacesolve gives a solution with global rotation and a wider pair of hornsingularities.

approximate the rotations. Although it will converge, a local min-imum with bad pre-scripted rotations could be encountered, whereFig.4b is just an example. The user interaction shown in Fig.4afixed the left horn singularity and drags the right one upwards andbackwards. Since only two vertex positions are constrained, largeglobal rotations could be involved to preserve surface details better,and this is what our subspace solve produces in Fig.4c.

In fact, while implementing ARAP, people usually slices the dis-placement of the constrained vertices via mouse drag into tinypieces and conduct the local-global solve between each mouse dragevent with the pre-scripted rotations of the last solve being the ini-tial guess for the current one. This can be viewed as an outer iter-ative framework for the deformation problem. If the displacementof the constrained vertices is small, or if the initial guess for the ro-tations is just close to the expected optimum, ARAP is able to giveresults containing large and global rotations, which is why slicingworks. Another way to provide initial guess on pre-scripted rota-tions for ARAP is to use bounded bi-harmonic weights [Jacobsonet al. 2011] based on user defined rotations on a small number ofhandles.

3.3 Subspace v.s. ARAP: Volume Preservation

Finally, we show the volume preservation property of our subspacesolve via another mesh deformation case where ARAP fails.

As shown in Fig.5, if we fix the two front legs of the horse andlift the mouse up and left, the ARAP solve is failed to preservethe shape of the horse mouse, while our subspace solve can. Thisis how our volume perserving nonlinear constraint is superior toARAP. ARAP tried to preserve volume via rotation approximatedsurface laplacian constraint, which only works for non-extremecases. For an extreme case like Fig.5, when the position constraintis strong enough, ARAP may even produce self-intersected outputmeshes.

However, our volume preserving constraint is not perfect, becauseit only tries to preserve the global volume, which means a situa-tion might probably occur in some extreme cases where one partof the mesh volume shrinks while another part of the mesh volumeincreases, e.g., in Fig.1-right, the root of the neck is shrinked whilethe mouse is elongated a little bit. Besides, once self-intersectionhappens during the iteration, when the interior of the mesh couldn’tbe well defined, the solve will proceed to unpredictable solutions.

4

(a) User interaction (b) ARAP iter100 (c) Subspace solve iter13

Figure 5: Comparison of ARAP and our subspace solve. The po-sition constraint is given by fixing the two front legs and displaceone point on the horse mouse up and left as shown in (a). From (b)we can see that although ARAP is able to lift the head up, it fails topreserve the volume of the mouse. But from (c) we can see that oursubspace solve gives a solution with the head lifted and the shapeof the mouse preserved.

4 Conclusions and Future Works

To conclude, we implemented a basic version of subspace meshdeformation method with a constrained nonlinear least squares en-ergy. The problem is solved by applying Gauss-Newton iterationwith Lagrange multiplier and backtracking line search. With thesearch space reduced to the expansion of low features via a map-ping between the original mesh and the corresponding coarse con-trol mesh, the subspace solve converges faster with better solutionthan that in the full space. Compared to ARAP, our subspace solveis able to preserve global volume exactly and estimate the local ro-tations well even if the displacement of the constrained vertices islarge.

For future works, it is very interesting on what will happen to thedeformed mesh when we apply SVD on the Laplacian matrix ofthe original mesh to generate a subspace that is truly with only lowfrequency features theoretically and then solve the problem in thissubspace. Since SVD for large matrices is extremely expensive,we can start with some fast approximation of SVD. Even with theapproximated SVDs, it is possible to get better results and insightssince constructing coarse control mesh by modeling or mesh sim-plification is hard to relate the mesh quality to the space reductiontheory. Besides, we can construct the mapping matrix by apply-ing positive mean value coordinates (PMVC) [Lipman et al. 2007],which is an improved version of mean value interpolation ensur-ing that the weights are all non-negative. The negative weights arecaused by the concavity of the shape when the interpolating ver-tices are not visible by the point to be interpolated. PMVC tacklethis problem by introducing visibility of the interpolating verticesinto the weight computation. Moreover, it is also very promisingto enable local volume preservation to our energy function by seg-menting the mesh into semantic parts and define volume preservingconstraints on those segments.

Acknowledgments

First, a big thank to Alla’s inspiring lectures on geometric model-ing and basic geometry processing techniques in the whole winterterm. Also thank all the classmates for question discussions andpaper presentations. Through the course, we all get a better under-standing on this research area, combined with a wider horizon onthose related mathematical topics. Thank Alla again for trusting

and supporting me on conducting this not very easy course project.The experience on coding Gauss-Newton iteration with backtrack-ing line search is just amazing! Last but not least, thank Enrique forteaching me to use 3ds Max to construct the control meshes.

References

ARMIJO, L. 1966. Minimization of functions having lipschitz con-tinuous first partial derivatives. Pacific Journal of mathematics16, 1, 1–3.

ASCHER, U. M., AND GREIF, C. 2011. A First Course on Numer-ical Methods, vol. 7. Siam.

BELLMAN, R. 1956. Dynamic programming and lagrange multi-pliers. Proceedings of the National Academy of Sciences 42, 10,767–769.

DESBRUN, M., MEYER, M., SCHRODER, P., AND BARR, A. H.1999. Implicit fairing of irregular meshes using diffusion andcurvature flow. In Proceedings of the 26th annual confer-ence on Computer graphics and interactive techniques, ACMPress/Addison-Wesley Publishing Co., 317–324.

FLOATER, M. S. 2003. Mean value coordinates. Computer aidedgeometric design 20, 1, 19–27.

HUANG, J., SHI, X., LIU, X., ZHOU, K., WEI, L.-Y., TENG, S.-H., BAO, H., GUO, B., AND SHUM, H.-Y. 2006. Subspacegradient domain mesh deformation. In ACM Transactions onGraphics (TOG), vol. 25, ACM, 1126–1134.

JACOBSON, A., BARAN, I., POPOVIC, J., AND SORKINE, O.2011. Bounded biharmonic weights for real-time deformation.ACM Trans. Graph. 30, 4, 78.

JU, T., SCHAEFER, S., AND WARREN, J. 2005. Mean value co-ordinates for closed triangular meshes. In ACM Transactions onGraphics (TOG), vol. 24, ACM, 561–566.

KELLEY, C. T. 2003. Solving nonlinear equations with Newton’smethod, vol. 1. Siam.

LIPMAN, Y., KOPF, J., COHEN-OR, D., AND LEVIN, D. 2007.Gpu-assisted positive mean value coordinates for mesh deforma-tions. In Symposium on geometry processing.

SANDER, P. V., GU, X., GORTLER, S. J., HOPPE, H., AND SNY-DER, J. 2000. Silhouette clipping. In Proceedings of the 27thannual conference on Computer graphics and interactive tech-niques, ACM Press/Addison-Wesley Publishing Co., 327–334.

SEDERBERG, T. W., AND PARRY, S. R. 1986. Free-form defor-mation of solid geometric models. ACM SIGGRAPH computergraphics 20, 4, 151–160.

SORKINE, O., AND ALEXA, M. 2007. As-rigid-as-possible sur-face modeling. In Symposium on Geometry processing, vol. 4.

SORKINE, O., COHEN-OR, D., LIPMAN, Y., ALEXA, M.,ROSSL, C., AND SEIDEL, H.-P. 2004. Laplacian surface edit-ing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPHsymposium on Geometry processing, ACM, 175–184.

WEDDERBURN, R. W. 1974. Quasi-likelihood functions, gener-alized linear models, and the gaussnewton method. Biometrika61, 3, 439–447.

WOLFE, P. 1969. Convergence conditions for ascent methods.SIAM review 11, 2, 226–235.

5