10
Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18 Fitting 3D morphable models using implicit representations Curzio Basso * and Alessandro Verri * DISI Universit` a di Genova Via Dodecaneso 35, 16146 Genova, Italy +39-010-3536609, email: [email protected] www: www.disi.unige.it/person/BassoC DISI Universit` a di Genova Via Dodecaneso 35, 16146 Genova, Italy +39-010-3536601, email: [email protected] www: www.disi.unige.it/person/VerriA Abstract We consider the problem of approximating the 3D scan of a real object through an affine combination of examples. Common approaches depend either on the explicit estimation of point-to-point correspondences or on 2-dimensional projections of the target mesh; both present drawbacks. We follow an approach simi- lar to [IF03] by representing the target via an implicit function, whose values at the vertices of the approxi- mation are used to define a robust cost function. The problem is approached in two steps, by approximat- ing first a coarse implicit representation of the whole target, and then finer, local ones; the local approxima- tions are then merged together with a Poisson-based method. We report the results of applying our method on a subset of 3D scans from the Face Recognition Grand Challenge v.1.0. Digital Peer Publishing Licence Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the current version of the Digital Peer Publishing Licence (DPPL). The text of the licence may be accessed and retrieved via Internet at http://www.dipp.nrw.de/. First presented at the Second International Conference on Computer Graphics Theory and Applications 2007, extended and revised for JVRB Keywords: 3D Morphable Models, non-rigid regis- tration, implicit surface representations 1 Introduction We consider the problem of approximating a target 3D surface with an affine combination of (registered) examples, under the assumption that both the exam- ples and the target belong to the same object class. When the assumption holds, the affine space gener- ated by the examples represents a model of the ob- ject class, usually known as 3D Morphable Model in case of faces [BV99]. The model is parameterized by the coefficients of the affine combination. We con- sider the class of human faces; our goal is to find the best possible combination of the available examples that approximates a given target. This is essentially a problem of non-rigid registration, where the avail- able deformations are constrained by the model; it can arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for images by [BV03]. In general, we might approach the problem in differ- ent ways, depending on the representation of the target surface: the target is explicitly represented as a triangular mesh, and the problem is solved in R 3 ; the target is projected to R 2 , as depth map or cylindrical projection; urn:nbn:de:0009-6-12799, ISSN 1860-2037

Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

Fitting 3D morphable models using implicit representations

Curzio Basso∗ and Alessandro Verri†

∗DISIUniversita di Genova

Via Dodecaneso 35, 16146 Genova, Italy+39-010-3536609, email: [email protected]

www: www.disi.unige.it/person/BassoC†DISI

Universita di GenovaVia Dodecaneso 35, 16146 Genova, Italy

+39-010-3536601, email: [email protected]: www.disi.unige.it/person/VerriA

Abstract

We consider the problem of approximating the 3Dscan of a real object through an affine combination ofexamples. Common approaches depend either on theexplicit estimation of point-to-point correspondencesor on 2-dimensional projections of the target mesh;both present drawbacks. We follow an approach simi-lar to [IF03] by representing the target via an implicitfunction, whose values at the vertices of the approxi-mation are used to define a robust cost function. Theproblem is approached in two steps, by approximat-ing first a coarse implicit representation of the wholetarget, and then finer, local ones; the local approxima-tions are then merged together with a Poisson-basedmethod. We report the results of applying our methodon a subset of 3D scans from the Face RecognitionGrand Challenge v.1.0.

Digital Peer Publishing LicenceAny party may pass on this Work by electronicmeans and make it available for download underthe terms and conditions of the current versionof the Digital Peer Publishing Licence (DPPL).The text of the licence may be accessed andretrieved via Internet athttp://www.dipp.nrw.de/.First presented at the Second International Conferenceon Computer Graphics Theory and Applications 2007,extended and revised for JVRB

Keywords: 3D Morphable Models, non-rigid regis-tration, implicit surface representations

1 Introduction

We consider the problem of approximating a target3D surface with an affine combination of (registered)examples, under the assumption that both the exam-ples and the target belong to the same object class.When the assumption holds, the affine space gener-ated by the examples represents a model of the ob-ject class, usually known as 3D Morphable Model incase of faces [BV99]. The model is parameterized bythe coefficients of the affine combination. We con-sider the class of human faces; our goal is to find thebest possible combination of the available examplesthat approximates a given target. This is essentiallya problem of non-rigid registration, where the avail-able deformations are constrained by the model; it canarise not only when building a 3D Morphable Model[ACP03, BV06], but also for performing 3D shapeanalysis as done for images by [BV03].

In general, we might approach the problem in differ-ent ways, depending on the representation of the targetsurface:

• the target is explicitly represented as a triangularmesh, and the problem is solved in R3;

• the target is projected to R2, as depth map orcylindrical projection;

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 2: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

• the target is represented implicitly in R3.

The first strategy is essentially based on the IteratedClosest-Point (ICP) registration algorithm [BM92,TL94]. Although originally used for rigid registra-tion, it can be extended to non-rigid registration and tothe case at hand. This was demonstrated in [ACP03],where the possibility of using examples-based modelswas also mentioned. The drawback of this type of ap-proaches is the need of an explicit estimation of thepoint-to-point correspondence between the model andthe target. Letting aside the problems due to holes inthe target, the search for corresponding points in R3

might be computationally inefficient.The second approach, adopted by [BV99] and

[BV06], avoids the explicit estimation of point-to-point correspondences, by minimizing the differencebetween the cylindrical projections to R2 of the tar-get and the model. As well as avoiding the directsearch for corresponding points, the projection to R2

has the advantage of reducing the problem dimension-ality. However, this comes at the cost of loosing someinformation about the 3D surfaces, since the projec-tion is not a parameterization of the original surface.Moreover, the projection is usually non-linear and oc-clusions frequently occur.

In this paper we investigate the possibility of solv-ing the problem representing the target implicitly inR3. In doing so, we avoid the explicit correspon-dence estimation, while still working in the originaldomain of the data. Although it has never been ap-plied to our specific problem, [IF03] have used an im-plicit representation to reconstruct surfaces from un-calibrated video sequences. In their work, however,it is not the target data that are represented implic-itly, but rather the model, a deformable template thathas to be matched to the data. Also related to ours isthe work of [SSB05], where the problem of registeringthe target with a fixed template (not an example-basedmodel) was solved representing both surfaces implic-itly. The implicit representations where found usingSupport Vector Machines (SVM).

Our work is based on a simpler implicit representa-tion of the target mesh, based on Radial Basis Func-tions (RBF), while the model is still represented ex-plicitly. The implicit representation defines a sort ofpotential field in the space surrounding the target sur-face, and can be used to define a robust cost func-tion. In order to reduce the complexity of computingthe RBF for large meshes, we first minimize the costfunction over a low resolution implicit representation,

and afterward we minimize it over higher resolutionpatches of the target. Finally, the different patchesare blended together with a Poisson-editing approach[YZX+04]. In our work we assume that the target isalready coarsely aligned with the model.

2 Background

In the following two subsection we briefly exposethe basic notions our method builds upon: three-dimensional Morphable Models, and implicit repre-sentations of surfaces.

2.1 3D Morphable Models

A triangular mesh is defined by a graph M = (V , E);V is a set of n vertices in R3 and E is a set of edgesconnecting the vertices. For convenience we write Vas a matrix

V = [v1 . . .vn] =

x1 . . . xn

y1 . . . yn

z1 . . . zn

∈ R3×n. (1)

A set of m registered meshes, denoted by Mi, willshare the same connectivity E but have different ver-tices positions Vi. We can naturally define the sub-space of their affine combinations, with coefficientsa = (a1, . . . , am) ∈ Rm, as

M(a) = (V (a), E), (2)

with

V (a) =m∑

i=1

aiV i and∑

ai = 1. (3)

The affine constraint is needed in order to avoid scal-ing effects.

Note that rewriting eq. (3) in terms of the barycenter

V =1m

m∑i=1

V i, (4)

we can eliminate the constraint on the sum of the co-efficients:

V (a) = V +m∑

i=1

ai(V i − V ). (5)

Three-dimensional Morphable Models (3DMM) use arepresentation for the affine combination based on thePrincipal Component Analysis (PCA) of the subspace

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 3: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

spanned by the V i−V . Without going into the detailsof the PCA [HTF01, pp. 62-63], we only report thatfor 3DMM eq. (5) is written as

V (α) = V +m−1∑i=1

αiU i, (6)

where theU i are the principal components and the co-efficient vector a is replaced byα ∈ Rm−1. See figure1 for a simple two-dimensional example.

Assuming the examples are correctly registered, anyrigid transformation has been factored out from themodel. It can be explicitly included by defining a ro-tation matrix R and a translation vector t which areapplied to V (α):

V (α,ρ) = R(ρ) ·

V +

m−1∑i=1

αiU i

+ t(ρ)⊗ 1T

n ,

(7)where the last term is simply an n-times repetition ofthe column vector t. R and t are parameterized by avector ρ ∈ R6, holding the coefficients of the transfor-mation (three for the rotation and three for the trans-lation). Taking into account the rigid transformation,the model is defined by M(α,ρ) = (V (α,ρ), E).

2.2 Implicit Representations of 3D Surfaces

An implicit representation of a given surface T ⊂ R3

is a function FT : R3 → R such that the surface isone of its level sets. That is, FT (v) = h for eachv ∈ T , and FT (v) 6= h otherwise, where h is a con-stant, typically zero. Clearly, an example of implicitrepresentation is the Euclidean distance

FT (v) = minw∈T‖v −w‖. (8)

In order to define a cost function based on such animplicit representation, we are interested in an analyticform for the implicit function FT ; we build it follow-ing the lines of [TO99, CBC+01]. Given a certain 3Dmesh, they look for a function F (v) which is zero onthe vertices and different from zero on a set of off-surface points. The off-surface points are required toavoid the trivial solution of a function identically zeroover the whole space. They are chosen to lie on thenormal to the surface at the mesh vertices. Let usdenote by wj the vertices and the off-surface points.Then, given a radial basis function φ(x), there exist a

choice of scalar weights dj and of a degree one poly-nomial P (v) such that the function

F (v) =n∑

j=1

djφ(v −wj) + P (v), (9)

satisfies the constraints F (wj) = hj and is alsosmooth, in the sense that minimizes the energy

E =∫

Ω⊂R3

F 2xx +F 2

yy +F 2zz + 2F 2

xy + 2F 2yz + 2F 2

zx,

(10)a generalization to R3 of the thin-plate energy. In fig-ure 1 we show such an example of FT (v), where T isa poly-line in 2D.

In practice, the unknown vectors d and p (coeffi-cients of the polynomial) are the solutions of a linearsystem. Given the matrices

Φ =

φ11 . . . φ1n...

...φn1 . . . φnn

(11)

and

A =

1 x1 y1 z1...

......

...1 xn yn zn

(12)

one has to solve the system[Φ A

AT 0

] [dp

]=[h0

]. (13)

Note that when the input mesh is large, the abovelinear system, if dense, might become intractable. Apossible solution is to induce sparsity by choosing abasis with compact support, but this on turn creates ex-trapolation problems. In order to maintain sparsity andachieve good extrapolation behavior an option is to de-fine multiple levels of resolution, as done by [OBS03].Without resorting to a compact support basis, one canuse Fast Multipole Methods to reduce both the storageand the computational cost, as done in [CBC+01].

3 Low Resolution Approximation

Given a 3DMMM(α,ρ) as defined in section 2.1, anda target mesh T , we formulate the approximation prob-lem as the optimization

α?,ρ? = arg minα,ρ

D(M(α,ρ), T ), (14)

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 4: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

Figure 1: On the left, an example of a 2D morphable modelM(α): two poly-lines, in red and blue, representingthe characters ’S’ and ’3’. The green curves are linear combinations of the two examples, and can be writtenas the average shape plus a deformation along the only principal direction. The amount of the deformation isgiven by the coefficient α, whose values for the three green curves are respectively α = −0.5, 0, 0.5. If weconsider the average shape (α = 0), it generates the implicit representation on the right, where the gray-levelscorrespond to the values of FT (x), as defined by eq. (9) with φ(x) = ‖x‖. The figure in the middle shows thecorresponding cost function D(M(α), T ) as defined by eq. (15) with `(x) = x2.

where D is a suitable function measuring the approxi-mation cost.

Assuming we have an implicit representation FT forthe target mesh, we can define the cost of approximat-ing T by M(α,ρ) as

D(M(α,ρ), T ) =1N

N∑i=1

`(FT (vi)), (15)

where the vi are the vertices of M(α,ρ), and ` mightbe a quadratic loss or an M-estimator. The choice ofan M-estimator might be necessary when parts of themodel M are not present in the target T and influ-ence the correct approximation of the rest. Observethat in the above definition we are essentially treatingthe values FT (vi) as residuals of the approximation.In fact, if FT was the Euclidean distance function and`(x) = x2, then D would correspond to an `2 norm.

3.1 Optimization Scheme

In order to find the minimum of the cost function (15),we use a modified Newton’s method. For the sakeof clarity, in the following discussion we denote byθ the vector of all model coefficients, without distinc-tions betweenα and ρ. Recall that the exact Newton’smethod consists in iteratively updating θ by adding thesolution p of the linear system

∇2D(θ) · p = −∇D(θ), (16)

where ∇2D(θ) denotes the Hessian matrix of D atθ. The exact method converges to a minimum of D

when sufficiently close to it, but it does not in general;nevertheless, there are a number of standard modifica-tions that make it more efficient and robust. We em-ploy a simple scheme, which keeps the Hessian matrixsufficiently positive definite by adding a multiple ofthe identity when required [NW99, Ch.6.3]. Anothermodification to the exact method is that the updatelength is not unitary, but it is determined by a back-tracking procedure which reduces the length if the up-date does not reduce D [NW99, Ch.3.1].

Having the exact form of the implicit representationallows us to compute analytically the gradient and theHessian matrices of the cost function:

∂D

∂θj=

1N

N∑i=1

`′i

(∇Fi ·

∂vi

∂θj

), (17)

and

∂2D

∂θj∂θk=

1N

N∑i=1

`′′i

(∇Fi ·

∂vi

∂θj

)(∇Fi ·

∂vi

∂θk

)

+1N

N∑i=1

`′i

(∂vi

∂θj· ∇2Fi ·

∂vi

∂θk+∇Fi ·

∂2vi

∂θj∂θk

).

(18)

In the above equations we used `′i and `′′i to denote thefirst and second derivatives of the loss function com-puted at FT (vi); ∇Fi and ∇2Fi denotes the gradientvector and the Hessian matrix of FT with respect tothe spatial coordinates, computed at vi.

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 5: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

In practice we choose ` to be the Tukey estimator,that is

`(x) =

c2

6

[1−

(1− (x/c)2

)3] if|x| ≤ cc2/6 otherwise

(19)

with derivatives

`′(x) =

x[1− (x/c)2

]2 if|x| ≤ c0 otherwise

(20)

`′′(x) =

[1− 5(x/c)2

] [1− (x/c)2

]if|x| ≤ c

0 otherwise(21)

Note that the effectiveness of the Tukey estimator inreducing the influence of model vertices not presentin the target depends on the scale of the constant c:the smaller the constant, the smaller the influence ofmissing vertices. On the other hand, small c’s requirea good pre-alignment of the model with the target. Inall our experiments we used c = 5.0.

The gradient and Hessian of FT depends on thechoice of the basis φ. Following [CBC+01], we usethe biharmonic basis function φ(x) = ‖x‖, whichresults in the smoothest solution to the interpolationproblem among all radial basis functions. Such a func-tion with non-compact support is also more suitableto inter- and extrapolation. The derivatives are easilycomputed

∇φ(x) = x/φ, (22)

∇2φ(x) =(I3 −∇φ · ∇φT

)/φ. (23)

Accordingly, we have

∇Fi =n∑

j=1

dj

‖vi −wj‖(vi −wj) + (p1, p2, p3),

(24)

∇2Fi =n∑

j=1

dj

‖vi −wj‖

(I3 −

(vi −wj)(vi −wj)T

‖vi −wj‖2

).

(25)

3.2 Regularizing Prior

In many cases, in particular to avoid overfitting, itmight be convenient to add to the cost function (15) aregularization term which penalizes excessively largemodel coefficients:

DR = D + ηαE(α) + ηρE(ρ), (26)

where the parameters ηα and ηρ weight the effect ofthe regularization on the shape and the rigid parame-ters, respectively.

A standard way to choose the regularization termsconsists in deriving them from prior probabilities. Inthe case of the shape coefficients, the PCA model as-sumes for α a normal distribution with unit variance;for the rigid coefficients we also assume a Gaussiandistribution with zero mean, but with empirically cho-sen variances. In both cases, we can define a regular-ization term proportional to the inverse log-likelihood.For a generic coefficient θ with variance σ2 we have

− log p(θ) =θ2

2σ2+ const. (27)

so that we set (Σρ is the diagonal covariance matrixfor the rigid coefficients)

E(α) =12‖α‖2 and E(ρ) =

12ρT Σ−1

ρ ρ. (28)

4 Segmented Approximation

As noted in section 2.2, a direct solution of the inter-polation problem by solving the linear system is com-putationally and storage intensive. In fact, for highresolution target meshes, the system is too large to beallocated in memory. Rather than following one ofthe methods mentioned in section 2.2, we decided toovercome the problem by adopting a simpler multi-resolution approach (refer to figure 2 for examples ofthe intermediate steps outputs):

1. Coarse Approximation. We first compute acoarse approximation of the target. We select asubset of the target mesh vertices, sampled withprobabilities proportional to the areas of the ad-joining triangles, and use them as constraints ofthe interpolation problem. The resulting implicitsurface is fit with the procedure described in theprevious section.

2. Partitioning. Once we have obtained the coarseapproximation of the target, we can achieve ahigher resolution by repeating the process onsmaller patches of the target and finally mergingthe local approximations. To this aim, we manu-ally defined four regions on the model topology;the regions are visible in figure 2, middle imageof the bottom row. Given the coarse approxima-tion, we compute the bounding box of each re-gion, we expand it in all directions by a fixed

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 6: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

Figure 2: Examples of input data and the results of each approximation step. On the left column are shown thetarget (top image) and the implicit representation of its subsampling (bottom). Fitting the model to the latterresults in the shape in the middle column, top image. The next step consist in separately fitting the segments ofthe model, shown in the bottom-center image, to corresponding parts of the target. The result is shown in theright column: on the top are the different approximations stitched together, on the bottom the result of blendingthem.

length (2 cm in our experiments), and select allthe target’s vertices falling inside the box. Thisresults in four overlapping subsets of the targetvertices, each one associated with a different seg-ment of the model.

3. Finer Approximations. For each subset of thetarget vertices we build an (approximate) implicitrepresentation by sampling its vertices as donein step 1. Although the subsets are again sub-sampled, the ratio between the number of sam-pled vertices and the total number of verticesclearly increases, resulting in a more precise rep-resentation. The implicit surfaces are fit againwith the usual procedure.

4.1 Blending

It is clear that the approximation method explained inthe previous section provides local results, which donot match precisely at their boundaries. In order to

merge them smoothly, we use a variational approachakin to the Poisson-based mesh editing of [YZX+04],to which we refer for more details. The main idea isto keep fixed the positions of the vertices in the interi-ors of the segments and let the other vertices relax tothe positions which minimize an elastic energy. Theprocedure is as follows:

1. identify the boundaries of the patches;

2. for each vertex, compute its minimum distancefrom the boundaries, by fast marching [KS98];

3. define the interior as the set of vertices with dis-tance greater than a certain threshold (we used 0.5cm);

Steps 1 to 3 define the mesh interior Ω and its comple-ment, the boundary region Ω: an area with given widththat surrounds the segments boundaries. We denote by∂Ω the set of vertices in the interior Ω that are con-nected to any vertex in the boundary region Ω. While

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 7: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

the vertices in Ω are fixed, the positions of the verticesin Ω are obtained by solving the Poisson equation:

4. for all the vertices in Ω, find their position solvinga discretized Poisson equation using as boundaryconditions the positions of the vertices in ∂Ω andas guidance field the gradient of the coarse ap-proximation.

In practice, one solves the Poisson equation for thefield of displacements from the coarse approximationto the fine one. This formulation yields a sparse linearsystem of equations of the type ∆di = 0, where ∆ isthe discrete Laplacian operator over the mesh [Tau95],and di is the displacement of the i-th vertex in Ω. Thesystem is solved under the boundary constraints at ∂Ω,where the displacements are known.

The only parameter that has to be set for the blend-ing process is the width of the boundary region Ω. Thesmaller the width, the smaller the blending effect, andmore details will be retained from the local approxi-mations of the segments. On the contrary, by increas-ing the width one extends the effect of blending. Inorder to obtain visually pleasant results, the user willhave to set for a trade-off between blending and de-tails. However, the choice of the width is highly task-dependant; for instance, if the main use of the approx-imations is to perform face recognition, the blendingmight be skipped altogether.

5 Results

The above method has been tested using as model asub-sampled version of the mixed expression-identity3D Morphable Model built with the algorithm de-scribed in [BV06]. The sub-sampling is simple, sincethe reference template was built by subdivision of alow-resolution one. Out of around 40k vertices atfull resolution, we retained approximately 2.5k. Aswell as reducing the number of vertices, we also dis-carded the expression shape components and all thetexture components. The targets were a set of 165range scans, randomly sampled from the range datadistributed for the Face Recognition Grand Challengev.1.0 [PFS+05]. The scans were distributed with a listof landmark positions, which we used to pre-align thefaces to the model. The approximations have been per-formed using 99 model components and 500 sampledvertices for building the implicit representations. Weshould remark that these are preliminary experiments,and a more in-depth study will follow.

Figure 3: Dependency of the approximation on thenumber of principal components used by the modeland the number of vertices sampled from the targetsurface.

We evaluated the goodness of the fit by cylindricallyprojecting the target and the approximation, and com-puting the average absolute error (a radial distance) onthe four segments. The rest of the head model was nottaken into account because of the hair and clothes of-ten present in the scans. Of the 165 results, we rejectedthree of them in which the error was more than threestandard deviations larger than the average. In the re-maining 163 results, the average radial error was 1.09mm on the blended high-resolution result, with an av-erage improvement with respect to the low resolutionapproximation of 0.39 mm. Some of the results areshown in figure 4.

Indeed, a comparison with our previous method,[BV06], confirms the good performance of the newmethod based on the implicit representations, both interms of stability and of approximation accuracy. Ofthe 165 examples used, the algorithm of [BV06] failedin 21 cases, ten times more than the new one, and inthe remaining cases, the average error was of 2.11 mm.This last result in particular is remarkable, since thealgorithm used for comparison is directly minimizingthe radial error, while the new algorithm minimizes itonly indirectly.

We also ran a small experiment to assess the depen-dency of the approximation on the number of princi-pal components used and the number of vertices sam-pled from the target surface. We repeatedly computedthe low resolution approximations for a small subset oftest data (only four examples), varying the two afore-mentioned parameters. The results are in figure 3. The

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 8: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

dependency on the number of principal componentsused in the model behaves as expected: up to a point,the more components are used the better the approx-imation performance. This is not true anymore wheninput is too noisy, as it probably occurs when the num-ber of sampled vertices is only 100. More interest-ingly, the dependency on the number of sampled ver-tices shows that an increase on the sample rate pays offonly up to a point, while the improvement from 500 to1000 is minimal.

6 Conclusion

We have presented a method which makes simple useof the implicit representation of a surface to find itsoptimal approximation in terms of an affine combina-tion of examples. Implicit representations are appeal-ing in general, because they are topology-free and typ-ically quite robust to holes in the data. In our setting,they also offer the advantage of completely avoidingthe problem of estimating correspondences.

As we saw, however, the use of implicit represen-tations poses serious computational problems whendealing with high-resolution meshes. Therefore, wehave proposed to tackle the problem in a multi-resolution fashion, and we have shown how this ap-proach can provide good results without computingfull-resolution representations. We should remarkthat, in assessing the method, we considered its per-formance only in absolute terms, without comparingit with respect to other, already published algorithms.This is certainly a deficiency of our work, and wewill have to correct it in the future. Nevertheless,we believe that the absolute performance achieved byour method on the Face Recognition Grand Challengerange data is a strong indicator of its applicability to areal-world scenario.

The sub-sampling of the original surface is furthermotivated by the consideration that the model has alsoa fixed level of resolution. The resolution of the modeldoes not only dependent on the number of verticesof its mesh, but especially on the number of trainingexamples and of components used. It seems there-fore reasonable to tweak the resolution of the targetso that it matches the one of the model. On the otherhand, sub-sampling poses problems. This is particu-larly clear when considering the approximation resultsobtained on targets containing clothes or hairs. In thebest case these data are irrelevant, in the worst they areharmful, since they will cause distortions in the low-

resolution implicit representation, which might be en-hanced by an unlucky sub-sampling. Preprocessing ofthe targets which removes these data would certainlyimprove the method’s performance and stability.

The future development will focus on two problems:first, the choice of the optimal segments in which themodel is partitioned, and second, the integration of atexture model in the approximation scheme.

7 Acknowledgements

This work has been partially supported by the FIRBproject RBIN04PARL.

CitationCurzio Basso and Alessandro Verri, Fitting 3Dmorphable models using implicit representations,Journal of Virtual Reality and Broadcasting4(2007), no. 18, Month 2007, urn:nbn:de:0009-6-12799, ISSN 1860-2037.

References[ACP03] Brett Allen, Brian Curless, and Zoran Popovic,

The space of human body shapes: reconstruc-tion and parameterization from range scans,ACM Transactions on Graphics (Proceedingsof ACM SIGGRAPH 2003) 22 (2003), no. 3,587–594, ISSN 0730-0301.

[BM92] P.J. Besl and N.D. McKay, A Method for Reg-istration of 3-D Shapes, IEEE Transactions onPattern Analysis and Machine Intelligence 14(1992), no. 2, 239–256, ISSN 0162-8828.

[BV99] Volker Blanz and Thomas Vetter, A Mor-phable Model for the Synthesis of 3D Faces,Proc. ACM SIGGRAPH 99, 1999, ISBN 0-201-48560-5, pp. 187–194.

[BV03] Volker Blanz and Thomas Vetter, Face Recog-nition Based on Fitting a 3D Morphable Model,IEEE Trans. Pattern Anal. Mach. Intell. 25(2003), no. 9, 1063–1074, ISSN 0162-8828.

[BV06] Curzio Basso and Thomas Vetter, Registrationof Expressions Data using a 3D MorphableModel, Journal of Multimedia 1 (2006), no. 4,37–45, ISSN 1796-2048.

[CBC+01] J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J.Mitchell, W. R. Fright, B. C. McCallum, andT. R. Evans, Reconstruction and Representa-tion of 3D Objects with Radial Basis Functions,Proc. of ACM SIGGRAPH 2001, 2001, ISBN1-58113-374-X, pp. 67–76.

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 9: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

Figure 4: Some examples of approximations (second row, the original are on the top row). The average per-vertex radial errors for the three examples are, respectively: 0.80, 0.86 and 0.80 mm. The bottom two rowsshow error maps for the segmented approximations (third row) and the low resolution approximation (last row).Black is zero error, white is 3mm or more. The improvements in the segmented approximations are especiallyevident in the nose region.

urn:nbn:de:0009-6-12799, ISSN 1860-2037

Page 10: Fitting 3D morphable models using implicit representations · arise not only when building a 3D Morphable Model [ACP03, BV06], but also for performing 3D shape analysis as done for

Journal of Virtual Reality and Broadcasting, Volume 4(2007), no. 18

[HTF01] T. Hastie, R. Tibshirani, and J. Friedman, TheElements of Statistical Learning, Springer Se-ries in Statistic, Springer, New York, USA,2001, ISBN 0-387-95284-5.

[IF03] Slobodan Ilic and Pascal Fua, Implicit Meshesfor Modeling and Reconstruction, Proc. Conf.Computer Vision and Pattern Recognition2003. Volume II, June 2003, ISBN 0-7695-1900-8, pp. 483–492.

[KS98] R. Kimmel and J.A. Sethian, Computinggeodesic paths on manifolds, Proc. Natl. Acad.Sci. USA 95 (1998), no. 15, 8431–8435, ISSN0027-8424.

[NW99] Jorge Nocedal and Stephen J. Wright, Numer-ical Optimization, Springer Series in Opera-tions Research, Springer, New York, NY, USA,1999, ISBN 0-387-98793-2.

[OBS03] Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel, A Multi-scale Approach to 3DScattered Data Interpolation with CompactlySupported Basis Functions, SMI ’03: Proceed-ings of the Shape Modeling International 2003(Washington, DC, USA), IEEE Computer So-ciety, 2003, ISBN 0-7695-1909-1, pp. 153–164.

[PFS+05] P. J. Phillips, Patrick J. Flynn, Todd Scruggs,Kevin W. Bowyer, Jin Chang, Kevin Hoffman,Joe Marques, Jaesik Min, and William Worek,Overview of the Face Recognition Grand Chal-lenge, Computer Vision and Pattern Recogni-tion (CVPR 2005) (Los Alamitos, CA, USA),vol. 1, IEEE Computer Society, June 2005,ISSN 1063-6919, pp. 947–954.

[SSB05] Florian Steinke, Bernhard Scholkopf, andVolker Blanz, Support Vector Machines for 3DShape Processing, Computer Graphics Forum24 (2005), no. 3, 285–294, ISSN 0167-7055.

[Tau95] Gabriel Taubin, A signal processing approachto fair surface design, SIGGRAPH ’95: Pro-ceedings of the 22nd annual conference onComputer graphics and interactive techniques(New York, NY, USA), ACM Press, 1995, ISBN0-80791-701-4, pp. 351–358.

[TL94] Greg Turk and Marc Levoy, Zippered polygonmeshes from range images, SIGGRAPH ’94:Proceedings of the 21st annual conference onComputer graphics and interactive techniques(New York, NY, USA), ACM Press, 1994, ISBN0-89791-667-0, pp. 311–318.

[TO99] Greg Turk and James F. O’Brien, Shape Trans-formation Using Variational Implicit Func-tions, Proc. of ACM SIGGRAPH 99, 1999,ISBN 0-201-48560-5, pp. 335–342.

[YZX+04] Y. Yu, K. Zhou, D. Xu, X. Shi, H. Bao,B. Guo, and H.-Y. Shum, Mesh Editing withPoisson-Based Gradient Field Manipulation,ACM Transactions on Graphics 23 (2004),no. 3, 641–648, ISSN 0730-0301.

urn:nbn:de:0009-6-12799, ISSN 1860-2037