Stochastic 3D Geometric Models for Classication, Deformation, and Estimationvisionlab.uncc.edu/.../2004_PhDThesis_complete.pdf · 2009-10-07 · Stochastic 3D Geometric Models for

Stochastic 3D Geometric Modelsfor Classification, Deformation, and Estimation

by

Andrew R. Willis

B. S. Computer Science, Worcester Polytechnic Institute, U.S.A., 1994

B. S. Electrical Engineering, Worcester Polytechnic Institute, U.S.A., 1995

M. S. Engineering Sciences, Brown University, U.S.A., 2002

M. S. Applied Math, Brown University, U.S.A., 2003

A dissertation submitted in partial fulfillment of the requirements for the

Degree of Doctor of Philosophy

in the Division of Engineering at Brown University

PROVIDENCE, RHODE ISLAND, U.S.A.

May 2004

c© Copyright 2004 by Andrew R. Willis

This dissertation by Andrew R. Willis is accepted in its present form by

the Division of Engineering as satisfying the

dissertation requirements for the degree of

Doctor of Philosophy

Date

Professor David B. Cooper, Director

Division of Engineering

Recommended to the Graduate Council

Date

Professor Benjamin B. Kimia, Reader


Date

Professor David B. Mumford, Reader

Division of Applied Math

Date

Professor Gabriel Taubin, Reader


Approved by the Graduate Council

Date

Karen Newman, Ph.D.

Dean of the Graduate School

iii

Vitae

Andrew R. Willis was born in Wilmington, Delaware on June 9, 1972. He lived in Holliston,

Massachusetts, where he graduated from Holliston High School in 1990. Mr. Willis graduated from

Worcester Polytechnic Institute cum laude in 1994 and in 1995, with degrees in Computer Science

and Electrical Engineering respectively. In 2002 and 2003 he finished Masters degrees in Electrical

Sciences and Applied Math respectively at Brown University. In 2002 he was admitted to Scientific

Research Society Sigma Xi.

iv

Acknowledgments

My Ph.D. studies were supported in part by a fellowship offered by Analog Devices for the aca-

demic years 2000 and 2001 and from the NSF KDI grant #BCS-9980091 and ITR grant #0205477,

as research-assistantships for the calendar years of 2002 and 2003. The support of my family,

friends, co-workers, and especially my fiancé Julianna, has made the completion of this work a re-

ality. The pottery assembly system reflects discussions with a diverse group of people from several

disciplines including Applied Math : David Mumford, Archaeology : Martha Sharp Joukowsky,

and Engineering : David Cooper, and Benjamin Kimia. Some programming and significant creative

input was provided by Jasper Speicher with regard to the interface design of the presented sculpting

system. Without the helpful discussions and direction provided by my advisor, David Cooper, this

thesis would not have been possible.

v

Contents

1 Introduction 1

1.1 Main Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 3D Puzzle Solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3.1 Estimating individual sherd Geometries . . . . . . . . . . . . . . . . . . . 3

1.3.2 Estimating sherd Configuration Geometries . . . . . . . . . . . . . . . . . 6

1.3.3 Bayesian geometric model estimation . . . . . . . . . . . . . . . . . . . . 8

1.3.4 Search algorithms for puzzle solving . . . . . . . . . . . . . . . . . . . . . 13

1.4 Deformable 3D Surface Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.4.1 Shape Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.4.2 Stochastic 3D Surface Models : Virtual Sculpting . . . . . . . . . . . . . . 18

2 Pottery Assembly 21

2.1 Estimating Sherd Surface Geometries . . . . . . . . . . . . . . . . . . . . . . . . 21

2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.1.2 Surfaces of Revolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.1.3 Axis / Profile Curve Model . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1.4 Axis / Profile Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.1.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.2 Assembling Configurations of Sherds . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

vi

CONTENTS vii

2.2.2 Parameters to Estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.2.3 Data Generation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.2.4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.2.5 Estimating the Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.2.6 Estimating the Transformation . . . . . . . . . . . . . . . . . . . . . . . . 44

2.2.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.2.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3 The Bayesian Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3.2 Model Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.3.3 Probability and Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.3.4 Physical Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4 The Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.4.2 Treatment of the assembly primitives : Pairwise Hypotheses . . . . . . . . 58

2.4.3 A word on multiscale boundary matching . . . . . . . . . . . . . . . . . . 61

2.4.4 Detecting Duplicate Configurations . . . . . . . . . . . . . . . . . . . . . 63

2.4.5 Controlling the complexity . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2.4.6 Pot Assembly Search Algorithm . . . . . . . . . . . . . . . . . . . . . . . 67

2.4.7 Analysis of the Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

2.4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3 Stochastic 3D Surface Models 75

3.1 Markov Random Fields for 3D Surfaces . . . . . . . . . . . . . . . . . . . . . . . 75

3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.1.2 Surface Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.1.3 Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.1.4 Deformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.1.5 Time Varying Surface Behaviors . . . . . . . . . . . . . . . . . . . . . . . 81

CONTENTS viii

3.1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.2 Example : 3D Sculpting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.2.2 Surface Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.2.3 Sculpting Potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.2.4 The Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3.2.5 The Physical Input Device . . . . . . . . . . . . . . . . . . . . . . . . . . 87

3.2.6 The sculpting process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.2.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Appendix A 93

Bibliography 96

Chapter 1

Introduction

1.1 Main Contributions

Current trends in computational shape theory are expanding from the traditional models of 3D

surface differential geometry originally specified by early geometers such as Gauss and Frenet more

than 100 years ago. New technologies such as 3D scanners and 3D reconstruction from images have

created an abundant source of measured 3D surface data for a wide variety of applications. This

dissertation proposes two new 3D stochastic shape models : (1) a model for the estimation and

classification of 3D axially symmetric shapes, and (2) a model for the deformation of 3D free-form

shapes.

In (1), a complete system is described which automatically estimates complete mathematical

models for 3D ceramic pots given 3D measurements of their fragments commonly called sherds.

The unique approach integrates solutions to 4 different problems : (i) an algorithm for accurately

estimating the surface geometry of an individual sherd from dense-data 3D laser scans, (ii) an al-

gorithm for accurately aligning assemblies of sherds, called configurations, (iii) a Bayesian per-

formance measure for sherd configurations, and (iv) a performance-driven search algorithm. For

individual sherds, estimation of the outer surface geometry is implemented as maximum likelihood

estimation (MLE) of the axially-symmetric surface parameters given the measured sherd data. Sherd

configurations are aligned along break-point segments which lie on the boundary of the sherd’s

outer surface. An algorithm is proposed for accurately aligning configurations of N sherds given a

hypothesized set of correspondences between the sherd break-point segments. This is also imple-

1

1.1. MAIN CONTRIBUTIONS 2

mented as MLE where the estimated parameters are the N −1 sherd alignment transformations, the

matched break-point segment parameters, and the global configuration surface parameters. A com-

mon Bayesian framework provides a performance measure for sherd configurations which is the log

of the probability of the measured sherd data given the computed configuration MLEs, referred to

as the configuration cost. The search mechanism is of the nature of a uniform cost search (in AI)

where cost is the log of probability . However, a considerable number of modifications are made to

eliminate redundant comparisons and reduce both time and space complexity. The assembly pro-

cess starts with a fast clustering scheme which approximates the MLE solution for all sherd pairs,

i.e., configurations of size 2, using a subspace of the geometric parameters, specifically, the sherd

break curves. More accurate MLE values based on all parameters are computed when sherd pairs

are merged with other sherd configurations. Merging takes place in order of constant probability

starting at the most probable configuration.

In (2), a new stochastic surface model for deformable 3D surfaces is proposed and its applica-

tion to unconstrained 3D free-form surface deformation is presented. A 3D surface is a sample of a

Markov Random Field (MRF) defined on the vertices of a 3D mesh where MRF sites coincide with

mesh vertices and the MRF cliques consist of subsets of sites. Each site has 3D coordinates (x, y, z)

as random variables and is a member of one or more clique potentials which are functions defined

on sites within a clique and describe stochastic dependencies among sites. Data, used to deform

the surface can consist of, but is not limited to, an unorganized set of 3D points and is modeled by

a conditional probability distribution given the 3D surface. A deformed surface is a MAP (Maxi-

mum A posteriori Probability) estimate and is the surface for which the joint distribution of the MRF

surface model and the data is maximum. The idea of modeling surface deformation as MAP estima-

tion using mesh vertices as MRF site variables provides a new and powerful tool for generic shape

modeling. Further, the MRF model is extended to accommodate cases where the surface deforms

according to a series of time-dependent data sets thus having the flavor of a tracking problem. The

generality and simplicity of the MRF model provides the ability to incorporate completely arbitrary

local and global deformation properties. A specific instance of the generic MRF shape model is

defined for the purpose of virtual surface sculpting. This is the problem of simple-to-use, intuitively

interactive 3D free-form model building. Development of the virtual sculpting system demonstrates

the simplicity and power of the proposed MRF surface model and includes a new data model, new

Stochastic 3D Geometric Models for Classification, Deformation, and Estimationby Andrew R. Willis Ph.D., Brown University, May 2004

1.2. STRUCTURE OF THE THESIS 3

anisotropic clique potentials, and an example of global surface deformations using cliques involving

sites that are spatially far apart.

1.2 Structure of the thesis

The remainder of Chapter 1 details the problems addressed by the thesis. This is divided into two

sections : (1) stochastic models for axially symmetric geometries, and (2) stochastic models for

free-form surfaces and their deformation. Subsections detail the different topics in which the thesis

makes a contribution. For each topic a general overview of the addressed problem is presented,

followed by a review of related previous work on the problem, and concluded with an overview

of how the thesis makes a contribution to each of these fields. Chapter 2 discusses the system for

pottery assembly and is divided into four parts : (1) the estimation of individual sherd geometries,

(2) the estimation of geometries involving 2 or more sherds, (3) the stochastic geometry model for

sherds and (4) the search problem. Chapter 3 discusses stochastic deformable surface models and

the implemented 3D virtual sculpting system.

1.3 3D Puzzle Solving

1.3.1 Estimating individual sherd Geometries

Problem Summary

Automatic assembly of pots from their sherds is facilitated by solving the difficult problem of ex-

tracting an accurate geometric model of the a priori unknown local pot geometry from measurement

data of a sherd. It is assumed the ceramic pot was made on a potter’s wheel and is consequently an

axially symmetric surface. Such a surface may be represented in terms of its axis of symmetry and a

profile or generating curve with respect to the axis of symmetry (see Fig. 1.1). The task is to obtain a

highly accurate estimate of the unknown axis of symmetry and profile curve given 3D measurements

of the sherd outside surface. Measurement data for a sherd provides information about the portion

of the pot axis/profile curve combination that is subtended by the sherd. Hence, as seen in Fig. 2.10,

measurements from more than one sherd are necessary for estimating the axis/profile-curve for a

pot.


1.3. 3D PUZZLE SOLVING 4

Figure 1.1: An axially-symmetric surface

Previous Work

Over the past decades there has been increasing interest in understanding axially symmetric shapes

from 3D surface data. Early work was done on the subject by Nevatia et. al. in [57, 58] and

continued in their paper [90]. Most published work concentrates on extracting the axis and profile of

pieces using shape from silhouette [59]. However, methods which use one image may only capture

the vessel profile curve and can do this only if the axis of the vessel is orthogonal to the camera

view. Note that the occluding contour of a fragment does not provide the information necessary for

estimation of the axis location. This requires the occluding contour from both sides of the vessel

or 3D data from possibly a 3D scanner or multiple images to estimate. Due to these problems,

the methods proposed for estimation of axially symmetric shapes have not shown to be effective

for small surface patches. But these cases are important in practice and challenging in concept.

Some initial work on the subject was proposed in [65] where the authors apply concepts from

algebraic geometry to develop a linear algorithm for estimating the axis of symmetry. This method

has the benefit of providing a quick and reasonable estimate. Yet, in the interest of preserving linear

computational complexity, the authors do not enforce the Plücker relation which guarantees the

solution is a valid Euclidean line. Additionally, by estimating the axis as a line intersected by locally



estimated surface normals, the method cannot make use of the fact that the measured data lies on a

continuous axially symmetric surface patch. Hence, the estimation accuracy is not sufficient for our

applications. In [55], a method of axis estimation is presented based on the fact that, for a surface of

revolution, maximal spheres tangent to the surface will have centers on the axis of symmetry. This

method differs from the method proposed in this thesis since the authors are estimating osculating

spheres for each data point/normal pair to obtain an estimate of the axis of symmetry. The centers of

these spheres depend upon the principal curvature of the surface parallel which passes through each

of the point/normal pairs (see Fig. 2.1 and associated text of §2.1 for a definition of parallel). The

authors add robustness to their estimator by detecting outliers in a weighted iterative least-squares

framework. Having computed their axis estimate, the authors then use this estimate to compute

the profile curve using a cubic spline model fit to the sherd data. Another technique developed

at the same time seeks the axis that generates the best set of cross-sectional circular curves [61].

Here, the data is binned according to height intervals along the axis where each bin represents a

parallel of the unknown axially-symmetric surface. An energy function is proposed which seeks the

axis which, when the data is binned, provides the best set of circular cross-sections. The proposed

estimation method, developed at the same time as [55, 61], fits algebraic axially symmetric surfaces

of specified degree to all surface data/normal pairs simultaneously using a weighted iterative least-

squares fitting method. In doing so, the model incorporates information from both the meridians

and the parallels of the surface of revolution and does not require use of any local operators such

as differentiation which amplifies the measurement noise. §2.1.3 illustrates the distinction between

the methods which lead to different axis/profile estimates. Researchers at the Technical University

of Vienna have developed a system for sherd classification based on qualitative sherd features, e.g.,

global shape of profiles, with human-driven pre-processing [81, 39, 38].

Contribution

The thesis describes an automatic method for accurately estimating an axis/profile curve pair for

each sherd (even when they are small) based on axially symmetric algebraic, i.e., implicit polyno-

mial, surface models. The method estimates the axis/profile curve for a sherd by finding the axially

symmetric algebraic surface which best fits the measured set of dense 3D points and associated

normals. Note that in contrast to other methods, the proposed method can be directly applied to



3D point data alone and does not require any local surface computations such as curvature and

normal estimation. Axis/profile curve estimates are accompanied by a detailed statistical error anal-

ysis. Estimation and error analysis are illustrated with application to a number of sherds. These

fragments, excavated from Petra, Jordan, are chosen as exemplars of the families of geometrically

diverse sherds commonly found on an archaeological excavation site.

1.3.2 Estimating sherd Configuration Geometries

Problem Summary

Consider the situation where an axially symmetric surface is broken into a set of pieces or sherds.

A subset of the sherds are available and for each of them noisy measurements of its surface and

boundary are obtained. Denote the portions of the sherd boundary where the original surface lo-

cally broke into two pieces as the sherd break-curves. Using the sherd surface and break-curve

measurements and knowledge of which sherds share a common break-curve, the problem is to au-

tomatically estimate the unknown axially-symmetric global surface. The estimation is an alignment

problem where the unknown axially-symmetric surface and break-curves must be estimated while

simultaneously estimating the Euclidean transformation that positions each measured sherd with

respect to the a-priori unknown global surface.

During the assembly process, the system constructs puzzle solutions by combining individual

sherds with larger sherd configurations. Sherds are combined by adding a new sherd to a config-

uration along one of its un-matched break-curves. The resulting arrangement of sherds must be

adjusted, i.e., its parameters must be re-estimated, due to the newly added sherd. Hence, estima-

tion of sherd configurations is crucial to the puzzle assembly process. The proposed estimation

algorithm is invoked each time the system constructs new configurations and allows the system to

convert to the collection of measured sherd data into a single scalar value which determines how

well the data represents an axially-symmetric surface.

Previous Work

The alignment of multiple 3D point sets is a common problem in computer vision which has been

incrementally improving in accuracy and performance for the past decade [12, 15]. Alignment



problems typically involve a set of 3D surface-point measurements taken from different viewpoints

of an object. The data sets measured from different viewpoints are then aligned with one another

to generate an improved model of the measured object. The alignment process starts by identifying

viewpoint pairs which contain measurements from the same portion of the object surface. These

viewpoint pairs are said to be overlapping and the points shared between the two point sets are

control points. The Iterative Closest Point (ICP) algorithm proposed by [12, 15] aligns pairs of

views using a two-step iterative process. In step 1, a correspondence is established between the

control points in each view. In step 2, an alignment error is minimized which reduces the distance

between corresponding points in each view according to some defined metric, e.g., the Euclidean

distance. Since its introduction, numerous variations of the algorithm have been proposed (see

[44, 68] for a survey of these variants for view pair alignment). Other researchers have proposed

model-based methods for point alignment [73]. In this case, a surface or curve model is fit to one

data set and the points of another data set are aligned with the estimated model. Since both methods

depend upon aligning control points which come from corresponding portions of the object surface,

alignment is difficult if there are few control points.

In multiple viewpoint alignment, such as the pot sherd alignment problem, there exist at least

2 viewpoint pairs and we must align the three or more viewpoints simultaneously. These meth-

ods typically align point sets by first performing a pairwise viewpoint alignment and subsequently

minimizing the global alignment error, i.e., the alignment error between the control points of all

corresponding viewpoints. Examples of such work include [26, 15, 22, 11].

Contribution

In the context of automatic axially symmetric surface estimation, each sherd data set is a partial

viewpoint of the unknown axially symmetric surface. Sherd pairs are related by their corresponding

break-curve segments, i.e, the curve segments along which the two sherds were separated, and

their shared axis/profile curve. The thesis treats the difficult problem of aligning axially symmetric

surfaces where the surface overlap is limited to only the break-curve segments, and incorporates

the constraint of axial symmetry into the global fitting error. The alignment problem is treated

as a two-fold problem of (1) estimating the unknown surface geometric parameters consisting of

an axially-symmetric surface and a set of break-curve segments and (2) estimating the alignment



parameters of the pieces which consists of N − 1 Euclidean 3D transformations for a group of N

pieces.

A stochastic approach is taken which computes the Maximum Likelihood Estimate (MLE) of all

the unknown parameters given the measured data where the unknown parameter vector, θ, consists

of : (1) the parameters of the axially-symmetric surface, which are the coefficients of an axially-

symmetric implicit polynomial surface; (2) the true break-curve, modeled as a sequence of 3D

points; and (3) the Euclidean transformation parameters for each of the pieces. The formulation of

geometric alignment of point and surfaces as MLE is new and provides robust, fast, and accurate

estimates of the configuration geometry. The formulation uses two different data types in the align-

ment process, i.e., break-curve points and axis/profile-curve pairs, which is also new. The alignment

algorithm handles chipped, eroded free-form fragments and noisy data. Experimental results are

presented which solves an application of interest, specifically the reconstruction of archaeological

pots from subsets of their surface fragments.

1.3.3 Bayesian geometric model estimation

Problem

Puzzle assembly solutions are constructed by combining puzzle pieces into larger configurations.

Given our assumption that the measured sherds come from an axially symmetric object, a set of

parameters θ1,θ2, ...θN are extracted which model the measured datasets D1,D2, ...,DN from

each of the N sherds. The parameters provide compact representations each sherd’s data as an

axis/profile-curve and an associated set of boundary curves. Typical puzzle assembly solutions seek

to define a similarity or affinity metric, between puzzle pieces. Metrics are formed by defining a

function, f(θ1,θ2), which operates on pairs of of points in parameter space Ωθ and satisfies a set

of defined constraints [67]; one of which is f(θ1,θ2) ∈ R > 0. For puzzle solutions in 2D, affinity

measures reflect the similarity between matched 2D boundaries of objects [69, 49, 30]. In the case

of assembling axially symmetric objects, this function must take into consideration the matched

break-curves between sherd pairs and the global axis/profile-curve of the unknown pot. Hence, the

problem is to define a metric which acts locally to estimate vessel break-curves from sherd boundary

curve data and globally to estimate the vessel axis/profile curve from sherd surface data.



(a) (b)

Figure 1.2: An example match between two pieces. The illustrated configuration pair represents oneof many possible matches between the sherd pair (A,B). Each sherd match assumes the following: “Sherd A (top –in pink) and sherd B (bottom –in brown) broke apart along the boundary regionsJ and K”. The boundary regions J and K from sherds A and B respectively are indicated by aseries of spheres and cylinders representing the sherds break segment points and their correspondingsurface normals respectively. Fig. (a) shows sherds A and B in close proximity but unaligned.Fig. (b) illustrates shows the best alignment between sherds A and B assuming that they brokeapart along the shown boundary regions. Solving the problem from §1.3.2 determines the optimalalignment of the sherd pair given the stated assumption. The system constructs new configurationsusing the proposed solution to the search problem (§1.3.4). The proposed solutions to all statedproblems relies on an underlying Bayesian framework from §1.3.3 which allows the system toconstruct puzzle solutions by adding pieces to configurations. Fig. 1.3 shows one such configuration.



In solving a puzzle problem, the system combines sherds with previously constructed sherd con-

figurations. Each time a sherd is added to a configuration, the similarity function must be evaluated

and the result stored for future use. Hence, one also seeks a metric which minimizes the compu-

tational cost of evaluating the similarity function as well as the storage space required to hold the

computed solution. This is referred to as the computational complexity in time and space of the

similarity metric.

Hence, for puzzle assembly, the problem is to define a function on the chosen parameters which

satisfies the following set of desirable properties : (1) the function must define a valid metric on the

space of parameters [67], e.g,. the triangle inequality must be satisfied; (2) the function must provide

significant delineation between good matches and bad matches, e.g., f(θ1,θ2) f(θ1,θ3) if the

pair (θ1,θ2) is a “bad” piece match and (θ1,θ3) is a “good” piece match; (3) the function must

incorporate local break-curve matching and global surface matching information simultaneously;

(4) the function must be applicable to configurations which may have an arbitrary number of pieces;

and (4) the metric should allow for incorporation of additional information by either (a) extending

the set of parameters, e.g., color attributes θ′

= (θ, Y, Cr, Cb)t where we have added the 3D color

parameters from the color space (Y,Cr,Cb)t or (b) giving prior information, i.e., all the fragments

come from one of ten different types of axially symmetric shapes.

Previous Work

There are many examples of affinity measures for 2D fragments. One example is [49], where

fragment boundaries are extracted from images of the 2D pieces. Here, the metric is based on the

extracted 2D boundary shape and intensity gradient of the image in the vicinity of the boundary

points. Another notable example is [69], where the authors define a metric based on the distance

between matched boundary points alone. Work on defining similarity metrics for puzzle assembly

in 3D are restricted to defining similarities between 3D curves [41, 82]. In this case the metric is

an energy expressed in terms of difference between the curvature and torsion matched 3D curves.

There are currently no existing similarity metrics for axially symmetric puzzle assembly.



(a)

(b) (c)

Figure 1.3: A correct seven piece configuration. Fig. (b) shows a view orthogonal to that shownin (a), i.e., the camera is looking at the side of the vessel. Local costs correspond to error in thematched break-curves of the configuration. Figs. (a) and (b) show these matched curves as renderedspheres on the sherd boundary and the surface normal at each point is rendered as a cylinder passingthrough the sphere center. There is also a single global cost, i.e., all 7 sherds lie on a common axiallysymmetric surface. Fig. (b) shows the estimated axes for each of the 7 sherds as red cylinders.These are noisy estimates of the true vessel axis, l, shown as a black line. In Fig. (c), a cylindricalcoordinate system with axis z is defined and the measured sherd data points from (b) are projectedinto a (r, z) plane where they are shown as dark green points. In Fig. (c), the profile curve, α(r, z) =0, is plotted in light green and interpolates the projected (r, z) data points. As shown in §2.1 theaxis/profile-curve model is equivalent to fitting a 3D axially symmetric surface to the data shown in(b).



Contribution

A probability metric is defined using an assumed Gaussian probability distribution on the sherd

surface and break curve data. In this framework, each configuration of N sherds has parameters

θ = (β, l,α, T) where β denotes the locally matched break-curves, l,α denotes a single vessel

axis/profile-curve pair, and T denotes the parameters of the N − 1 transformations. The similarity

function is the negative log of the probability of the configuration’s sherd data given the MLE of

the parameters θ. For a configuration containing a set of sherds S , the similarity function is then

f(θ) = − ln(p(∪i∈SDi|θ)

). Evaluation of the similarity function is a two step process, the first

step is to compute the MLE θ, and the second step is to evaluate f(θ) whose value is referred to as

the cost of the configuration.

The MLE of the geometric parameters for each configuration integrates the local cost of es-

timating the vessel break-curves and global cost of estimating the vessel axis/profile-curve into a

single framework. Vessel break curve costs are considered local since only 2 sherds may share a

common break-segment (see Figs. 1.2(a) and 1.3(a)). The vessel axis/profile-curve costs are consid-

ered global since many sherds may describe the same portion of the pot profile (see Fig. 1.3(b,c)).

Since the sherds are measured in different coordinate systems and our chosen parameterization is

not Euclidean invariant, the parameters of the 3D Euclidean transformation TB which aligns sherd

B to sherd A must be known to evaluate the hypothesis.

We now see that the problem of puzzle assembly has been posed as a recognition problem. In

this framework, the system seeks to find the value of the parameters which best explains the sherd

data. The value of the MLE reflects the best axially symmetric model possible given the sherd data

and the assumed correspondence between the break-curve segments. The probabilistic approach to

puzzle assembly implemented as MLE of the unknown vessel geometry is a heretofore unexplored

viewpoint. This viewpoint has the benefit of being able to improve the parameter estimates of all the

geometric parameters as new data is supplied. Previous proposed assembly schemes work strictly

from the estimated piece parameters which makes global re-estimation of the solution difficult. The

probabilistic framework for puzzle assembly is new and its demonstrated utility for addressing this

computationally difficult task marks an important theoretical contribution to the computer vision

community and an important practical contribution to the archaeological community. In §1.3.4, a



Figure 1.4: A set of sherd hypothesis for the sherds A and B where the boundary correspondencesJ and K are allowed to vary.

search method is chosen which complements the chosen metric.

1.3.4 Search algorithms for puzzle solving

Problem

As detailed in §1.3.3, configurations of sherds are generated by making the following assumption :

“sherd A and sherd B broke apart along break curve segments J and K.” Since A, B, J, and K can

vary, a group of sherds can generate a large number of configurations. Fig. 1.4 shows configurations

generated from a subset of the possible hypotheses between a sherd pair. The goal of the search

problem is to construct the best configuration of sherds with as little computational cost in space and

time as possible. Here best denotes the configuration of sherds which has least cost or, equivalently,

highest joint probability. For even a modest number of sherds, the size of the solution space, i.e., all

possible configurations, is huge. For illustration, consider a small puzzle having 10 sherds where

each sherd has approximately 10 break segments. The number of possible triplets for this modest

puzzle is roughly 3E6 and the number of quadruplets exceeds 2E9. Hence, the total number of

possible alignments between sherds is huge for even a modest number of sherds.



Previous Work

Early research on the more general problem of puzzle solving in general subject includes works

such as [45, 89, 27] and more recently [30]. Here, puzzle pieces are commercially produced jigsaw

puzzle pieces which simplifies the problem since jigsaw pieces have similar sizes, easily identifiable

areas which need to be matched, and a mostly regular outline, i.e., the piece contour is differentiable

with a few exceptions at corner pieces which require special treatment. Solving more difficult puz-

zles based on real-world data remains an active area of research today with recent results published

in [69, 30, 41], which attempt to improve the algorithmic speed and remove these restrictions.

Yet, examples of automatic puzzle solving in 3D are limited to [62, 41, 82]. In [41, 82], the au-

thors deal with fragments of arbitrary shape and propose to assemble them elegantly by matching

the fragments break curves, i.e., the curves on the pot surface along which the sherds break apart

(Fig. 2.10). Both methods match curves based on their curvature and torsion signatures which, un-

fortunately, is prone to instability when the data is noisy. [82] proposes a greedy search which, in

the context of large puzzle problems, is likely to fail due to the number of false positives which are

known to commonly occur in solving real life puzzles [69, 49]. [41] proposes a best-first search

using observed triplet matching errors. This search method is known to have exponentially increas-

ing complexity in both time and space and no clear criterion for terminating the assembly is given.

Hence this method will work well when there is little chance of classifying an incorrect match as a

correct curve match, i.e., false positives are rare. However, in practice, false positives are quite com-

mon, especially with break curves that have little curvature and torsion which is very common on

real-world fragments and especially on archaeological fragments (see Fig. (2.17) for 2 examples).

In [62], the authors match broken 3D shapes by matching of their fractured surfaces, i.e., the surface

through the pot wall at which the pot breaks, using simulated annealing. Their approach requires

both of the fractured surfaces to be parameterizable with respect to a common plane, i.e., for align-

ing 2 fractured surfaces about the xy-plane the two surfaces may be represented as f1(x, y) = z1

and f2(x, y) = z2. For all three methods, no results are provided for 3D fragments with more than

2 pieces. Specifically, no results for assembly of more than two 3D sherds are given in [41], a single

pairwise match is shown in [82], and several pairwise matches are shown in [62].


1.4. DEFORMABLE 3D SURFACE MODELS 15

Contribution

The thesis proposes a modified uniform cost search where the metric or cost is the negative log-

likelihood of the sherd(s) data given the computed MLE of the geometric parameters. This corre-

sponds to forming new configurations by appending the most probable pairwise configuration to

the most probable existing configuration. Note that the most probable existing configuration at any

time in the search is the most likely, i.e., lowest cost, collection of aligned sherds which has not

been previously considered.

There are five contributions proposed by the thesis which expedite the search of the solution

space by (1) quickly computing candidate configuration pairs, (2) limiting the breadth of the search

to make the search computationally tractable, (3) quickly computing new configurations by using

solutions to previously computed pairwise configurations, (4) incorporating a mechanism for de-

tecting the creation of duplicate configurations, and (5) incorporating a mechanism which avoids

considering configurations which, when merged, produce solutions of high cost.

1.4 Deformable 3D Surface Models

1.4.1 Shape Representation

Problem

A mathematical formalism of shape is fundamental to a wide variety of important problems. In

fields such as computer-aided design, shape models are used to design and manufacture the tools

and machines that power our civilization. Other fields such as medicine and biology use shape for a

wide variety of purposes such as diagnosis, analysis and surgery planning. In computer vision and

machine intelligence, shape representations provide a computational way to recognize objects and

patterns in images, video, or any other device which obtains measurements from real world scenes.

It is common to find situations where the evolution of the object shape over time is important, e.g.,

tracking the growth of an object over time or the deformation of an object under external forces. In

situations such as this, the shape model must be dynamic. Deformable models seek to explain how

a specific object changes or evolves it shape.



Previous Work

Deformable curve and surface models are commonly used in computer vision for a variety of tradi-

tional problems such as shape estimation [53, 78], recognition [18], tracking [77, 8, 52], and have re-

cently become popular for segmentation, particularly in the field of medical imaging [53, 78, 31, 51].

There are three distinct approaches to deformable surface representation: 1) parameterized models;

2) implicit models; and 3) discrete models. Parameterized models are the most popular and may be

divided between physics-based Partial Differential Equation (PDE) models such as 3D mass-spring

systems [79] and variational models such as 3D snakes [84, 31, 17]. [48] and [10], are representative

examples of implicit deformation models which are computationally efficient but these deformation

models have been found more difficult to control than parameterized and discrete models. The lit-

erature on discrete models represents the family of deformable models which are most similar to

the proposed shape model [78, 54, 52, 29, 36]. Discrete models describe the geometry of an object

using a set of 3D point samples taken from the object surface and point-to-point connections which

provide a polyhedral interpolations of the surface between points, e.g., a surface triangulation. Each

point and its associated connections provide a local representation of the surface shape and the total-

ity of these local relationships provides a global shape description. Deformation for discrete models

is implemented by defining deterministic functions which operate on the 3D point samples causing

them to move and consequently deform the surface.

There exists a vast body of MRF models defined for images (see [51, 46] for an overview).

Some have also defined MRF deformable models on range maps with a regular grid such as [71].

However, few examples exist of 3D deformable MRF models. An early example of a 3D MRF shape

model is provided in [85]. Here, the authors define an MRF surface model based on Terzopoulos’s

deformable superquadric models [76] and concentrate on transitioning between local and global

models using a wavelet basis as a vehicle for implementing changes in scale. [47] represents a recent

example of a 3D MRF shape model. In [47], MRFs are defined on a symmetry-based representation

of shape referred to as M-reps (see [64] for details on the shape model). Here, the M-rep surface

representation is quite different from the proposed surface-based MRF and their emphasis is placed

on understanding shape deformation at different scales. Recent work by Cipolla et. al. has also

made use of MRF models for the purpose of 3D surface reconstruction from images [16]. Hence,



the popularity of stochastic deformable models seems to be increasing rapidly.

Contribution

A generic stochastic shape and deformation model is specified which can serve as a common frame-

work for defining the discrete models described in [78, 54] as well as some newly introduced statisti-

cal shape surface models such as [16]. As in other discrete models, the model is defined on an initial

3D mesh which consists of vertices, points which lie on the original shape, and edges, connections

between vertices. An MRF site is assigned to each vertex and a set of cliques, which consist of

subsets of sites, are defined. Clique members and the associated clique potentials which operate

on cliques must be either learned or specified by the system designer. They represent application-

specific information regarding the desired deformation behavior. These cliques and clique energies

can be anisotropic, i.e., can be different in different directions, and can be non-homogeneous, i.e.,

can vary over the surface.

The complete deformation model is expressed in two parts: 1) a model for the probability

density of the surface which is based on cliques involving only MRF sites, i.e., surface vertices; and

2) a conditional probability density for the newly observed data given the surface. The probability

density for the surface specifies a functional relationship among MRF sites, or equivalently, surface

points. The conditional density for the data specifies a functional relationship between MRF sites

and the measurements. Deformed surfaces are sample functions from the joint probability density

and correspond to Maximum A-posteriori Probability (MAP) estimates of site random variables

given the joint probability of the MRF sites and the data. The notion of surface deformation as MAP

estimation using the vertices as site variables is new and its utility and power for specifying surface

deformation models is demonstrated. Simple clique potentials coupled with a new data model are

combined to generate a simple deformation model useful for the interpolation of data. While it has

been known in the literature that deformation can be expressed as MAP estimation (in fact [85, 36,

51] explicitly mention this viewpoint ), there are no instances of stochastic MRF models defined on a

generic 3D surface mesh in which mesh vertices are MRF sites. Other important distinctions which

identify the proposed MRF as unique are that the mesh vertices may be distributed arbitrarily, i.e.,

non-uniformly, on the mesh surface and may have arbitrary connectivity. In addition, scale changes

in the mesh are handled via standard contemporary graphics remeshing techniques (see §3.2 for



remeshing details).

The thesis extends the stochastic deformation model to deal with sequences of data sets provided

at different time instances. In this case, the model becomes a time-varying stochastic process where

the system state can in theory consist of the surface vertices at a succession of two or more instants

(i.e., cycles). It is this fully 3D stochastic model for the representation and modification of surfaces,

and the ability to incorporate unlimited local and global deformation properties (either physically

realizable or physically unrealizable) that is new, tremendously powerful and computationally fast

(real time on a PC).

1.4.2 Stochastic 3D Surface Models : Virtual Sculpting

Problem

One application of the shape model from §1.4.1 is the interactive specification of free form surfaces

to a computer, which is referred to as virtual 3D surface sculpting. This is a twofold problem of (1)

providing a computational model to represent the surface and how it deforms and (2) developing a

intuitive interaction by which the user can specify the desired object or deformation.

Previous Work

Development of an intuitive virtual sculpting paradigm has been pursued by many researchers over

the past decade. Early work on deformable models with user interaction was done by Terzopoulos

[79]. Since then, numerous examples of systems for the purpose of virtual sculping have emerged.

In [80], Terzopoulos and Qin adopted their deformable NURB surface models of [79] to the specific

task of virtual sculpting. Several years later, Qin et. al. developed a system for virtual sculpting

using a dynamic spring-mass model where mesh vertices are masses and mesh edges are springs

which connect 2 masses. Hence, a 3D mesh defines a network of springs and masses and defor-

mation results from introducing a deformation in the spring mass network or by modifying the rest

lengths of the springs. Further work regarding physically-based models such as that of Qin is being

done for volumetric or voxel representations where the object is modeled as a collection of cubical

volume elements [66, 25]. In [91], objects are represented as a dense point cloud of surface mea-

surements. Each point and its associated surface normal defines a small patch of the surface and



(a) (b)

Figure 1.5: A sculpture created using the proposed MRF-based virtual sculpting system. The initialcapsule shape shown in (a) was deformed by the artist to generate a model of a human head. Thehead on the right has been flat shaded to enhance the faceting effect of the triangulation and providea more accurate representation of the underlying geometric mesh structure.



local re-parameterizations of the surface allow for coherent deformations of adjacent points.

In [32, 5, 21, 50] sculpting is done using a haptic device, i.e., the device seeks to simulate

how the tool would actually feel if in contact with the virtual surface. This is accomplished by

tracking the user’s position and orientation using an articulated mechanical arm. Motors integrated

into the articulated joints are controlled by the computer generating force-feedback resistance to

the artist’s tool. Interaction of this sort makes a deformation analogy based on surface contact,

i.e., the user makes contact with the surface using the tool and introduces a force which serves to

deform the surface. Specific instances of the many implemented sculpting systems include all of

the mainstream surface models mentioned in 1.4.1.

Contribution

This thesis details how the MRF surface model from §1.4.1 may be implemented for generating

3D free-form shapes interactively, i.e., virtual sculpting. In contrast to the large number of sys-

tems based on haptic devices, the proposed sculpting systems allows the user to sketch 3D curves

as a series of points in arbitrary relation to the surface, and the surface deforms to interpolate the

sketched 3D curves using a configurable deformation model. In addition to the interactive differ-

ences, the proposed sculpting system introduces a new set of specific clique potential functions for

virtual sculpting. The potential functions proposed include functions which serve to perform local

deformations similar to that in previous physically-based models and a new class of potential func-

tions which serve to enforce global structure such as symmetry about a pre-defined plane. Arbitrary

clique configurations and clique potential functions can be designed and used to extend the base

system. The sculpting system developed uses a custom-built 3D point-generation pen with an inte-

grated 3D tracker and an immersive 3D virtual environment to allow the user to modify a free-form

3D surface given an initial surface mesh. The system performs in real-time and is being used to

create original works of art by professors and students from the Brown University School of Art. In

addition, the system has been used for sculptural restoration of damaged archaeological artifacts.


Chapter 2

Pottery Assembly

This chapter describes the system developed for the purpose of automatic estimation of axially

symmetric geometries from their fragments. The chapter is divided into four sections each of which

describe distinct areas to which the thesis contributes. The first section describes a method for

estimating the axially symmetric surface model for each of the measured sherds. Section 2 details

how configurations of N sherds are aligned given an assumed correspondence between portions of

their boundary. Section 3 discusses the stochastic model for sherd assemblies and how this model

contributes to the assembly process. Section 4 details how the system assembles individual sherds

into axially-symmetric vessels.

2.1 Estimating Sherd Surface Geometries

2.1.1 Introduction

This section discusses a highly accurate solution to the difficult problem of extracting a geometric

model of the unknown pot structure in the region associated with 3D measurements obtained from

a sherd. The model extracted may be used in a variety of applications. Some examples are: shape-

based searching of 3D sherd databases; sherd classification; and pot reconstruction.

21

2.1. ESTIMATING SHERD SURFACE GEOMETRIES 22

Figure 2.1: Geometry of a surface of revolution

2.1.2 Surfaces of Revolution

A surface of revolution S ∈ R3 is obtained by revolving a planar curve α ∈ R

2 about a line l ∈ R3.

α is called the profile (or generating) curve and l the axis of S. When the z-axis is taken as the axis

of revolution with profile curve α(z), the surface S may be represented parametrically as (2.1).1

S ( θ, z) = (α(z) cos θ, α(z) sin θ, z) (2.1)

With this parameterization, the curves z = constant are parallels of S and the curves θ = constant

are meridians of S. The profile curve characterizes how the radius and height of the surface change

for a fixed meridian (see Fig. (2.1)). In equation (2.1), the radius function, r = α(z), is a single-

valued function of z. For archaeological sherds, this presents a problem since profile curves which

are multi-valued with respect to the z-axis commonly occur. Examples include sherds which come

from pot bases and rims. Figs. 2.5 and 2.7 represent typical examples of sherds which have multi-

valued profile curves. For this reason, the profile curve model from (2.1) has been generalized to

include these situations by using a planar implicit polynomial, i.e., the zero set of an algebraic planar

curve : α(r, z) = 0. The problem of interest is: given an unorganized set of 3D measured points of

a small patch of a larger axially symmetric surface, estimate a surface geometry model for the patch.

This geometry is completely specified by an axis in 3D and a 2D profile curve with respect to that

axis. Consequently, estimation of axially symmetric surface models is the estimation of axis/profile

curve pairs (l, α(r, z)). The algorithm seeks the axis/profile-curve pair which best fits the measured

1The adopted notation and terminology for surfaces is that used in classic texts such as [70, 43].



sherd data.

2.1.3 Axis / Profile Curve Model

The pose of a rotationally symmetric object is defined by the position and direction of its axis of

symmetry. The axis of symmetry is parameterized using a standard parametric equation of a 3D

line:

x = mxz + bx

y = myz + by

(2.2)

The equations from (2.2) contain four unknown parameters which describe the 3D axis of symmetry,

l = (mx,my, bx, by). Two of these parameters, mx and my , describe the slope of the line when it

is projected into the xz-plane and the yz-plane respectively. The remaining two parameters, bx and

by, specify the x and y coordinates where the axis line intercepts the xy-plane at z=0.

The profile curve is represented using an implicit polynomial curve model, α(r, z) = 0. Implicit

polynomial curves provide a compact and low computational-cost method of representing shape

[73, 13]. In addition, closed-form linear least-squares fitting methods have been developed which

provide stable and robust curve and surface fits in the presence of noise [72, 13]. The specification

of a 2D implicit polynomial curve of degree d has [(d + 1)(d + 2)/2] unknown coefficients and is

the set of points satisfying (2.3).

αd(r, z) =∑

0≤j+k≤d; j,k≥0

ajkrjzk = 0 . (2.3)

Here, d is a parameter which is related to the geometric complexity of the pottery sherd to be

estimated. Typically one assigns the smallest value to d which is large enough to represent all

objects of interest. In this way, objects which may have little geometric complexity are described

as degenerate cases of the more complex model. For the artifacts in this paper, all experiments are

performed with d = 6.

Axis estimation is posed as a problem where we seek to find the 3D axially symmetric algebraic

surface which best approximates the measured sherd data. However, fitting a 3D axially symmetric



Figure 2.2: Several (x′

, y′

, z′

) points (in blue) are projected into the (r, z) plane (in red) defined bythe estimated axis l and a vector orthogonal to the axis. The highlighted portion is further discussedin Figure (2.3).



surface directly to the data is difficult since the coordinate system of the basis monomials is a priori

unknown. Instead, an equivalent method is proposed which makes use of the axis of symmetry

to simplify the fitting problem. To this end, consider a plane which contains the unknown surface

axis, l, in a general 3D position. A local cylindrical coordinate system is defined in this plane with

origin o, a point on the axis, and two vectors in the plane, z and r, where z is a unit vector in the

direction of l, and r is a unit vector orthogonal to z which passes through the point pi. All of the

3D data points and normals are transformed to the local cylindrical coordinate system using the

transformation equations from (2.4). The exact choice of the origin is arbitrary. The system defines

the origin to be the point on the axis closest to the mean of the measured (x′

, y′

, z′

) surface data.

Then the ith measured data point, pi, in 3D becomes the 2D data point (ri, zi) by rotating pi about

the axis l and into the plane via (2.4). The measured surface normal ni associated with the point pi

will not lie in the rz-plane. Its projection into the plane is denoted npi and is computed via (2.5). A

graphical example of the projection is shown in Fig. (2.2).

ri =√

(pi − o)t(I − zzt)(pi − o)

zi = (pi − o)t · z(2.4)

npi = (ni · r,ni · z) (2.5)

This projection (2.4) takes each measured data point, pi, from x′

y′

z′

-space to rz-space and pre-

serves the distance between the axis and the data point. The projection (2.5) takes only those

components of the measured surface normal, ni, in the directions of r and z. Here, r is the vec-

tor through pi and the axis which is perpendicular to the axis, i.e., r = (pi−o)−(pi−o)tzz

‖r‖ . Hence,

for any 3D normal, ni, the corresponding projected normal, npi , may not be of unit length, i.e.,

‖npi ‖ ≤ 1. The system then solves for the unknown profile-curve coefficients, i.e., the ajk from

(2.3), using the objective function (2.6) defined on the projected data. (2.6) is a least-squares fit of

an algebraic implicit polynomial to the projected 2D data and is a modified version of the energy

function in [72], which is referred to as the Gradient-1 fitting algorithm.

egrad1 =

I∑

i=1

(α2(ri, zi) +1

λ‖np

i −∇α(ri, zi)‖2) , (2.6)



The Gradient-1 2D curve and 3D surface fitting algorithm appends a penalty term to the usual

least-squares objective function (algebraic distance in this case) which makes use of the gradient of

the polynomial in order to get a more stable estimate of the curve. This fitting method makes use

of all the available information: the hypothesized axis, the measured spatial data and the observed

normals. This information is used to compute the global surface model which best fits the data and is

symmetric about the hypothesized axis. Note that ∇α(ri, zi) =

[∂α∂r

∂α∂z

]t

denotes the gradient

and α2(ri, zi) is the surface point fitting error in (2.6) and the latter term fits the polynomial normals

to the measured surface normals. The value chosen for λ will depend upon the noise present in the

measured surface normals. For the experimental results in §2.1.5, λ = 0.01. This corresponds to

weighting errors due to the data normals less than errors due to data points. In §2.2.3, a noise model

for the measured data is assumed which places more emphasis on normal information.

An alternative objective function may make use of the full 3D measured normals ni which

changes the second error term of (2.6) from ‖npi −∇α(ri, zi)‖

2 to

∥∥∥∥∥∥∥ni −

∇α(ri, zi)

0

∥∥∥∥∥∥∥

2

, where

a zero has been appended to the profile curve gradient vector. Denote this alternative version of

the fitting objective function as equation (2.6)’. For this version, components of the vector n i

perpendicular to the (r, z) plane correspond to observed asymmetries in the measured normal data

given the assumed axis. The proposed projection of the normal ni → npi disposes of these observed

errors and hence causes the curve fit to respect the measured spatial data more when the normals

are inaccurate. Note that (2.6)’ is an error function for 2D curve fitting that gives exactly the same

result as does the error function in [72] for 3D surface fitting.

The scalar residual, egrad1, is a measure of asymmetry resulting from the axis/profile-curve

fit. Fig. 2.3 illustrates how each measured surface point and normal impacts the observed fit error

egrad1. Each of the measured data points will not lie exactly on the profile curve model due to

measurement noise and possible asymmetries in the sherd fragment. Fig. 2.3 decomposes this error

between the profile curve and the measured data into its r and z components : (∆ri,∆zi). Errors

in the direction of r, indicated as ∆ri, are associated with asymmetries due to local fluctuations of

a surface parallel which passes through the measured surface point. Errors in the direction of z,

indicated as ∆zi, are associated with asymmetries due to local fluctuation in the surface meridian

which passes through the measured surface point. Fig. 2.3 illustrates these error terms graphically



Figure 2.3: Decomposition of fit error: An example profile curve for a hypothetical pot is shownas α(r, z) = 0 (in red). Two points are illustrated: (1) a measured sherd sample (ri, zi), shown asan outlined circle; and (2) a point from the zero set (i.e, a point from the curve α(r, z) = 0 whichis closest to the sherd sample, shown as a solid circle (in black). The algebraic distance α(ri, zi)

2

approximates the Euclidean distance between the two points (see [74] for details). Two vectorsemanating from the measured surface point are labeled n

pi , representing the measured surface nor-

mal, and 5α(ri, zi), representing the normal of the implicit polynomial profile curve at the point(ri, zi). The difference between the vectors is shown as n

pi − 5α(ri, zi). The size of this error

term, ‖np − ∇α(r, z)‖, approximates the difference between the axially symmetric algebraic sur-face normal and the measured sherd surface normal. Minimizing (2.6)’, which is almost the same,is completely equivalent to performing a 3D axially symmetric surface fit using the method from[72].



: spatial asymmetry for the point (ri, zi) in the directions of the surface parallel and meridian are

indicated with (∆ri,∆zi) respectively. The important thing to notice is that the objective function

(2.6) seeks to minimize the approximate orthogonal distance of the measured data to the unknown

surface. Use of the orthogonal distance as an error measure makes this method distinct from [61].

In [61], a method is proposed which fits a set of circular cross-sections to binned data. The data

is binned according to a set of divisions along the z-axis and the circular fit within each bin is a

measurement of error in the surface parallels alone, i.e., this method minimizes observed errors

in ∆ri. In [55], a method is proposed which approximates the osculating sphere to the unknown

surface for each measured data point. This method is more accurate than that of [61] because the

errors being minimized here depend upon a second order expansion of the surface at the measured

surface point (see [56] for details on the theory). However, the surface continuity at orders higher

than two are not taken into consideration in such a model and estimation accuracy decreases with

decreasing angle between the surface normal and the object axis. In contrast, the algebraic surface

α(r, z) models the measured sherd surface as a C∞continuous surface, i.e., the model is infinitely

differentiable everywhere. This approximation will generate improved models where the surface

is smooth, which is often the case for archaeological sherds (see Figs. 2.4 and 2.6 for examples of

such sherds). It is also interesting to note that the approximation can be done without computing

any local information from the measured data which makes the estimation procedure robust to noisy

sherd data and estimates are still possible when there are no measured surface normals (here the term

‖npi −∇α(ri, zi)‖

2 from (2.6) is zero).

2.1.4 Axis / Profile Estimation

The method utilitizes a two-step iterative algorithm to estimate the axis and associated profile curve

which best describes the observed 3D data.

1. Based on the value of the objective function after the preceding iteration, choose a new value

for the axis parameters, l.

2. Based on the new l, compute egrad1, the value of the objective function by solving the

weighted linear least-squares problem (2.6). Note that (2.6) has an explicit solution which

may be computed quickly from the projected data.



Since the surface model depends upon the axis, the resulting objective function is highly non-linear.

Consequently, convergence to a local minimum may occur if minimization is started far from the

true parameter value. For this reason, we use the Nelder-Mead simplex method to minimize this

function. While standard gradient-based optimization techniques are possible for solving this prob-

lem, the chosen method was found to be more robust with respect to the numerous local minima

that can occur in the objective function for a given data set. The estimation algorithm needs only a

hypothesized axis of symmetry in order to begin.2

2.1.5 Experimental Results

A series of experiments were run on the sherds obtained from an excavation of the Great Temple

located in Petra, Jordan [37]. The sherds were scanned by a ShapeGrabber 3D laser scanner which

provides surface point and normal measurements for the sherd’s outer surface [4]. Results are pre-

sented for five sherds which are examples of the families of geometrically diverse sherds commonly

found on an archaeological excavation site. For each of these sherds there is a detailed error analysis

which is discussed to emphasize various subtle aspects of the problem which are more apparent for

specific types of sherd geometries.

Error analysis proceeds by applying the bootstrap method with a sample size of B = 500.

Each bootstrap sample uses the same number of points and normals as the original sherd data set

and is generated by a random re-sampling of the sherd point/normal data where a single data point

may be selected more than once [19]. For each of the 500 bootstrap samples, an axis/profile curve

estimate is generated and the value of the estimated axis is stored. The bth bootstrap axis estimate

is referred to as lb. It is also assumed that the resulting 500 axis/profile curve estimates represent

independent samples taken from a multivariate normal distribution with mean µ and covariance

Σ. Hence, one may estimate the distribution of the axis parameters by computing the estimated

mean µ = l = 1B

∑Bb=1 li and covariance Σ = 1

B−1

∑Bb=1(lb − l)(lb − l)t. These are sufficient

to construct an approximation of the true distribution of the axis parameters which are assumed2Typically, the initial axis estimate is obtained using the linear algorithm described in [65]. However, an additional

validation step is applied to detect degenerate surfaces of revolution by determining if the sherd data is well approximatedby a sphere, hyper-ellipsoid, or saddle surface. If the sherd data is determined to be spherical in nature, then no axisdirection is estimated and the axis location is taken as the sphere center. If the sherd data lies on a saddle surface or on anearly spherical ellipsoid such that the principle curvatures of the surface satisfies |κ1| ≈ |κ2|, then there are two possibleaxes of symmetry. In these cases, additional information is necessary to determine which of the two axes corresponds tothe unknown vessel axis of symmetry.



to be normally distributed∼ N (µ, Σ). Note, the error function minimized could result in slightly

biased estimates. Hence, the most important use of the bootstrap results is estimating the posterior

probability density function for the axis given the 3D measurement data set. This information

seems to be very important in deciding on the confidence the system can have in hypothesized

configurations of sherds.

For the purpose of comparing results between sherds of different shapes and sizes, a global

coordinate system is defined in which analysis of the axis/profile curve parameters for all sherds

takes place. This coordinate system transforms the sherd data such that the mean estimated axis, l,

is the world coordinate z-axis and the sherd data centroid lies on the x-axis.

Four sets of information are presented for each sherd : 1) an image of each measured sherd; 2)

the covariance matrix of the bootstrapped axis parameters (mx, bx,my, by); 3) a set of three profile

curve fits corresponding to the profiles generated from three different axis estimates: the mean

bootstrapped axis and two additional axes representing what we refer to as 95% confidence interval

for the largest mode of variation around the mean (see discussion for details); and 4) A plot of the

standard deviation of the laser scan 3D data points in the direction perpendicular to the axis as a

function of height, z, along the axis. The standard deviation is measured with respect to the profile

curve estimate obtained from the mean bootstrapped axis in (3).

In order to compute the 95% confidence interval for the largest mode of variation, the slope

parameters of the axis, (mx,my), from (2.2) were normalized by the height extent of the sherd, z0.

The new axis parameterization is given by (2.7).

x = mx

z0z + bx

y =my

z0z + by

(2.7)

Here z0 denotes the height range spanned by the profile of a sherd with respect to its mean estimated

axis. This normalization changes the axis slope parameters (mx,my) into parameters which repre-

sent slope of the axis line as a fraction of the sherd height when the line is projected into the xz-plane

and yz-plane respectively. This provides a meaningful measure which has units, %sherd heightmm.

, for

those components of the eigenvector which characterize the directions of the axis. For the param-

eter mx, the normalized parameter mx

z0represents the change in axis direction in the xz-plane as a

fraction of the total sherd height per incremental step, ∆x, in the direction of the x-axis: ∆z(%)∆x(mm.) .



(a) Sherd image

mx bx my bymx 0.0111 -0.0661 0.0002 -0.0005

bx -0.0661 0.4179 -0.0004 -0.0022

my 0.0002 -0.0004 0.0002 -0.0015

by -0.0005 -0.0022 -0.0015 0.0100

(b) Axis parameters covariance matrix (mm.)

(c) profile curve estimates for the largest mode of variation (mm.) (d) standard deviation (mm.)

Figure 2.4: Experimental Results for sherd 654

Fig. 2.4 : The sherd profile curve has a simple shape which is commonly associated with body sherds

found at a typical archaeological excavation. A body sherd is any sherd which comes from neither

the rim nor the base of the pot. Since curvatures are small in direction of both sherd meridians and

sherd parallels, axis estimates here are difficult. The accuracy in estimating the axis of this piece

demonstrates robustness to data noise.



(a) Sherd image

mx bx my bymx 0.0290 -0.1748 -0.0034 0.0264

bx -0.1748 1.6484 0.0389 -0.3441

my -0.0034 0.0389 0.0013 -0.0109

by 0.0264 -0.3441 -0.0109 0.0979




Fig 2.5 : The sherd has parallels with small curvatures, yet the meridians of this sherd have signif-

icant curvature information. Axis estimation for geometries such as this is especially difficult for

methods based on surface normal lines. This is due to several reasons : 1) the piece is very small, 2)

it has a multiple-valued profile curve, and 3) it has normals which are almost parallel to the axis of

symmetry. Since the proposed method incorporates normal information differently than those based

on normal lines [55, 65], these apparent obstacles do not cause the algorithm problems. Uncertainty

in the axis location is represented by parallel profile curves in (c) and again in (b) by examining

elements (2,2) and (4,4) of the axis parameter covariance matrix.



(a) Sherd image

mx bx my bymx 0.0979 -0.5373 -0.0255 0.1404

bx -0.5373 2.9906 0.1406 -0.7820

my -0.0255 0.1406 0.0069 -0.0380

by 0.1404 -0.7820 -0.0380 0.2111




Fig 2.6 : The sherd here has small curvatures along sherd parallels and a small region of high

curvature along sherd meridians. As in Fig. 2.5, it is difficult to accurately estimate the axis location

when there is little data (i.e. a small sherd) combined with small curvatures in the direction of the

sherd parallels. This is evident in the parallel translations of the profile curves in (c) as well as in

(b) by examining the covariance matrix. This leads to an important inference: Sherds which have

little data with respect to their subtended angle about the axis of revolution and small curvatures in

the direction of the surface parallels have more variation in their axis location parameters (bx, by).



(a) Sherd image

mx bx my bymx 0.0042 -0.0026 0.0001 0.0004

bx -0.0026 0.0031 0.0003 -0.0004

my 0.0001 0.0003 0.0002 -0.0001

by 0.0004 -0.0004 -0.0001 0.0002


(c) profile curve estimates for largest mode of variation (mm.) (d) standard deviation (mm.)


Fig 2.7 : The sherd here has highly curved parallels and meridians. Yet the sherd ridges are chipped

and worn (see part (a)) which generates a large amount of asymmetric data measurements. The

axis/profile curve estimates here are very accurate which reflects good robustness to asymmetry in

regions of high curvature and to local surface noise.



(a) Sherd image

mx bx my bymx 0.2317 -0.0771 0.0193 -0.0076

bx -0.0771 0.0715 -0.0078 0.0058

my 0.0193 -0.0078 0.0088 -0.0027

by -0.0076 0.0058 -0.0027 0.0022




Fig 2.8 : Particularly interesting aspects of the curve are the points of high curvature along the

profile found at the sherd rim and again just slightly down from the sherd rim. These points appear

to be nearly discontinuous in their second derivative, i.e., the curvature at these locations jumps. In

cases such as this, the actual surface may not be C∞ continuous and the resulting algebraic surface

fit regularizes the locations which violate this assumption. Hence, pieces such as this are more

difficult to estimate using our axis approximation method.



(a) p967 data projection (mm.) (b) p1313 data projection (mm.)

Figure 2.9: Sherd data projected into (r, z) plane using the mean axis estimate, l.

Figure 2.9 provides perspective on the axis estimates which are obtained via the method. For

Figs. 2.4-2.8, plotted profile curves are zoomed to accentuate the source of the small observed

errors in the estimates. Since variations typically range between 0.3-0.001 mm., without zooming

in closely to examine small fluctuations, it is difficult to discern any variation in the three 95%

confidence interval profile curves for a sherd.

In Fig. 2.9, a non-scaled version of the original 3D data points is shown which have been rotated into

the (r, z) plane. The y-axis is height along the estimated axis, and the x-axis is distance from the

estimated sherd axis for each 3D data point. The best axis estimate will be that for which the scatter

is smallest locally in the direction perpendicular to the true surface. The data scatter for sherds 967

and 1313 is shown in Fig. 2.9. What is amazing is the small data scatter, especially in Fig. 2.9(a)

where the sherd is almost horizontal with respect to it’s estimated axis. Qualitative confidence in

the axis direction estimation is obtained by observing rings on the inside of the sherd resulting from

pot construction on a spinning wheel.



Having normalized the axis directional parameters, we proceed by taking the eigenvector, v,

associated with the maximum eigenvalue of the axis parameter covariance matrix, i.e., the largest

mode of variation, and choose the parameter values at the two ends of the 95% confidence interval

in the direction of the eigenvector assuming the parameters have the distribution N(µ, Σ). This gen-

erates three axes : the mean axis parameters given by µ, and the parameters defining the confidence

interval [µ + 2σ, u − 2σ]v. Note that this confidence interval does not capture all major modes of

variation for the parameters (there are 3 additional modes of variation). Yet, it provides some in-

sight regarding the dominant direction of variation for the bootstrap estimates. This dominant mode

indicates those parameters which have the most variation or equivalently are less reliable. Three

profile curves are shown which are the different profile curve fits obtained by solving (2.6) using

the three different axes. The profile curve associated with the mean axis estimate is shown in red

and the profile curves associated with the axes from the confidence interval are shown in blue and

green.

2.1.6 Conclusion

This section of the thesis has detailed a highly accurate method for automatically estimating axially

symmetric surface models given dense 3D laser scanner data from pot sherds. The end result is

an axis/profile curve pair for a sherd, and this completely specifies the sherd surface geometry. If

desired, the accuracy of an estimate can be further improved by not using data that occurs at high-

curvature points on the sherd surface in the estimation procedure. This data can be automatically

identified by detecting data outliers with respect to the presently estimated profile curve models.

The estimated axis/profile curve models are close to being maximum likelihood estimates, and

should therefore be close to maximum accuracy because MLEs are known to be asymptotically of

minimum variance, and the number of data points used in estimating sherd geometry is of the order

of 10,000. Multivariate Gaussian distributions were estimated for the sherd axis for a variety of

sherd geometries. The range of geometries included in the experimental data set allows for insightful

comments which accompanies the presented statistical data. Finally, these distributions could be

used for pot reconstruction because they tell the reconstruction algorithm how much confidence it

can have in various estimated parameters.


2.2. ASSEMBLING CONFIGURATIONS OF SHERDS 38

Figure 2.10: A broken vessel and its parametric description.

2.2 Assembling Configurations of Sherds

2.2.1 Introduction

This section discusses a solution to the problem of estimating an unknown free-form axially sym-

metric surface from an incomplete set of its pieces. The pieces individually describe small portions

of the unknown surface and are connected to each other by break-segments, i.e., locations where

the pieces were broken apart. During the assembly process, the surface reconstruction program

must determine which break-segments are shared between individual pieces in order to discover

the unknown global structure of the pot. This is a difficult search problem whose solution is given

in §2.4. This section, §2.2, treats the problem of aligning N sherds where it is assumed that the

correspondence between the sherd break-segments is already known. The problem here is to simul-

taneously estimate all of the geometric parameters of the unknown axially symmetric surface while

simultaneously aligning the N sherds.

2.2.2 Parameters to Estimate

The break-curve parameters, β, are points on the surface where the surface has broken into two

pieces. These locations include vertices—locations of Y and T junctions (see Fig. (2.10)). Note that

junctions denote points which are high-curvature points as one travels along the piece boundaries.

T-junctions are points which are high-curvature points on two of the three piece boundaries and



Figure 2.11: Break-point segments : a sherd outer surface, in grey, and one of the sherd verticesis shown as a large opaque red sphere. Two sets of break-point data which are called break-pointsegments are generated shown as light grey and black points on the break-curve. Each break-pointsegment has four ordered elements, starting with the vertex and then listed in order of increasingdistance from the vertex. Note that break-point segment points lie at locations where a sphere (intransparent blue) of radius kR intersects the break-curve where k = [1, 2, 3].

Y-junctions denote points which are high-curvature points for all three piece boundaries. These

high-curvature points are referred to as vertices. Each break curve is represented as a sequence of

K points starting on a vertex. The points in each sequence occur at successive intervals of fixed

length from the vertex (see Fig. (2.11)) and each sequence is referred to as a break-point segment.

These points, along with a surface normal at each point, constitute β—the parameterization of the

surface break-curves for the axially-symmetric surface. Hence, break-point segment v is written

βv = ((p1,p2, . . .pK) , (n1,n2, . . . ,nK))t where pk denotes the kth 3D point and nk denotes the

kth 3D normal for βv . The group of all break-curve parameters is β = ∪v=1βv .

The kth point in a break-point segment is the location where the break-curve intersects a sphere

of radius kR centered at a vertex point where k = [0, 1, 2, . . . ,K − 1] and R is the radius of a

sphere centered on the vertex. Figure (2.11) illustrates two break-point data segments for a vertex

where K = 4. Note that the break-point segment data used is that which is illustrated in Fig. 2.11.



This choice is not rigorous because it is the point on the true break-point segment that should be a

distance kR from the vertex.

The surface axis of symmetry is parameterized using the standard parametric equation of a 3D

line as shown in (2.2). Hence l = (bx, by,mx,my)t and consists of the pair (mx,my) as specified

in §2.1.3. A profile curve α(r, z) with respect to the piece axis l is our model for the unknown 3D

axially-symmetric algebraic surface with axis l where r denotes radius, i.e., the shortest distance

between a 3D point and the axis and z denotes height along the axis (see Figs. 2.10,2.2,2.3). The

surface parameters are the coefficients of the algebraic profile curve in (2.3) and the axis of symme-

try in (2.2). Hence α = (∪j,kajk)t is the vector of coefficients for the implicit polynomial curve of

degree d. Note that for all experiments contained in the thesis, d = 6.

It is assumed that each piece undergoes an arbitrary rigid Euclidean transformation which moves

the piece to its measurement position. Hence, for the ith piece the transformation, Ti, which moves

the piece from its measurement position to its aligned position must be estimated. The 3D transfor-

mations are parametrized with 6 parameters consisting of 2 parts : (1) a 3D translation vector t, and

(2) a 3D rotation R. The 3x3 rigid rotation matrix R is represented using the so called axis-angle

parameters which describe rotation in terms of a rotation angle ψ about a 3D unit vector nR. Hence

pure rotation is the 3D vector (ψnR) and the equivalent 3x3 rotation matrix is referred to as R. The

3D transformation parameters are T = (t, ψnR)t (see Appendix A of [63] for additional details on

this parameterization). Further, transformations of piece datasets are denoted as T(D) for surface

point/normal data and T(B) for break segment point/normal data. In this notation it is assumed that

T(D) indicates that a single transformation, T, operates on each of the points and normals in D

according to the rules shown in (2.8).

Tp = Rp + t

Tn = Rn

(2.8)

2.2.3 Data Generation Model

The surface measurement data is provided by a Shapegrabber laser/camera scanner [4]. It produces

15,000 3D points/sec. at a resolution and accuracy of the order of 0.25mm. The data is divided into

two sets : (1) 3D surface points and normal measurements, denoted Di for the ithsurface piece, and



(2) 3D break-segment points and their surface normals, denoted Bv for the vth break-segment.

Data for each break-segment is extracted according to a simple parameterization of the un-

known surface break-curves adopted in §2.2.2. Hence, break-point segment data is simply a special

sequence of K measured 3D points and outer surface normals (see the discussion on break-curve

parameters in §2.2.2 and Fig. 2.11 for clarification).

Assumptions

Surface measurement points are i.i.d. N(0, σ2DI)

These are independent, identically distributed, spherically symmetric Gaussian perturbations in 3-

space about each point on the true surface with mean 0 and variance σ2D. See [14] for a justification

of this model.

Surface measurement normals are i.i.d. N(0, λσ2DI)

These are independent, identically distributed symmetric Gaussian perturbations on the unit sphere,

i.e., in SO(3), about the true surface normal for each point on the sherd outer surface with mean 0

and variance λσ2D. These are independent and for the j th piece are distributed over a spherical cap

about a mean that is normal to the surface as represented by an axis/profile-curve for the j th piece.

Break-segment measurement points are i.i.d. N(0, σ2BI)

These are independent, identically distributed spherically symmetric Gaussian perturbations in 3-

space about each point on the true break-curve, with mean 0 and variance σ2B. Note that more

appropriate but more complicated models can be used.

2.2.4 Algorithm

Alignment is done by adding one piece at a time, each addition producing a new configuration. For

each new configuration, all parameters are re-estimated to minimize the joint probability density

for all the break-point segment and surface data associated with the pieces in the configuration.

However, this is not done in one simultaneous nonlinear minimization, since that would be both

much too time consuming and would likely converge to a local maximum. Note, for an N piece

configuration, the number of parameters to be estimated is : 6(N − 1) Euclidean transformation

parameters; 4+ d(d+1)2 global surface parameters for a dth degree surface (if d = 6 then there are 29



parameters); and a minimum (and likely many more) of 3K(N−1) break-point segment parameters

where K is the number of break-points in a segment. Estimating an axis/profile curve for a surface

data set of a few thousand points may require up to 1 minute if using an arbitrary starting point.

Using a small-error starting point can result in orders of magnitude computation reduction. Such

is the case when the system constructs configurations during the search (see §2.2.5 for details on

how a suitable initial point is found). Estimating a Euclidean transformation for aligning a pair of

matched break-point segments is a linear least squares computation on the order of a millisecond.

1. Given a N piece configuration denoted ConfigN , the algorithm begins by treating the con-

figuration as though a single piece. The global axis/profile-curve parameters for ConfigN

are estimated by solving the non-linear equation (2.9) (see §2.2.5 for details).

l, α = arg maxl,α

ln

∏

Di∈ConfigN

P(Di|l,α, β, Ti

) (2.9)

Upon convergence, the parameters of the estimated axially symmetric surface are stored :

(l, α). Note that the break-point segment parameters β do not play a role in this optimization

yet are included for completeness in (2.9).

2. A new candidate piece N + 1 is taken and the Euclidean transformation is computed which

moves this piece so that its break-point data segments align with the matched break-point

data segments of ConfigN and its surface data fits the 3D surface model for ConfigN by

maximizing (2.10) (see §2.2.6).

eTN+1,eβ = arg max

TN+1,βln

“P

“DN+1,BN+1,BcN |el, eα, β,TN+1

””(2.10)

Where TN+1 denotes the Euclidean transformation for piece N + 1 , DN+1 denotes trans-

formed surface data of piece N + 1. BcN and BN+1 denote the matched break-point data

segments of ConfigN and the transformed piece N + 1 respectively. β denotes the true

values of the unknown parameters for the matched break-point segments.

3. Add piece N + 1 to ConfigN and go to step (1) if the number of pieces in ConfigN is

smaller than the total number of available pieces.



It is possible to adjust all N pieces slightly by re-estimating the transformation for piece 1 to make

its data better fit the configuration defined by piece 2 through N , then repeat with piece 2, etc., but

the improvement is small, and doesn’t appear to be necessary.

2.2.5 Estimating the Surface

Surface shape parameters are obtained by finding the axially symmetric algebraic surface which

best fits the aligned data sets using the method outlined in §2.1. This is a non-linear minimization

consisting of two steps:

1. Based on the value of the objective function after the preceding iteration, choose a new value

for the axis parameters, l.

2. Fixing the axis l = l, the system computes the axially symmetric surface α which maximizes

the probability of the data by fitting an axially symmetric surface to the 3D data with axis l.

This is equivalent to solving the weighted linear least-squares problem (2.11) which has an

explicit solution and incurs little computational cost. Note that (2.11) is an abbreviated form

of (2.6) where the points included in the summation and the specific value of λ may differ.

e(α|l = l) =∑

p,n∈ConfigN

(α2(p) +

1

λ‖n −∇α(p)‖2

)(2.11)

The objective function (2.11) is very similar to (2.6) from §2.1 and is minimized in the same way,

i.e., via simulated annealing. However, a very important distinction is the choice of the initial point

for the non-linear minimization. Consider a merge between an assembled configuration pair and

a new sherd. In this case, the system starts minimizing (2.11) using the MLE value of the axis

parameters, l, obtained as a solution for the pairwise configuration. For correctly matched sherds,

this initial point provides a good starting point for the global parameters (l,α) and greatly expedites

the minimization process. Note that the error metric α2(p) is the so-called algebraic distance which

can be used as an approximation of the Euclidean distance (for details see [74]), and 1λ‖n−∇α(p)‖2

is for regularization [87].



(a) Example of algorithm Step 1. (b) Example of algorithm Step 2.

Figure 2.12: Surface Alignment : ConfigN consists of two pieces as shown in (a). The axiallysymmetric surface (α, l) obtained from step (1) of the algorithm is shown in transparent cyan andthe profile α is shown as a black curve. Piece N + 1, in green, is added to the configuration usingthe rough alignment given by solving (2.13), the resulting 3-piece configuration is shown in (a).The algorithm then solves (2.15) to obtain an accurate alignment for piece N + 1 using its matchedbreak point data segments, BN+1, and a subset of its surface points, DN+1. DN+1 denotes thosesurface points of piece N + 1 whose height along the axis l is within the height interval defined bythe ConfigN data set, i.e., the bracketed region in (b).

2.2.6 Estimating the Transformation

A coarse estimate for the piece N + 1 transformation is obtained by aligning the subset of its

corresponding break point segments to those of ConfigN , i.e., solving the optimization problem

(2.12).

ˇTN+1 = minTN+1

‖TN+1(BN+1) − β‖2 + ‖BcN − β‖2 (2.12)

From the assumptions stated in §2.2.3, this corresponds to solving (2.13).

TN+1 = minTN+1

‖BcN −TN+1(BN+1)‖2 (2.13)

Where the difference between two corresponding break-point segments is defined in (2.14).



‖BcN −TN+1(BN+1)‖2 =

K∑

k=1

‖pcN,k −TN+1pN+1,k‖2 +

1

λ‖ncN,k −TN+1nN+1,k‖

2 (2.14)

The optimization (2.13) is commonly referred to as the absolute orientation problem to which there

exist several explicit linear solutions [83, 35, 33]. The system uses the solution proposed by [83] (see

Appendix A for details). Note that (2.12,2.13,2.14) include a term involving the surface normals at

each break segment data point which aligns the measured surface tangent-planes of corresponding

points. This adjustment improves the alignment especially in cases where the break-point segment

data has little curvature.

Using surface estimate (l, α) from step (1) of the algorithm (see §2.2.4), the system solves the

non-linear alignment problem (2.15) using the rough estimate TN+1, and β =TN+1(BN+1)+BcN

2 as

an initial point.

e(TN+1, β) =1

σ2D

α2(TN+1( eDN+1)) +

1

σ2B

‖TN+1(BN+1) − β‖2 +1

σ2B

‖BcN − β‖2 (2.15)

Note that the assumed distribution stated in §2.2.3 makes solving for β trivial. However, the param-

eters of the unknown break-point segments in (2.15) are present to show that the system seeks to

estimate the unknown geometry of the unknown break-point segment β. This solution for β may be

non-trivial in a general situation where the measurement noise distributions differ from that in §2.2.3

. DN+1 denotes the subset of the pieceN+1 surface data as shown in Fig. 2.12(b). This is the subset

that lies within the extent of the estimated profile curve for ConfigN . The term α2(TN+1(DN+1))

is the approximate Euclidean distance of the transformed data set DN+1 to the estimated axially

symmetric surface for ConfigN . The surface alignment error being minimized here is the square

of the algebraic distance (see Section XI of [74] for details where a similar alignment problem is

solved). The break-segment alignment error is ‖TN+1(BN+1) − β‖2 + ‖BcN − β‖2 which de-

notes the Euclidean distances between the matched breakpoints on piece N + 1 and ConfigN and

the true locations of the break points β. The solution to (2.15) is computed by direct non-linear

minimization using the well-known Levenberg-Marquardt algorithm.


2.3. THE BAYESIAN FORMULATION 46

2.2.7 Results

Given the large number of unknown parameters being estimated, the algorithm succeeds in finding

the global MLE solution quickly. The 3 piece configuration in Fig. 2.12(b) required approximately

10 seconds to compute. Note that this minimization involves 221 parameters (12 transformation pa-

rameters, 29 surface parameters, and 180 break segment parameters). Fig. 2.20 shows the solution

for a 13 piece axially symmetric vessel where only 10 pieces are available which required approxi-

mately 45 seconds to compute, the number of estimated parameters here is 893 (54 transformation

parameters, 29 surface parameters, and 810 break segment parameters).

2.2.8 Conclusion

A robust and fast method was proposed which estimates an unknown axially symmetric surface

from measurements of a subset of size N of its pieces. As others have in the past, the problem

is decomposed into a set of N − 1 pairwise piece alignments. The proposed method is unique in

that it solves the problem using a 2 step recursive algorithm. In step 1, the geometric constraint of

axial symmetry is used to obtain an estimate of the unknown surface for a configuration of aligned

pieces. Step 2 uses this surface estimate to obtain accurate estimates of the break segment and

transformation parameters for a new piece added to the configuration. There is a rationale for this

approach. The algorithm is shown to work and performs well even though dealing with a vast

number of unknown parameters.

2.3 The Bayesian Formulation

2.3.1 Introduction

Estimation of the true global shape and structure of the assembled pots is implemented as Maximum

Likelihood Estimation (MLE) of a set of geometric parameters which characterize the unknown pot

surface and the unknown curves along which the pot originally broke. Since these parameters are

not Euclidean invariant, we must also estimate the 3D transformation parameters which align the

sherds into a global coordinate system. Individual sherds and configurations of more than one sherd

are cast into a common Bayesian framework. This section details the structure of the Bayesian



Symbol Significancel the vessel axisα the axially symmetric surface of the entire vessel

(i.e., profile-curve, α(z), with respect to z-axis which is the vessel axis)β break-curves for entire vesselTi a 3D transformation which takes

sherd i into its aligned position in the vessel

Table 2.1: Basic geometric parameters.

model and how this model is used to represent the geometric knowledge the system has obtained

from the measured data.

2.3.2 Model Parameters

Axially symmetric vessels are modeled using a subset of the complete geometric information that

can be used. It consists of the outer-surface break curves, break curve vertices at sherd junctions (see

Fig. (2.10)), an axis/profile curve for the entire pot and portions for individual sherds, and Euclidean

transformations that take each sherd from its data-measurement position to its aligned position in a

configuration.

The goal is to estimate the global pot parameters in Table (2.1) by hypothesizing matches, i.e.,

transformations, between sherd datasets. The most probable matched set of sherd data given the

sherd transformation values and the geometric parameters is considered the most likely pot. §2.1 and

§2.2 detail methods for estimating these parameters for individual sherds and sherd configurations.

2.3.3 Probability and Knowledge

The Bayesian framework provides a completely general probabilistic framework from which one

may construct computational models of knowledge. The knowledge being modeled is application

dependent and is represented in terms of a set of parameters, θ, whose true value are assumed to be

unknown. The typical Bayesian parameter estimation problem assumes the following [24]:

• The form of the distribution p(x|θ) is known but not the value of the parameters θ.

• All previous knowledge about the parameters is contained in a known prior distribution p(θ).



• All additional knowledge about θ is contained in the data set D which consists of M samples

drawn independently according to the unknown distribution p(x).

Bayes rule (2.16) provides a method for converting the posterior probability distribution p(D|θ)

into a posterior distribution on the parameters p(θ|D).

p(θ|D) =p(D|θ)p(θ)∫p(D|θ)p(θ)dθ

(2.16)

The unknown quantity in (2.16) is p(D|θ), by the assumption of independence this is the product

(2.17).

p(D|θ) =

ND∏

i=1

p(xi|θ) (2.17)

In the context of estimating the geometry of an axially symmetric vessel from its sherds, the data

consists of N data sets D1,D2, ...,DN , each of which consists of two distinct subsets : (1) the

collection of sherd surface measurements, Gi, for the ith sherd, and (2) the collection of sherd

boundary measurements, Bi, for the ith sherd. The data independence assumption and the form of

the distribution p(x|θ) are detailed in §2.2.3. It is also assumed that no prior information is available

to the system, i.e., p(θ) ∼ Uniform(θ).

Suppose the system is assembling a single vessel and that knowledge of the original shape

of the vessel surface and the shapes of all break curves are known probabilistically. In Bayesian

estimation, this knowledge specifies a distribution on the unknown parameters p(D|θ)p(θ) which

could be used to improve estimates of the parameters. However, it is assumed that this distribution

is not known and further, that the configurations being constructed could come from more than

one axially symmetric vessel. Since there is no prior information available about p(θ), the system

considers all possible axially symmetric surfaces and all their surface curves as equally likely events

in nature. In this case, p(D|θ)p(θ) = kp(D|θ) where k is a constant. The geometric interpretation

of this is that the system assumes only that given a sherd pair (i, j) matched along their break

segments (m,n), the sherds surface data lies on an axially symmetric surface which contains the

surface curve described by their matched space curve pair (m,n). This is actually a combination

of 2 assumptions : (1) the sherd break-point segments describe the same 3D space curve, and (2)



(a) (b) (c)

Figure 2.13: A sherd triplet decomposed into its global and local parameters. There are two localbreak-curve segments being estimated, i.e., the curves for sherd pair 6,10 and 5,6. The matchedbreak-curve data is shown in Fig. (a) where points are rendered as spheres on the sherd boundaryand the surface normal at each point is rendered as a cylinder passing through the sphere center.There is also a single global cost, i.e., all 3 sherds describe the same axially symmetric surface.Fig. (b) shows the estimated axes for each of the 3 sherds as red cylinders, the global geometricparameters l models these as noisy estimates of the true vessel axis, shown as a black line labeled z.Fig. (c) shows the (r, z) projection of the measured sherd data points for all 3 sherds in dark green,the global geometric parameters for the axis/profile-curve l,α models the measured sherd data lieson a continuous axially symmetric surface, represented as a profile-curve with respect to the globalvessel axis. The profile curve, α(r, z) = 0, is plotted in light green and interpolates the projected(r, z) surface data points.



when the sherds are aligned along this space curve, their surface data lies on an axially symmetric

surface.

The assumption p(θ) ∼ Uniform(θ) is clearly not true for archaeological pottery. For exam-

ple, ancient pottery objects commonly have an open end for purposes of placing material inside the

pot. Knowledge such as this is not reflected in how the system treats configurations. In theory, the

parameters of the distribution, p(θ), could be estimated from a set of learning data. In this case, the

system can exploit this prior information using (2.16) to define a more accurate probability metric

on the parameter space p(θ|D) by computing what is now a Maximum A-Posteriori (MAP) estimate

of the unknown parameters θ given the measured data D. It is also theoretically possible to remove

the assumption that the sherds from a single vessel have matching boundaries. In this case, the sys-

tem must uncover the global shape of the vessel by aligning the sherds using only their axis/profile

curve pairs. However, the implemented system constructs hypotheses which depend upon an as-

sumed correspondence between sherd boundaries, i.e., break-point segments. Hence, solutions of

this type cannot be constructed by the proposed system. In cases where many sherds are missing,

this may make it impossible for the proposed solution to construct the global pot solution. Fig. 2.14

provides some information regarding the susceptibility of the system to missing sherd data.

The proposed method of break-curve alignment is tractable for all break-curve pairs only due

to the fact that the pot vertices provide a method for computing break-curves which have the same

parameterization based on the vertex location. Two possible avenues for increasing robustness to

missing sherd data is to introduce more break correspondences between sherds by attempting to

match triplets as done in [49] or by assembling sherd configurations using their axis and profile

curve estimates. While both avenues improve the system robustness this is at the expense of in-

creased computational cost. Specifically, additional correspondences increase both the number of

potential matches between sherds and the computational cost of each match. This is because eval-

uating matches reduces to the problem of matching two curves where each curve has a different

parameterization. For additional break-curve matching, this is a 3D curve-matching problem and

for axis/profile-curve matches this is a 2D curve matching problem.

Cost is used as the metric for determining whether a constructed configuration is correct. It

takes all of the evidence available, i.e., all of the sherd data, and maps it to a single positive real

number, i.e., for a configuration involving a set of sherds S , cost(∪i∈SDi) → R > 0, where S is



(a) (b)

(c)

Figure 2.14: Shown are three pots assembled by humans (photograph by Jill L. Baker, courtesyof the Leon Levy Expedition to Ashkelon). Each sherd in the assembled pots is labeled with anumber and break-curve segment correspondences between sherds are indicated as edges betweenthe labels. For each of the rendered images, a red circle has been chosen to show the location ofa T-junction of interest. In particular, T-junctions on large sherds such as sherd 1 in Figs. (a) and(c) and sherd 10 in Fig. (b) can make computation of global solutions difficult if there are manymissing sherds. For example, assume that only the sherds shown in Figs. (a-c) are available forcomputation, in this case, half of the entire object is missing for both (a) and (c). In this situation,consider how the global solution will depend upon the omission sherds of the shown set of sherds.In Fig. (a) if sherds 11 and 10 are missing from the puzzle, the proposed system will not be able toconnect the cap of the amphora to the remaining body of the amphora. In Fig. (b) the omission ofsherds 4 and 15 will not allow the system to connect the lower portion of the pot to the remainingpieces. In Fig. (c) omission of sherd 2 will not allow the system to add sherd 1.



a subset of sherd indices, e.g., S = 3, 8, 2defines a triplet configuration involving sherds with

indices 3, 8, and 2 for a puzzle which may consist of 10 sherds. The system computes the cost of

a configuration by estimating the MLE, θ, of the configuration parameters and then computing the

logarithm of the probability of the data given the MLE value (2.18).

cost(∪i∈SDi) = cost(∪i∈SDi|θ) = − ln(p(∪i∈SDi|θ)) (2.18)

As mentioned in §1.3.3, each configuration of N sherds has parameters θ = (β, l,α, ∪i∈STi)

where β denotes the locally matched break-curves, (l,α) denotes a single vessel axis/profile-curve

pair, and ∪i∈STi denotes the parameters of the N −1 transformations which align the N−1 sherds

to a global coordinate system defined by the N th sherd. Hence the local match between breaks

curves and the global requirement that the sherd surface data lie on an axially symmetric surface

are integrated into a single expression. Fig. 2.15 uses a triplet configuration of sherds to illustrate

how all of the measured sherd data is taken into account when evaluating the cost of a sample

configuration.

There is a rationale for using cost as a metric. The MLE estimate represents the most probable

value of the unknown vessel parameters given the assumed correspondence between the measured

sherd data (2.19).

θ = arg maxθ

− ln(p(∪i∈SDi|θ)) (2.19)

Configurations which result in sherd configurations whose break point segments match well and

surface data lie close to a perfect axially symmetric surface will have low cost and vice-versa (see

Fig. 2.13). Hence, the MLE provides a natural ordering of configurations in terms of the probability

that the configuration is an actual surface of axial symmetry. As discussed previously, the true

distribution of the vessel parameters is unknown which makes it difficult to accept or reject any

single configuration as correct or incorrect. Hence, in practice, the system retains a sequence of

configurations which are ordered by their cost value. Since the number of pairwise configurations

for even modest puzzles (<10 pieces) is extremely large, the system retains only those hypotheses

which have low cost, i.e., those configurations where the probability of the data given the estimated

MLE is high.



2.3.4 Physical Constraints

The cost (2.18) represents how well a given set of sherds describe a single axially symmetric vessel

given a postulated set of matches between their break point segments. However, there are additional

physical constraints which the sherds must satisfy which is not taken into consideration in the con-

figuration cost. The system imposes the constraint that sherds in a configuration cannot coexist in

the same space, i.e., their surfaces may not overlap nor intersect. This is implemented by projecting

the sherd boundaries, represented as an ordered sequence of points, into the (r, θ) and (r, z) planes

of a cylindrical coordinate system whose origin is a point on configuration’s MLE axis of the con-

figuration and whose z-axis is in the direction of the MLE axis. Each sherd boundary will define a

closed (but possibly complicated) polygonal shape. Samples are uniformly taken from the plane in

the interior of each of the polygons. Those samples which lie inside more than one polygon denote

areas in which sherd surfaces are found to overlap. The cost from (2.18) is increased by a penalty

which is a quadratic function of the maximum sherd overlap in each of the two projections (2.20).

cost(sherd overlap) =

(k ∗

maxi(overlap(r,θ)(sherd i)) + maxi(overlap(z,θ)(sherd i))

2

)2

(2.20)

Where the function overlap(sherd i) is computed in terms of the percentage of samples from sherd

i which lie inside the polygons defined by other sherds :

overlap(sherd i) =# of samples inside the boundaries of sherds i and j for j 6= i

# of samples inside the boundary of sherd i

Fig. 2.15 shows the projection generated from the configuration in Fig. 2.13. In the case of the

(r, θ) projection, faulty overlap costs will be computed if the surface is not a monotonic monotonic

function of the MLE axis z. However, in more general situations one must locally project the sherd

boundaries within intervals where the profile curve is locally monotonic with respect to z, height

along the axis of symmetry.

The cost (2.20) is added to the cost of the original configuration (2.18) creating a total cost for

the sherd configuration. The total cost is exploited both as a measure of how well the hypothesized

sherd set represents an axially symmetric object and as an ordering statistic for generating new



Figure 2.15: The boundaries of sherds 5,6,10 from figure 2.13 are projected into the (r, θ)and (r, z) planes of a cylindrical coordinate system whose origin is a point on the MLE axis ofthe configuration whose z-axis is in the direction of the MLE axis. For this configuration thecost(sherd overlap) = 0. Note that the (r, θ) projection is accurate since the profile curve is amonotonic function of the axis as shown in Fig. 2.13(c).


2.4. THE SEARCH 55

configurations.

2.3.5 Conclusion

A Bayesian approach has been outlined for the estimation of mathematical representations for pots

based on sherds found at archaeology sites. The framework discussed in this paper is for estimating

arbitrary a-priori unknown axially-symmetric pot models. Hence, it is unsupervised pot geometry-

learning from sherd data. If instead we know a-priori that the pot sherds present are not arbitrary

but rather that each belongs to one of a group of 10 known pot shapes e.g., the problem is computa-

tionally much easier because the sherd alignment problem is then more of a pot shape-recognition

problem and less of a shape-estimation problem. The framework presented can accommodate addi-

tional geometric and pattern information which should result in doing the pot estimation faster, or

with fewer sherds, or estimating models for more complex objects.

2.4 The Search

2.4.1 Introduction

The search problem determines which sherd configurations are constructed and in what order they

are made. The order of construction is important since the number of possible configurations is huge

which makes consideration of all possible configurations computationally intractable. To develop

the discussion, we consider the problem of configuration assembly as a graph problem. A directed

graph G is a pair (S, E), where S is a finite set of nodes and E is a set arcs which specify binary

relations between nodes [20]. Arcs are directional and specify a relation from one node to another

node, e.g., the arc from nodeA to nodeB is denoted A→ B. Note that in some terminologies nodes

are referred to as vertices and arcs are referred to as edges. However, to avoid confusion between

the geometric problem involving pot vertices and sherd edges, we do not use this terminology.

Arcs are created by applying an operator on a node pair which assigns a positive cost value to

the relation A → B and is denoted cost(A → B). Graphs which contain directional arcs are

directed graphs or digraphs. If the graph arcs are bi-directional and the cost of arcs is symmetric,

i.e., cost(A→ B) = cost(B → A), then the graph is an undirected graph.

For puzzle assembly, the set of all nodes is simply the set of all the available pot sherd data. For


2.4. THE SEARCH 56

example, given an N sherd puzzle, the complete set of nodes is then Sall = D1,D2, ...,DN. Arcs

on the graph express binary relations between sherd data and represent the assumed correspondences

between the matched sherds in a configuration. Each sherd has two types of arcs : (1) a local arc

which represents matched break-point data segments, and (2) a global arc which represents how the

sherd matches the global axis/profile-curve model.

All sherds in a configuration have a global arc to the MLE of the vessel axis/profile-curve.

The cost of this arc is due to errors between the measured sherd surface data and the MLE of

the axis/profile-curve parameters. Hence, the cost of global arcs depends upon the subspace of

axis/profile-curve parameters. This is denoted cost(Gi|θg) where Gi is the measured surface data

of the ith sherd and θg

= (l, α) is the MLE of the vessel axis and profile-curve parameters. Each

matched break curve on a sherd defines a local arc between the two sherds whose cost is the cost due

to errors between the measured break-point segment data and the MLE of the corresponding vessel

break-point segment parameters. Hence, the cost of local arcs depend upon the subspace of matched

vessel break-curve parameters. This is denoted cost(Bi|θl) where Bi is the ith matched sherd break-

point data segment and θl= (β) is the MLE of the unknown vessel break-point segment.

Given our definitions of global and local arcs, the configuration cost (2.18) has an equivalent

graph representation GS . Each break-point segment match between sherds in S defines a local arc

on the configuration graph GS and all sherds share a global arc with the global vessel axis/profile

curve model. Fig. (2.16) shows the graph corresponding to a pairwise configuration of sherds.

For any given node, the maximum number local arcs is limited by the number of its break-point

data segments, i.e., a sherd with 4 break-point data segments may have up to 4 arcs. Rendered

graphs show unconnected break-point data segments as unconnected line segments associated with

each node (see Fig. (2.16)). There is only one global arc for each sherd whose cost is the cost of

matching the sherd surface data to the global vessel axis/profile-curve model. The total matching

cost of the configuration, cost(∪i∈SDi|θ), is then the sum of all local and global arcs defined in the

graph GS (2.21).

cost(GS |θ) =∑

local arcs (i,j)

cost(Bi ∪ Bj|θl) +

∑

s∈S

cost(Gs|θg) (2.21)

For a puzzle involving a set of sherds known to come from a single axially symmetric vessel the


2.4. THE SEARCH 57

Figure 2.16: A graph representation of a pairwise configuration between sherds A and B whichhave been matched along break curves I and J respectively. Extra un-connected line segments onnodes A and B denote break-curve segments which have not been matched. Hence, for an arbitrary

configuration sherd A may have up to 8 arcs and B may have up to 4 arcs. cost(Bi ∪Bj|θl) denotes

the cost of matching break-curve segment-data for segments I and J to a common unknown break-segment on the vessel β. cost(GA|θ

g) and cost(GB |θ

g) denote the costs associated with matching

sherds A and B, respectively, to the global axis/profile-curve model.

puzzle solution or goal state is assumed to be the graph of lowest cost which connects all of the

graph nodes. This graph is assumed to be a member of the set of all possible graphs which is also

referred to as the solution space of the search. For puzzle assembly, the solution space consists of

all possible relative break-curve segment alignments of the complete set of sherds, i.e., the set of all

possible graphs created from all the nodes, or equivalently, all the sherd data :Sall .

The proposed system addresses the more general situation where sherds from more than one

vessel may be included in the set of sherds to be assembled. Here the solution space is made more

complex since one must consider all possible subsets of the submitted set of sherds as possible

solutions. Equivalently, the solution space consists of all possible graphs consisting of all subsets

of the submitted sherd set. Hence, the goal state of the search is also unknown. This makes the

presentation of solutions problematic. The system allows the user to examine the most probable

solutions for of all puzzle sizes < N . Hence, the user ultimately decides which puzzle solutions are

best and the system simply computes the best solutions for all sizes ≤ N .

The system is not given any prior information about the nature of the puzzle solution. Such

search problems are referred to as an uninformed or blind search in artificial intelligence (AI) lit-

erature [60, 88]. There are two general approaches to finding the best solution for an uninformed

search problem 1) depth-first search and 2) breadth-first search. If there are infinite time and space

available, either algorithm will find the goal state, i.e., the graph of size i of minimal total cost where


2.4. THE SEARCH 58

i = [2, 3, ..., N ]. However, since the search space of a modest sherd set is huge, the computational

space and time considerations become extremely important.

The system uses a search similar to the breadth-first search algorithm referred to as a uniform

cost search. This search algorithm attempts to construct the most probable configuration given

the collection of all previously considered configurations. The probability of new configuration is

approximated using the costs computed from previous configurations to determine best candidate

configuration at any point in the search (see §2.4.5 for details on this approximation).

2.4.2 Treatment of the assembly primitives : Pairwise Hypotheses

Sherd pairs are used as the basic building block of the assembly algorithm. To estimate parameters

for a sherd pair, we hypothesize that two distinct sherds say sherds (i, j) share common break seg-

ments (m,n) respectively. Based on the model from §2.2.3, two things are known if this hypothesis

is true : (1) the break-point data segments are noisy measurements of a portion of the vessel global

break curve parameters β, and (2) the sherd surface data lies on an axially symmetric surface defined

by the pot axis, l, and a portion of the global profile curve α. To estimate these parameters, we ar-

bitrarily assign the coordinate system of one of the two sherds, say sherd i, as the global coordinate

system, hence Ti = (t = (0, 0, 0),R = I). We then estimate Tj , the best alignment transforma-

tion between sherds i and j given that break-point segments m and n correspond, by computing

the maximum likelihood estimates of (Tj , l,α,β) given the measurement data for sherds i and j.

Toward this end, consider the joint distribution of the data given the parameters shown in (2.22).

P(Gi,Gj ,Bm,Bn|l,α,β,Tj) (2.22)

In (2.22) variables Gi and Gj denote the measured surface data for the sherds i and j respectively.

Bm and Bn denote the break-point segments (m,n) that are being used in matching sherds i and j.

Since the noises for the surface-point and break-point measurements are statistically independent,

(2.22) can be written as (2.23).

P(Gi|l,α)P(Bm|β)P(Gj |l,α,Tj)P(Bn|β,Tj) (2.23)

P(Gj |l,α,Tj) denotes the probability of the data set T(Gj), i.e., Gj Euclidean transformed by


2.4. THE SEARCH 59

(a) (b) (c)

(d) (e) (f)

Figure 2.17: Incorrect matches (configurations) of high probability : Two configurations are shownwhich are incorrect. However the break-curve matches shown in (a,b) and (d,e) match well and thecorresponding axis/profile-curves in (c) and (f) indicate that these are reasonable pot configurations.These are among the top 5 most probable pairs for the configuration shown in (a,b) and among thetop 5 most probable tripets for the configuration shown in (d,e) construct by the assembly system.(c) and (f) are the profile curves for configurations shown in (a,b) and (d,e), respectively. Thethe measured surface points of the sherds in (a) and (d) are projected into the (r, z) plane and areplotted as grey dots in (c) and (f) respectivly. The points are scattered in the vicinity of the estimatedprofile curve (in black) in the (r, z) plane. Although these configurations are known to be incorrectthe computed configurations these figures show that both satisfy the requirements of an axiallysymmetric surface since their surface points lie very close to the estimated profile curve in bothcases.


2.4. THE SEARCH 60

(a) (b) (c)

Figure 2.18: A correctly constructed configuration of 4 sherds. Figure (a) shows a view where thevessel axis defines the viewpoint of the camera. Sherds are shown in different colors and are labeledwith their assigned puzzle indices. Figure (b) shows a view taken from the side of the vessel, thematched break-point segments and their corresponding surface normals in figures (a,b) are indicatedwith spheres and cylinders respectively. Note that we are using a subset of the available break-pointsegments to align the sherds. Different color spheres indicate break-point data from different sherdsin the configuration. Figure (c) illustrates the resulting 4 node graph and the cost of the local arcsgenerated by each of the 3 pairwise break-curve matches.

Tj , given the 3D surface parameterized by l and α. Similarly, P(Bn|β,Tj) is the probability of

break-point data segments Tj(Bn) which are measurements after their transformation by Tj , given

break-point parameters β. Hence, we desire (l, α, β, Tj) which are values of these parameters

for which (2.23) is maximum. Unfortunately, this is a nonlinear problem and is computationally

expensive and possibly prohibitively so for a large number of sherd pairs and many alignments for

each pair (see §2.2 for details on the computational cost). Hence, the systems solves a simpler

problem (2.24) which corresponds to computing the MLE of a projection of the higher-dimensional

joint distribution from (2.22).

Tj = arg maxTj

ln (P (Bm|β)P (Bn|β,Tj)) (2.24)

Given the noise assumptions for the break segment points and normals from §2.2.3, the MLE so-

lution to this problem is the optimization problem (2.13) where λ is a constant from the data noise


2.4. THE SEARCH 61

pdfs specified in §2.2.3. Unlike maximizing (2.23), equation (2.13) is a linear least-squares problem

and has an explicit solution which may be computed at little cost, for details see [35, 63, 83]. The

system then computes the most probable value of the parameters given the transformation Tj by

solving (2.25).

l, α, β = arg maxl,α,β

ln(P

(Di,Dj ,Bm,Bn|l,α,β, Tj

))(2.25)

Why is this a reasonably useful result? In matching two break-point data segments, one for each

sherd, the system is not only computing the distances between pairs of break-point measurements,

one of each sherd, but it is also comparing the measured surface-normals at these points. Hence,

some surface measurement information is involved in the estimate Tj . This makes Tj a useful

approximation to Tj and l, α, β useful approximations to l, α, β, respectively.

The pair is then assigned a preliminary cost called the match cost which is the negative log-

likelihood of the aligned sherd data given the estimated pot parameters as shown in (2.26).

ε(i,j),(m,n) = − ln“P

“Gi,Bm|el, eα, eβ

”P

“Gj ,Bn|el, eα, eβ, eTj

””(2.26)

It is assumed that the match cost is a unique value associated with matching sherd (i, j) along

break segments (m,n) (see §2.4.4 for justification). In summary, although the quick solution is

sub-optimal, it exploits the computational speed of solving (2.13) to filter out matches which are

obviously wrong. Additionally, match costs are necessary for detecting configurations of sherds

which contain the same set of matches (see §2.4.4). Eventually, the MLE solution to (2.22) is

computed (see §2.2), but only when the pair is chosen for assembly into a larger configuration.

2.4.3 A word on multiscale boundary matching

Since pairs of sherds may match along short or long break segments there is a need for multi-scale

boundary curve matching. Let the parameter K denote the distance from a sherd vertex to the last

break point in a break-point segment, i.e., it indirectly defines the overall length of the break-point

segment scale (to see why refer to our break-point segment parameterization shown in Fig. 2.11 in

§2.2.2). Let s = (1, 2, . . . , S) define a set of break-point segment scales, where the break segment

defined for scale s has elements Bv,s = ((p1,p2, . . .psK) , (n1,n2, . . . ,nsK)), where v denotes the


2.4. THE SEARCH 62

break segment index and s denotes the scale. The system computes match costs for all pairs of sherd

break segments at all scales and stores the best set of break parameters for each pair. Our criterion

for determining the best break-segment match is the match whose MLE given the break-segment

data is most probable. Our assumptions from §2.2.3 specify that the MLE of highest probability

corresponds to the point set with minimal average distance between corresponding points.

A heuristic has been added to favor matching long break-curve segments. The problem here is

that for two different scales s1 and s2 , s1 < s2, the error in the longer break-segment match is

likely to be larger and hence less probable. Reasons for this are numerous, two dominant causes of

this phenomenon are :

• Longer break segments involve more points and surface normals, any one of these points or

normals could be generated by chipping or erosion which violates our assumptions from

• Break segments may have slightly different parameterizations due to error in the vertex loca-

tions, this generates errors in the matched points.

Why does this heuristic make sense? For one reason, the heuristic compensates for the fact that the

vertex locations and break curve noise models are imperfect. A second problem is that the MLE es-

timate of longer break curves with more complex shapes generally have higher costs than matching

the same curves at smaller scales. However, matches at larger scales lead to better estimates of the

relative sherd transformations and improved global axis/profile-curve models. The heuristic reflects

a preference for obtaining an accurate global model instead of accurate break segment alignments.

This preference is reasonable since an accurate global model of the original vessel is the end-goal

of the system, i.e., discovering the shape of the unknown vessel, estimates of the break segments

are considered less important.

Since the approximate method is very fast, the system can do this exhaustively even for many

pieces and at several different scales. For example, assume that there are 50 sherds and that

on average each sherd has 4 vertices and there are 3 different matching scales. This results in

approximately

50

2

∗ 82 ∗ 3 = 235200 comparisons which require approximately 4 minutes on

contemporary CPUs which can solve (2.13) for K = 5 in less than 1ms.


2.4. THE SEARCH 63

Each assembled sherd group is assigned a configuration cost (2.18) which represents the prob-

ability of the configuration given the connected, i.e. hypothesized subset, of break segment corre-

spondences and the sherd surface data. This cost is used to determine the order of comparisons for

the search algorithm (see § 2.4.6). Note that the expression (2.18) does not depend on the order of

the sherd data sets. This means that configurations built from the same set of sherds and break-curve

segments are indistinguishable, e.g., the triplet h1 = A,B,C, is identical to any permutation of

that set, e.g., h2 = B,A,C. Although one may generate the same configuration of three pieces

several different ways, the resulting solution to all of these is theoretically identical. However, since

each match requires a non-linear minimization which may encounter a local minima this may not

be true in practice. The system takes the first constructed instance of any configuration as the best.

The reasoning for this follows from the chosen search method. Since the search method constructs

configurations in order of their probability, the first assembled instance of any configuration uses the

best fitted estimate of the vessel parameters. A convention is used to detect all subsequent instances

of this configuration and prevent their computation (see §2.4.4).

2.4.4 Detecting Duplicate Configurations

Permuted correspondence hypotheses are simply different sequences of constructing the same con-

figuration by matching the same break segments in a different order. Instances of duplicate hypoth-

esis sets are detected trivially using the match costs computed in §2.4.2. Since each match cost

represents the value of a continuous random variable, i.e., the exponent of the pdf from (2.25), the

probability that two match costs are the same is simply 0, which allows us to index a specific break-

curve segment match with a unique match cost. Hence, sets of break-curve segment matches are

simply represented as sums of match costs which also have this uniqueness property. All permuta-

tions of a set of hypotheses add to the same constant. Hence, when the system considers merging

a sherd with a configuration it computes the match cost of the new set of hypotheses and simply

checks if the match cost is a member of the set of previously computed match costs, i.e., ”Have

I already constructed a permutation of this hypothesis?”. If so, then the hypothesized match is

discarded, else, the system constructs the new configuration.


2.4. THE SEARCH 64

2.4.5 Controlling the complexity

The complexity of the typical uniform cost search is known to have complexity O(bd) in both space

and time, where b denotes the branching factor, i.e., the maximum number of unique arcs which

may connect any two nodes in the graph, and d denotes the depth, i.e., the number of arcs contained

in a solution graph. If all of the solutions can be stored and there is unlimited amount of time to

achieve the goal, the uniform cost search is complete, i.e., the goal state will be found if it exists,

and optimal, i.e., the first configuration of depth d considered is the best solution. For our puzzle

assembly problem, the number of unique arcs between any two nodes is the product of the number

of break-curve segments on each of the two pieces, i.e., for a sherd with 8 break-curve segments

and a sherd with 10 break curve segments the branching factor is b = 80. The depth of the search is

simply the number of sherds in the puzzle, i.e., for a 15 sherd puzzle d = 15.

The complexity estimate makes two assumptions which are invalid for the chosen method of

puzzle assembly : (1) a single initial node is provided as the search starting point, and (2) arc costs

are exactly known between graph nodes. When the initial node is not known then the complexity

of the search is larger and when the arc costs change, the search is no longer optimal. For the prob-

lem of puzzle assembly, arcs have costs associated with the estimated MLE of the vessel geometric

parameters. The value of the computed MLE changes as additional nodes are added and conse-

quently the arc costs also change. Hence, the proposed search method is an approximate uniform

cost search, it is approximate in the sense that the system does not know the exact cost of adding

any sherd and the cost of any arc is never exactly known, i.e., arc costs may (and do) change after

adding a sherd. The predicted cost of adding a new sherd i to a sherd j which is part of a larger

configuration, referred to as ConfigN , is approximated by (2.27). At any given time, the system

uses (2.27) to construct the configuration of minimum predicted cost.

cost(∪m∈ConfigNDm,Di) = cost(∪m∈ConfigNDm|θConfigN ) + cost(Di ∪ Dj |θ(i,j)) − cost(Gj |θg

(i,j))

(2.27)

Where θ(i,j) denotes the MLE of the pairwise configuration of sherds (i, j) and cost(Gi|θg

(i,j))

denotes the redundant global costs of the sherd j surface data which is common to both configura-

tions (see Fig. 2.19). New configurations are only constructed by combining a compatible pairwise


2.4. THE SEARCH 65

Figure 2.19: A merge between two pairwise configurations to create a triplet configuration. Notethat the two configurations have a common global arc cost(GB |θ

g). The system predicts the ex-

pected cost of adding the sherd C to the configuration by adding the cost of the two configurationsand subtracting the redundant cost cost(GB |θ

g) from the smaller of the two configurations. For

merging pairs, the lower cost of the two pairs is used as the value for cost(GB |θg).

configuration with the current configuration of lowest cost. The term compatible denotes that the

current configuration and the pairwise candidate configuration share exactly one common sherd and

no common matched break-curve segments. This convention allows the system to construct new

configurations by applying the solution to the merge problem shown graphically in Fig. 2.19.

In the case of assembling a single vessel consisting of 10 pieces, each of which have 10 break

curve segments, the complexity of the standard uniform cost search becomes O(10010). It is easy

to see that in practice, the solution space cannot be stored for even simple puzzle problems. Hence,

the system limits the complexity of the search in order to compute a sub-optimal solution using the

computational resources available on typical contemporary computers. It does this by restricting

the number of configurations assembled for each size. A set of constants Q = Q2, Q3, ....QN

specifies number of stored solutions where Qn specifies the number of stored configurations having

n sherds. By specifying the maximum number of allowed configurations for all sizes, the search

complexity is limited. Specifically, a constant Qi determines the complexity of creating configura-

tions of size i. For example, consider the following constants for a 3 sherd puzzle Q = 10000, 1.

In this case the search for the triplet sherd is completed very quickly and requires comparisons be-


2.4. THE SEARCH 66

tween very few sherd pairs. However, consider Q = 10000, 100000 for the same puzzle. Here

the search for the best 100000 triplets is much more complex than solving the only the single best

triplet. Ultimately, due to the nature of the uniform cost search, very large datasets (i.e., in ex-

cess of 100 sherds), will require too much time to process and may exceed the time/space of the

available computing resources. Assembly of sherd configurations of this size requires that the large

dataset be manually broken down into smaller assembly problems and then the results of the smaller

assemblies must be integrated to generate solutions for larger numbers of sherds.

As the search proceeds, configurations occupy the allotted space for solutions. In theory, since

configurations are created in order of their cost, the first Qn configurations of size n correspond to

the best available solutions and therefore no further computation on configurations of size n− 1 is

necessary. However, since the cost of all arcs in a configuration graph may change when a new sherd

is added, this property is only approximately satisfied. Hence, we continue to compute solutions of

size n until the expected cost of the resulting configuration is significantly larger than the maximum

cost of the stored configurations of size n. This allows the system to avoid computation on a large

number of configurations which, after adding a sherd, will generate configurations that cannot be

stored.

The computational cost of the search not only depends upon the alloted space for solutions

Q, but also depends upon the distribution of the observed costs at any one level of assembly. For

example, consider 2 puzzles with the same number of sherds. In puzzle 1, suppose that the pair

configuration costs are distributed such that 10 sherd pairs have very similar and low costs and

the remaining sherd pairs have costs which are at least an order of magnitude larger. In this case,

assembled triplets would consist mostly or entirely from matches made from these 10 sherd pairs.

In puzzle 2, suppose that the pair configuration costs are distributed such that 1000 sherd pairs

have very similar and low costs. In this case, assembled triplets would consists of the best matches

generated from the 1000 sherd pairs. Here we see that the computational cost of generating the best

triplet is vastly larger for puzzle 2 than in puzzle 1 although the number of sherds and their number

of associated break curves are equal. This means that computational cost of creating solutions is

related to the distribution of the observed matching costs which will change for different sets of

sherds. Intuitively, the searching algorithm spends more time in areas of uncertainty, i.e., when

there are a large set of candidate matches that are equally probable, or equivalently have similar


2.4. THE SEARCH 67

costs. On the other hand, less time is spent where there are only a few highly probable candidate

matches.

Currently, the constants Q are specified to the system by the user or are assigned a set of appro-

priate default values. The default values depend upon the size of computer memory available to the

system but follow the following rules of thumb : 1) assign a large number for Q2 to ensure that the

true puzzle solution can be constructed from the stored pairwise solutions, 2) the number of triplets,

quadruplets, etc. is then drastically smaller than Q2 but increases slowly to allow for backtracking

due to the numerous false positives present in the pairwise matches (see Fig. 2.17). Work exists

which suggest automatic methods for computing the constants Qi as the search proceeds using the

observed histograms of configuration costs for configurations of size j < i [28, 42]. This may prove

useful for generating more efficient and robust searches of the solution space given an arbitrary set

of sherds.

2.4.6 Pot Assembly Search Algorithm

The sherds may only describe a single or several small portions of the overall vessel. Therefore,

the pot assembly algorithm must accommodate for situations where some sherds may be missing or

sherds come from a number of different vessels. To this end, a MLE-based algorithmic search is pro-

posed in order to robustly perform pot assembly. The algorithm examines significant configurations

of sherds according to the method described in §2.2. Here and afterwards significant denotes those

configurations whose joint-data probability, or equivalently, the total graph cost (2.21), represents a

possible (i.e. not-improbable) solution. We proceed as follows:

1. Estimate the axis/profile-curve for each of the sherds §2.3.2 (Computationally fast).

2. For each pair of sherds, compute all reasonable alignments of their break segments §2.4.2 (at

all scales § 2.4.3) (Computationally fast).

3. For each significant configuration, improve the alignment using §2.2 and store configurations

and individual sherds in order of increasing cost in a stack (Computationally of medium cost

at present).

4. Starting with the top item in the stack, go down through the stack and merge the configuration


2.4. THE SEARCH 68

with those lower in the stack that result in roughly lowest cost configurations according to

§2.2. Update the stack. Note, a sherd can appear only once in a configuration, though the

same sherd can appear in many configurations. Return to step 3 or stop (Computationally of

medium cost at present).

Note, this search proceeds along contours of constant configuration probability to find the most

probable virtual pot for all sizes up to N . Hence realizations of high probability will perpetuate

through the search algorithm whereas improbable geometries will eventually not be considered. In

practice, the number of configurations which must be considered grows exponentially with the con-

figuration size. To alleviate this problem, the system limits the number of configurations considered

for each size which makes the search problem proceed faster. For the results shown in figure 2.20,

the number of allowed configurations, Q, of size N is Q = 60, 10, 20, 20, 20, 20, 50, 80, 100 for

N = 2, 3, 4, . . . , 10, respectively. This allows for correct assembly of a 10 sherd pot in 1.75

hours and the search is completed in less than 5 hours.

2.4.7 Analysis of the Solution

The solution presented in Fig. 2.20 and Fig. 2.21 was further examined to determine how the

bootstrap estimates of sherd axis parameters obtained from the individual pieces in §2.1.5 relate

to the MLE axis of the 10 piece solution. For this analysis, the axis/profile-curve estimate for the

assembled pot is treated as truth and we are interested in the probability distribution of the bootstrap

axis/profile-curve for each sherd with respect to this global “truth.” Towards this end, consider the

sherd axis/profile-curve for a sherd positioned in the estimated global pot model. The goal of this

analysis is to understand the uncertainty of the sherd axis estimate with respect to the global “truth.”

To this end, the bootstrap method was applied as discussed in §2.1.5 in order to obtain ap-

proximate distributions for the axis parameters. For each sherd, all of the bootstrap samples, con-

sisting of the 4-dimensional parameter vector (mx,my, bx, by) were converted to a 3-dimensional

point, b = (bx, by, 0), and a 3-dimensional unit vector in the direction of the estimated axis,

m =(mx,my ,1)

‖m‖ . Using the N − 1 estimated transformations from the solution, the bootstrapped

parameters (m,b) were transformed to lie within the coordinate system of the global solution, i.e.,

b′

= Tb and m′

= Tm. Note that the transformations use the notation of §2.2.2 to specify how


2.4. THE SEARCH 69

Figure 2.20: A correctly assembled pot of 13 sherds where only the 10 matched sherds shown wereavailable. Each sherd has been labeled with a sherd number to aid in the interpretation of the tablesand plots provided in §2.4.7.


2.4. THE SEARCH 70

Figure 2.21: Another view of the correct 10 piece assembly and its axis/profile-curve pair.


2.4. THE SEARCH 71

sherd 1 2 3 4 5σr(mm.) 12.918129 7.746765 1.2533e+05 5.584628 2.6435e+04σθ(radians) 0.011000 0.020632 0.032096 0.032042 0.36065

sherd 6 7 8 9 10σr (mm.) 0.2334567 33.509230 1.544730 182.36284 1.338422σθ (radians) 0.0015407 0.031060 0.023963 0.12190 0.032019

Table 2.2: This table contains the distribution parameters r and θ estimated by applying the boot-strap method to the the 10 sherds presented in the solution shown in fig. 2.20 (see the discussionbelow for an explanation).

the point b and the vector m transform.

Two distributions were obtained for each of the 10 sherds : (1) the distribution of the radius

between the bootstrapped axis estimates and the sherd MLE axis and (2) the distribution of the angle

between the bootstrapped axis estimates and the sherd MLE axis. Radius values are computed as

the length of the vector passing through the sherd MLE axis and the bootstrap estimate b orthogonal

to the axis. Angle values are computed as the arc cosine of the angle between the unit vector in the

direction of the MLE axis and the bootstrapped unit vector m. Hence, for each sherd a distribution

is obtained for radius, r, and angle, θ. For simplicity, the distributions generated are assumed to be

normal and the distribution for r is assumed to be independent of that for θ. This is a reasonable

assumption that is supported empirically by direct inspection of the (r, θ) covariance matrices. The

resulting standard deviations for the scalar distributions of r and θ for each of the 10 sherds is shown

in Table 2.2. The mean value for each distribution is taken as the axis estimate obtained using each

of the measured data points once, i.e., the result of a typical axis estimation. Note that sherds 5

and 9 have large standard deviations for both r and θ which reflects a large degree of uncertainty in

their axis estimates due to the fact that their geometry is close to spherical. On the other extreme,

sherd 6 has an very small standard deviation, this is a large sherd from the pot base where accurate

estimation of the axis is possible.

In fig. 2.22 the distributions of each of the random variables from Table 2.2 has been converted

to a standard normal random variable. The global axis estimate for the 10 piece solution is plot-

ted on the standard normal curve for each of the bootstrapped distributions. A table is provided

immediately below indicating the location of the global MLE axis with respect to each of the 10


2.4. THE SEARCH 72

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

-6 -4 -2 0 2 4 6 8 10 12 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

-6 -4 -2 0 2 4 6 8

(a) radius (b) angle

sherd 1 2 3 4 5radius -2.875448 -0.968033 -0.153619 -0.165727 -0.208602angle -1.424248 0.291058 1.926485 -0.087036 0.101607

sherd 6 7 8 9 10

radius 10.666067 -2.646050 0.320419 -0.463090 1.576709angle 7.345754 -0.620436 1.449499 -1.227619 0.252705

Figure 2.22: Figs. (a) and (b) depict the location of the global pot axis with respect to normaldistributions for each of the 10 sherd axes obtained via bootstrap methods. The distribution of eachsherd has been normalized such that all sherds have a standard normal distribution, i.e., N ∼ (µ =0, σ = 1). The location of the global axis for each of the 10 sherds is indicated by impulses withinthe shown standard normal distributions. Fig. (a) shows the distance between the global axis andeach sherd axis estimate. Fig. (b) shows the angle between the global axis and each sherd axisestimate (see text for more detailed discussion).


2.4. THE SEARCH 73

sherd 1 2 3 4 5radius (mm.) 3.418678 5.083485 5.513502 4.552377 21.610805

angle (radians) 0.038015 0.043757 0.110435 0.029032 0.617846

sherd 6 7 8 9 10radius (mm.) 2.995324 2.146782 1.586111 18.358024 3.500830

angle (radians) 0.014008 0.039639 0.046932 0.180691 0.025214

Table 2.3: Absolute distances and angles between the MLE axis and the mean estimates for each ofthe 10 sherds presented in the solution from fig. 2.20.

estimated bootstrap distributions. Note that our computation of the MLE axis solution does not use

the estimated bootstrap distributions for each of the assembled sherds and therefore outliers such as

the plotted points for sherd 6 in (a) and (b) are not surprising given its extremely small estimated

standard deviation from Table 2.2. Cases such as this show that an axis/profile-curve for a sherd

may be extremely accurate, but the profile curve may not fit the estimated global profile curve for

the entire pot accurately. This can happen if the sherd is not positioned accurately in the global

pot model. Hence, this must be considered when examining the accuracy of the estimated sherd

axis/profile-curve with respect to the sherd true axis/profile-curve.

Table 2.3 shows a plot of the absolute distances between the 10 individual sherd axis estimates

and the axis of the final 10 piece solution. Note that the relative errors between sherds and the

axis are large for those sherds which have unreliable axis estimates such as sherds 5 and 9. The

average distance error between the estimated sherd axes and the global pot axis for the remaining

sherds (excluding 5 and 9) is 3.6 mm with a standard deviation of 1.38 mm.. The average angular

error between the estimated sherd axes and the global pot axis for the remaining sherds is 0.043379

radians or equivalently 2.5 degrees with a standard deviation of 0.029156 radians or equivalently

1.6705 degrees. The complete pot has a diameter of approximately 200 mm. at it’s rim and is

approximately 180 mm. tall in overall height.

2.4.8 Conclusion

A method for searching the vast solution space of the puzzle assembly problem is proposed. It

uses the probability of computed MLEs for sherd configurations to construct the most probable

hypothesis at any given time which has not already been considered. This method of puzzle as-


2.4. THE SEARCH 74

sembly corresponds to a constant probability search of the solution space which is known to re-

quire vast amounts of time and space in order to obtain the optimal solution. Several modifications

have been made which cause the resulting search to be sub-optimal, yet computationally tractable.

These include a quick approximate clustering approach for identifying an initial set of plausible

sherd matches, a method for eliminating the computation of duplicate or permuted hypothesis sets,

heuristics which favor curves of longer length, strictly limiting the number of configurations stored,

and ignoring configurations which, when built upon, will generate configurations which cannot be

stored.


Chapter 3

Stochastic 3D Surface Models

3.1 Markov Random Fields for 3D Surfaces

3.1.1 Introduction

A shape model is specified on an initial 3D mesh which consists of vertices, points which lie on the

original shape, and edges, connections between vertices. An MRF site is assigned to each vertex and

a set of cliques, which consist of subsets of sites, are defined. Clique members and the associated

clique potentials which operate on cliques must be learned or specified by the system designer.

They represent application-specific information regarding the desired deformation behavior. These

cliques and clique energies can be anisotropic, i.e., can be different in different directions and can

be inhomogeneous, i.e., can vary over the surface.

The complete deformation model is expressed in two parts: 1) a model for the probability

density of the surface which is based on cliques involving only MRF sites, i.e., surface vertices; and

2) a conditional probability density for the newly observed data given the surface. The probability

density for the surface specifies a functional relationship between MRF sites, or equivalently, surface

points. The conditional density for the data specifies a functional relationship between MRF sites

and the measurements. Deformed surfaces are sample functions from the joint probability density

and correspond to Maximum A-posteriori Probability (MAP) estimates of site random variables

given the joint probability of the MRF sites and the data. The notion of surface deformation as

MAP estimation using the vertices as site variables is new and its utility and power for specifying

75

3.1. MARKOV RANDOM FIELDS FOR 3D SURFACES 76

surface deformation models is demonstrated. A simple clique potential is coupled with a completely

new data model to generate a deformation model useful for the interpolation of data.

The thesis extends the stochastic deformation model to deal with sequences of data sets provided

at different time instances. In this case, the model becomes a time-varying stochastic process where

the system state can in theory consist of the surface vertices at a succession of two or more instants

(i.e., cycles). It is this fully 3D stochastic representation and modification of surfaces, and the

ability to incorporate unlimited local and global deformation properties (either physically realizable

or physically unrealizable) that is completely new, tremendously powerful and computationally fast

(real time on a PC).

3.1.2 Surface Model

The model requires an initial polyhedral shape, e.g., a triangular surface mesh, which consists of N

vertices. Associated with vertex pi is a site i. Hence, the MRF sites are labelled 1, 2, . . . , N where

the ith site has random variables pi = (xi, yi, zi). The vector P ≡ p1,p2, . . . ,pN denotes all site

variables. The initial surface is referred to as the set of vertices at time t = 0 and indicate the time

with a superscript, e.g., P0 denotes the vertices of the initial surface and Pt ≡pt

0,pt1, . . . ,p

tN

denotes the surface vertices at time t.

A clique is a subset of indices into the set of MRF sites, e.g., 1, 5, 3, and has an associated

clique potential function defined on the clique site variables V3(p1,p5,p3). Pc denotes the set of

site variables, i.e., vertices, in clique c. For our example, Pc = p1,p5,p3. Cliques and their

associated potential functions are distinguished by the cardinality of the set they define, i.e., single-

vertex cliques, two-vertex cliques, three-vertex cliques, etc. We denote the sets of such cliques and

clique potentials as C1, C2, C3, . . ., and V1, V2, V3, . . ., respectively, and C =⋃

iCi. For the simple

mesh with indices shown in Figure (3.1), an example set of clique structures for site i = 8 and its

neighbors are shown to illustrate the correspondence of clique definitions to their associated mesh

vertices.

Cliques and clique potentials determine the forces among MRF site variables, i.e., the surface

material behavior. The total energy of the MRF is described in terms of the sum of the defined

clique potentials over all cliques (see §3.1.2).



C1 = 8,C2 = (8, 1), (8, 2),

C3 = (2, 8, 1), (2, 8, 3)

Figure 3.1: A example clique structures C1, C2, C3.

U(P|P0) =∑

c∈C

Vc(Pc|P0) (3.1)

In general, potential functions are tailored to applications.

The surface probability distribution is modeled as a Gibbs Distribution which is a representation

for an MRF. The Gibbs pdf pS(P|α) is specified as (3.2) (see §3.1.2).

pS(P|α,P0) =1

Zexp

−U(P|α,P0)

(3.2)

Surface Model Example

A very simple surface model has the following cliques containing the ith site: C1 = i and C2 =

(i, j1), (i, j2), (i, j3), . . . where jk denotes the label of a site in a clique with site i, and the jk

sites are mesh neighbors of site i, i.e. are connected by a mesh edge. The site clique potentials

are V1(pi) = α1‖pi − p0i ‖

2 and V2(pi,pj) = α2‖pi − p0i − pj + p0

j‖2. Here the parameters

are the same for all sites. The potential V1 is the squared distance from the deformed vertex pi to



the initial vertex p0i and is a function of this local deformation from the surface. V2 is the squared

distance between the surface deformations at sites i and j and is a function of the relative positions

of vertices at sites i and j before and after the deformation. α1 and α2 are weighting coefficients

controlling the contributions of V1 and V2 to the total MRF energy. The vector of clique potential

coefficients is α, here α = (α1, α2).

3.1.3 Data Model

The data is modeled by (3.3).

pD|S(D|P,β) (3.3)

This is a probability density function for the input data vector D given the surface shape P and

parameters β. Here, the components of D ≡ d1,d2, . . . ,dM are the data points and β is a

vector of parameters having values specified by the user.

Data Model Example

There are a number of data distribution models that come to mind as appropriate. Ideally, the data

should lie on the surface we wish to represent. However, because of noise in the data generation

process, each data point is a noisy perturbation of a point on the desired surface: but a perturbation

of which surface point? The approach taken is similar to that from [14] which assumes an a-

priori distribution for N′

points on the surface from which the data point could have occurred.

If the perturbation distribution is isotropic Gaussian having 3x3 covariance matrix σ2I, then the

conditional pdf of data-point dm given the surface S is equation (3.4).

p(dm|P) =1

N ′

∑

i∈N′

1

(2π)32 |σ2I|

12

exp

−

1

2σ2‖dm − pi‖

2

(3.4)

Note that p(dm|P) is then a mixture of N′

Gaussians, but it is only the Gaussians located at the

vertices that are near dm that contribute significantly to this conditional probability. If it is assumed

that theM data points are generated as independent random vectors, then the conditional probability

p(D|P) is the product of the conditional probabilities for the individual data points, hence, a product



of mixtures of Gaussians. Note, the pdf for data point di is characterized by the two parameters

N′

and σ2 which control the weight to be associated with a point and the extent of the region

on the surface that is attracted by the data point. Hence, in the general framework from §3.1.3,

β =(N

′

, σ2)

.

3.1.4 Deformation

Surface deformations occur by moving the mesh vertices P0 to a modified vertex vector P1 where

P1 is:

P1 = arg maxP

ln(pS(P|α,P0)pD|S(D|P,β)

)(3.5)

Note, pS(P|α,P0)pD|S(D|P,β) is the joint pdf of D and P which is :

pD,S(D,P|α,P0,β) (3.6)

P1, the P that maximizes (3.6), is the so call MAP (maximum a posteriori probability) estimate of

P given the data D. Figure (3.2a) shows a surface mesh obtained from a laser scanner. The mesh

structure is irregular and contains large holes and tears. Figures (3.2b) and (3.2c) show the low

resolution and fine resolution MRF mesh interpolations where the mesh vertices from Fig. (3.2a)

are data and a sphere is the initial MRF shape, P0. The meshes produced do not have holes and are

more regular. The interpolation uses the surface model from §3.1.2 and the data model from §3.1.3.

Optimization and an Example

The joint pdf (3.6) has an energy (3.1) defined for all possible positions of the mesh vertices, i.e.

values which the site (x, y, z) random variables may assume. Optimization corresponds to maximiz-

ing the joint pdf (3.6) by applying any standard non-linear maximization method such as gradient

ascent.

For optimization, one must compute the gradient of (3.5) which is fg(P) = 5P ln(pS(P|α,P0))+

5P ln(pD|S(D|P,β)) where 5P indicates that we are differentiating with respect to the vector of

site variables P. Hence, the clique potentials and data pdf combine to generate a motion vector field

for the MRF sites, fg; this may be interpreted as a generalized force. fg is the resultant of two dis-

tinct forces : (1) 5P ln(pS(P|α,P0)) is the force between MRF sites, which models the intrinsic



(a) (b) (c)

(c) (d) (e)

Figure 3.2: Surface interpolations : A simple deformable MRF is defined using the surface modelfrom §3.1.2 and the data model from §3.1.3. Figure (a) shows a rendered surface mesh obtainedfrom a laser scanner and figure (c) is a rendering of the same mesh where only the mesh edges areshown. The scanned mesh structure is irregular and contains large holes and tears. Figures (b) and(c) show rendered surface meshes for a low resolution and fine resolution MRF interpolation wherethe mesh vertices from Figure (a) are uses as the data and a plane is the initial MRF shape, P0.Figures (d) and (e) show only the mesh edges of the surfaces rendered in (b) and (c). One can seethat the meshes produced by the MRF interpolation do not have holes and are more regular.



surface behavior (e.g., flexibility); (2) 5P ln(pD|S(D|P,β)) is the force exerted on the MRF sites

by the data. Hence, the cliques and their associated clique potentials exert virtual force fields on the

MRF sites.

3.1.5 Time Varying Surface Behaviors

In §3.1.2 the time index for surfaces could assume only two values t = (0, 1), here one simply

allows the index t to vary t ∈ [0,∞) for the site random variables, Pt, the potential weighting

coefficients, αt, and the data, Dt. In this case, (3.7) denotes a time-varying pdf for 3D surfaces, or,

alternatively, can be thought of as a pdf having indexing parameters (i, t) for site location and time.

pS(P|αt,Pt) =1

Zexp

−U(P|αt,Pt

(3.7)

Elasticity & Plasticity

Assume that the surface vertices from the previous time step, Pt−1, remain fixed at the initial surface

i.e., Pt−1 = P0 ∀ t. If the clique potentials for the ith MRF site are all convex quadratic functions

symmetric about pi,0, then pS(P|αt−1,Pt−1) defines a Gaussian distribution on the MRF sites

N ∼ (P0,Σ) where Σ depends on the weighting coefficients αt. The maximum is located at the

mean, hence, arg maxP pS(P|αt,Pt−1) = P0. Note that the clique potentials from §3.1.2 are a

special case of this model where αt = α01, α

02 ∀ t. In this case, the surface will return to its

original vertex positions in the absence of external forces from data. This behavior for the MRF is

analogous to that of an elastic surface deformation for physically-based models.

Using the same Gaussian MRF site distributions, consider a new data set from time t. Then

optimization of (3.8) provides the deformed MAP surface Pt.

Pt = arg maxP

ln(pS(P|αt−1,Pt−1)pD|S(Dt|P,β)

)(3.8)

A deformation is made plastic by updating the mean of the MRF site distributions at the end of the

time step i.e., each MRF site is assigned a new distribution,N ∼ (Pt,Σ). Hence, when the data set

Dt is absent, the MLE surface is the deformed surface Pt. This behavior for the MRF is analogous

to that of a plastic deformation for physically-based models.


3.2. EXAMPLE : 3D SCULPTING 82

3.1.6 Conclusion

A new stochastic model for surface deformation has been proposed. The model specifies a Markov

Random Field (MRF) on an existing surface mesh where MRF sites correspond to mesh vertices.

In this framework 3D surfaces correspond sample functions of the MRF. Deformation corresponds

to MAP estimation of the site random variables given the joint distribution of the surface model and

the measured data. The framework of deformation as MAP on the surface vertices of a 3D mesh

is new. The model may be used on regular uniformly sampled shape models or on models which

have irregular connectivities and non-uniform distribution of vertices. However, in the latter case,

one must be careful to define clique potentials which are numerically stable. A simple model is

proposed which includes a new data model and details are provided on how to compute the solution

given the simple model. The model is demonstrated to perform well for interpolation.

3.2 Example : 3D Sculpting

3.2.1 Introduction

Artists constantly seek new media within which to generate new art forms. The immersive virtual

environment of a computer is one venue which has become popular for generating a new genre

of art referred to as digital art. To this end, systems have been constructed for virtual sculpting.

A typical virtual sculpting system provides the user with a blob of “virtual” clay which the user

molds using a wide variety of tools. Completely new forms may be created in such an environment

since the artists are free to specify the virtual universe to their liking. Typical examples of changes

range from specifying different material properties for the object, i.e., the surface elasticity, or the

properties of the universe, i.e., gravity. Physical instantiations of sculpted objects may be rendered

as a video image or created using a 3D prototyping machine. The usefulness of a virtual sculpting

system is limited by the versatility of the interface and the versatility of the virtual model of both the

universe and the object. The computer graphics community has been working for quite some time

to develop surface sculpting systems. Existing systems provide examples of many different surface

representations and work continues to develop their utility for intuitive sculpting [23, 10, 86, 91]

(see §1.4.2 for a review of currently existing systems for virtual sculpting).



Description c Vc(Pc)

Mean Field 1 ‖pi − pt−1i ‖2

MeanSmoothing

2 ‖pi − pt−1i − pjk

+ pt−1jk

‖2

EdgePreserving

2 ‖pi − pjk‖ − ‖pt−1

i − pt−1jk

‖

SurfaceFlattening∗

K + 1 ‖κ−→v ‖2

CurvaturePreserving∗

K + 1 (κ−→v⊥− 1

KΣK

k=1κ−→v⊥(jk))2

Symmetry(see Fig. 3.4a)

2‖(pi + Apj) · np‖2+‖(pi + Apj) × np‖

2

Table 3.1: Sculpting Clique Potentials.∗For potentials involving curvature, i.e., κ, values are obtained using the method from [75] to obtain principle curvatures

and principle directions: (−→κ1,−→κ2). In general, the notation κ−→v denotes the curvature in the direction of the vector −→v .

3.2.2 Surface Behaviors

A simple virtual sculpting system is proposed as a particular instantiation of the stochastic sur-

face model from §3.1. The basic sculpting system uses three clique structures : C1 = i, C2 =

(i, j1), (i, j2), . . . , (i, jK) and CK+1 = (i, j1, j2, . . . , jK) where jk denotes a mesh neighbor

(see §3.1.2 for clarification) and K denotes the number of sites adjacent that are mesh neighbors to

site i. The clique potentials and their associated cliques are provided in Table (3.1).

Each of the cliques and associated clique potentials in Table (3.1) is assigned a descriptive

phrase symbolizing how the clique potential influences interpolations of sculpted data sets. For

example, the Mean Field description is assigned the clique potential ‖pi − pt−1i ‖2 which increases

as a quadratic function of the displacement of the vertex. Hence, this potential encourages points to

remain at their original or mean position, i.e., pt−1i for the ith vertex. Hence, we can think of each

clique and its associated clique potential as specifying a specific type of force field for the MRF

sites which is identified by a descriptive phrase. For example, we can refer to the mean force field

as those forces which move vertices from their current (x, y, z) position to their original (x, y, z)

position.

The relative strength of each force field type causes the surface to interpolate the sculpted data

differently. Hence, using this force-field analogy, the sculptor can modify force field strengths,



i.e., clique potential function coefficients, to intuitively control the surface. The user controls the

6 potential function coefficients, α = (α1, α2, α3, α4, α5, α6), for each of the rows from Table

(3.1) which, combined with the 2 data model parameters β =(N

′

, σ2)

from §3.1.3, makes 8

user-controlled parameters in total. Their geometric meanings are given in §3.2.3.

The sketched data is provided at different time instances and is consequently a situation where

the time varying MRF models of §3.1.5 are appropriate. The sculptor is empowered to deform the

shape via a single custom built device which can interactively (add/remove) data, control the data

and material parameters, and define primitive cliques. The sculpting process is then described in

terms of cycles. At the beginning of each cycle the sculptor may do any of the following functions:

• Introduce a data set, Dt, to the system for surface deformation with varying data types:

– 3D space curves, i.e., 3D points obtained from 3D tracker integrated into a sculptingpen,

– Directional feature enhancements, i.e., indicate areas to apply anisotropic diffusion,

– A cloud of 3D data points from a template.

• Make the current deformation plastic (see §3.1.5).

• Remove previous datasets.

• Change deformation parameters of the system.

• Change the clique structure of the system.

• Remesh regions of the surface using the method in [40].

At the end of the cycle the system computes a solution to the optimization problem (3.5) as described

in §3.1.4. The system is implemented in Java, and performs in real time for deformations involving

in excess of 5000 vertices.

3.2.3 Sculpting Potentials

For each of the potentials listed in Table (3.1) the time index t−1 refers to the time of the last plastic

deformation specified by the user, see §3.1.5. The following list provides a geometric interpretation

of the potentials listed in Table (3.1) (see §3.2 for Mean Field).



(a) (b) (c) (d)

Figure 3.3: Figures (a-d) show the reconstruction of a nose on a damaged archaeological sculpturediscovered in Petra, Jordan. (a) an image of the archetype nose used for reconstruction (b) aninterpolation of the damaged mask data (c) interactive modification for rebuilding the missing noseusing anisotropic clique potentials and only a few data points shown in blue (d) the reconstructedmask.

Mean Smoothing Strengthening this force will encourage neighboring MRF sites to have similar

displacements from their original position which makes the surface smoothly interpo-

late the data.

Edge Preserving Strengthening this force will encourage the mesh edges to remain fixed which

makes the surface material rigid.

Surface Flattening Strengthening this force encourages the surface region to be linear in the direc-

tion of −→v .

Curvature Preserving Strengthening this force encourages the surface region to preserve curvature,

i.e., mesh neighbors are encouraged to have the same curvature in the direction of −→v⊥.

Symmetry Strengthening this force encourages the surface to be symmetric about a predefined

plane as shown in Fig. (3.4).

3.2.4 The Interface

Development of an intuitive paradigm for 3D surface specification is the goal of the interactive as-

pects of the system. The proposed approach and previous approaches define virtual tools which



(a)

(b) (c)

Figure 3.4: Symmetry Cliques : Spatially distant sites are associated by their height and (x, y)positions relative to a user-defined plane as shown in (a). In (b), the symmetry clique potential fromTable (3.1) is used with A = I to create an original sculpting of a head from an initial capsule shapeas shown in Fig. 1.5(a). In (c), the symmetry clique potential has been changed to be non-isometricscaling : diag(A) = (1.25, 1.25, 1).



Figure 3.5: The LEMS-Pen

serve to deform the virtual shape. The user controls the tool in the virtual world via a real-world

input device. In previous approaches, the typical device used is haptic, i.e., the device seeks to sim-

ulate how the tool would actually feel if in contact with the virtual surface. This is accomplished by

tracking the users position and orientation using an articulated mechanical arm. Motors integrated

into the articulated joints are controlled by the computer generating force-feedback resistance to

the artist’s tool [21, 32, 50]. Interaction of this sort makes a deformation analogy based on surface

contact, i.e., the user makes contact with the surface using the tool and introduces a force which

serves to deform the surface. In contrast, the deformation analogy allows the user to sketch 3D

curves as a series of points in arbitrary relation to the surface, and the surface deforms to interpolate

the sketched 3D curves using a configurable deformation model. The deformation model can be

simple, e.g., a homogeneous behavior modeling how an balloon would deform (see Fig. 3.2), or

more complicated, e.g., the artist wishes to form ridges such as that shown in Fig. 3.3.

3.2.5 The Physical Input Device

A custom-built sculpting tool for the purpose of 3D sculpting was created called the LEMS Sculpting

Tool. This is a two part tool : 1) a multi-sensor hand-held device with a shape similar to an airbrush,

referred to as the LEMS-Pen (see Fig. 3.5), and 2) a box which attaches to the periphery of the

computer screen and includes many different knobs and switches, referred to as the Control-Box

(see Fig. 3.6). The LEMS-Pen is used to specify 3D data to the system and produces six streams of

sensor-data :



1. An embedded Polhemus 6DOF tracker [3] in the handle gives the tool’s position and orienta-

tion

2. Two pressure sensitive buttons on a removable, replaceable tip indicate the “strength” of the

user’s input. The device’s pressure sensitivity gives immediate, intuitive control over the

“weight,” or importance, of each data point entered.

3. Three binary momentary switches — two of which have the following default configuration :

(a) Pressing this button causes the existing sculpture to be saved to the shape history which

saves evolving versions of the sculpture (see §3.2.6).

(b) Pressing this button makes the current deformation plastic (see §3.1.5).

The Control-Box panel consists of the following inputs :

1. Two binary switches — one of which has the following default configuration:

(a) (Start / Stop) deformation of the object. Typically, the surface is constantly deforming

to specified user data. Use of this switch, allows the sculptor may make the object

static while he/she specifies groups of new data. For example, a sculptor wishes to add

structure which can be described with a set of 3 strokes. To do this, the object is held

static while the strokes are sketched and then the switch is toggled to start the surface

interpolation of the new strokes as a single data set.

2. Four rotary knobs — two of which have the following default configuration :

(a) Changes the value σ2 from §3.1.3 which changes the domain of influence for the sculpted

points. Large values of σ2 generate large-scale deformations of the surface and vice-

versa.

(b) Changes the value of N ′ from §3.1.3 which serves to modify the magnitude of the force

of the data on the surface vertices, i.e., it is used to scale the dynamic range of the pen’s

pressure sensitive input.



Figure 3.6: The Control-Box

3. One six-position switch which is used to change the interactive mode of the LEMS-Pen —

this has four default operating modes :

(a) 3D Sketching

Somewhat analogous to drawing or painting in a 3D environment, this mode lets the

user enter "strokes," which are discretely sampled 3D space curves. Each point from a

sketched space curve has the distribution from §3.1.3 and serves to deform the surface.

The resulting deformation depends upon the chosen surface model and is controlled via

the rotary knobs from (2).

(b) Surface Selection

Deformations can be restricted to happen only on specific regions of the surface, which

are selected by the user in this mode. This increases the performance and controllability

of the system.

(c) Direction Specification

In this mode, the user specifies the direction −→v for anisotropic potential functions. This

direction is visualized as a long, blue cone, as in Fig. 3.3.

(d) 3D Cloud Sketching

Here the sculptor selects from a list of different shape templates and the tool shape is

changed to indicate the shape of the chosen template. Each shape template is a collection

of 3D point samples taken from a shape of interest which the sculptor wishes to integrate

into the existing sculpture. The sculpting process using the shape template is similar to



that described in 3D Sketching. However, each sketched point will now introduce a

copy of the shape template with location and orientation defined by the position and

orientation of the LEMS-Pen. The templates can be inserted one-at-a-time, resulting

in a straightforward interpolation between the sculpture and the template shape or they

can be used to define swept surfaces where the sculptor introduces a sequence of the

templates to generate surface structure swept out along the space curve defined by the

path and orientation of the LEMS-Pen.

During normal operation, one hand sketches with the LEMS-Pen, and the other hand operates the

Control-Box. The LEMS-Pen and Control-Box together allow for intuitive sculpting, free from the

encumberment of manipulating the mouse and keyboard. Although the developed system works best

with the custom-built tool, it is also operable by using the keyboard in conjunction with a standard

2D mouse, or the Microscribe 3D mouse [2]. To allow for variation in the used input devices, an

interface is incorporated which allows sculptors to customize the sculpting behavior of any device to

their liking. Device customization is done by re-mapping the device inputs to one or more sculpting

operations. The default mappings of the LEMS Sculpting Tool are given in this section. In general,

the tool configuration depends upon the sculptor’s preferences and the chosen input device.

3.2.6 The sculpting process

The sculpting process is slightly different and depends upon the user’s goal. The system is currently

in use for two slightly different tasks: generative sculpting and reconstructive sculpting.

A typical generative sculpting session begins by choosing an initial shape analogous to a real-

world sculptor choosing a block of clay or marble for instance. In the proposed system, the shape is

typically an existing triangle mesh with a primitive shape that roughly matches the form of the end

goal. Some common initial shapes are a sphere, plane, or cylinder. Large scale deformations are

specified on the primitive initial shape so the sculptor can quickly “rough out” a basic representation

of the goal shape. This is typically done at a low resolution, i.e., the surface representation has few

vertices. Detail is added at higher resolutions, and remeshing, as defined in [40], is used as a vehicle

for moving between resolutions.

In typical reconstructive sculpting sessions, the sculptor starts with a pre-existing and possibly



Figure 3.7: Using the system

complex geometry rather than a primitive shape. Here, the sculptor performs local sculpting oper-

ations on local regions of the object which have incomplete or damaged geometry. This is done by

selecting the damaged surface region and specifying deformations within this region as shown in

Fig. 3.3.

At any point in the process, the sculptor can adjust parameters, add and remove data, and repo-

sition the model using the LEMS-Pen and Control-Box–all without leaving the 3D fishtank VR

environment. This has been done in order to allow the user to feel like they are working with a

material, rather than operating a computer.

Shape History

The shape history provides a record of the evolution of the sculpted shape. Typically shapes

recorded to the shape history are milestone shapes, i.e., shapes having significant newly incorpo-

rated geometry of significance. This functionality is analogous to the history functions provided in

other popular 2D graphics software packages [1, 7], which prove effective in expediting the creative

work flow. A list of previous shapes is available to the user in a window, and selecting a shape in



the list recalls its geometry and connectivity.

3.2.7 Conclusion

A sculpting system for intuitive virtual sculpting has been specified based on the stochastic surface

deformation model from §3.1. Sculpting is performed by sketching free form 3D curves which serve

to deform the surface and deformation is controlled by the MRF parameters for both the surface and

data potential functions which are referred to as virtual force-fields. The cliques and their associated

clique potentials are new contributions which represent some initial work on developing useful

instantiations of the stochastic deformation model from §3.1. While the individual clique potentials

from Table 3.1 are somewhat simple, the MRF model integrates their influences into a complex and

much more powerful deformation model. This allows the user to specify complex deformations

using less data than typically necessary in other sculpting systems as shown in Figs. (3.3) and (3.4).

New tools for interactive virtual sculpting are proposed which increase the usability of the system

and facilitate interactive 3D sketching free from the encumberment of the standard keyboard and

mouse inputs. The power available due to the generality, versatility, and computational simplicity

of the model makes it perfect for building arbitrary free-form shapes where the user provides high

dimensional data to the computer which is interpolated by the MRF model at an interactive rate.


Appendix A

Solving the Absolute Orientation Problem

The absolute orientation problem refers to the problem of determining the transformation which

minimizes the sum of squared distances between two corresponding point sets measured in dif-

ferent coordinate systems. We denote the two point sets x1,x2, ...,xN and y1,y2, ...,yN and the

correspondence assumption between each pair of points requires that xi = Ryi + t for all i where

R denotes a 3D Euclidean transformation and t denotes a 3D translation vector. To align the two

point sets, one seeks to minimize the sum of squared distances between the corresponding points

written mathematically as equation (9).

e(T) =

N∑

i=1

‖yi − (Rxi + t)‖2 (9)

In 1987, Berthold Horn proposed two methods for solving this problem in closed form : one using

orthonormal matrices and one using unit quaternions in his papers [35, 34]. At the same time, Arun

et. al. proposed a solution to the same problem using a least-squares framework in [9]. Given 3D

point data as input, the method using orthonormal matrices was found to be more computationally

efficient and hence became more popular in practice. Work by Umeyama four years later [83],

solved an existing problem in the proposed solution using orthonormal matrices : For data which is

highly corrupted with noise, the method would sometimes fail to give a correct rotation matrix and

give a reflection instead, i.e., det(R) = −1.

The pottery assembly system uses the solution of Umeyama to compute rough initial alignments

between two sherds given a correspondence between their break-point segments (see §2.2.6). In the

context of break-point segment alignment, we seek to align both the corresponding point vectors

93

APPENDIX A 94

and their corresponding associated normal vectors for the matched break-point segments (10).

‖BcN−TN+1(BN+1)‖2 =

K∑

k=1

‖pcN,k−(RN+1pN+1,k+t)‖2+1

λ‖ncN,k−RN+1nN+1,k‖

2 (10)

Note that (10) uses the notation adopted in the thesis. Here, BcN and BN+1 denote the matched

break-point segments for the two different sherds which describe a single unknown break-point

segment on the axially symmetric surface. Each break-point segment consists of measured points

and outer surface normal pairs which are denoted pk and nk respectively for the kth point/normal

pair. The association of the points and normals to their break-point segment is indicated using a pair

of indices, e.g., the index pcN,k denotes the kth point of the cN th break-point segment and normals

are indexed similarly. Transformations of the break-point segments are defined as an operation on

each of the points and normals in the break-point segment : T(BN+1) = Rpk + t for all points

pk ∈ BN+1 and T(BN+1) = Rnk for all normals nk ∈ BN+1.

Due to differences in the assumed noise distributions for the points and normals, this results

in a linear weighted least-squares problem instead of the standard least squares solution originally

discussed in [9]. The procedure for computing the solution is as follows:

1. Translate both point sets to the origin by subtracting their means.

µcN = 1K

∑Kk=1 pcN,k

µN+1 = 1K

∑Kk=1 pN+1,k

pcN,k = pcN,k − µcN

pN+1,k = pN+1,k − µN+1

2. Compute the best rotation matrix R, by finding the orthogonal matrix R which maximizes

the cross terms of the quadratic expression (10) which are shown in (11).

R = maxR

(trace(RtpN+1ptcN,k +

1

λRtnN+1,kn

tcN,k)) (11)

The solution to (11) is found using singular value decomposition where the covariance matrix


APPENDIX A 95

S is expressed as a product of 3 matrices, i.e., S = UDVt. Where VVt = UUt = I and

D = diag(di), d1 ≥ d2 ≥ d3 ≥ 0. The rotation matrix is then :

R = U

1 0 0

0 1 0

0 0 det(UVt)

Vt

The solution for R is unique if the rank of S > 1 and det(UVt) = 1 or if the rank of S > 1

and the minimum singular value is a simple root.

3. The best translation vector t is then computed using (12).

t = µcN − RµN+1(12)

The specific form of the solution shown here is based upon a solution from [6].


Bibliography

[1] Adobe photoshop. http://www.adobe.com/products/photoshop/main.html.

[2] Immersion corp. http://www.immersion.com/digitizer/.

[3] Polhemus corp. http://www.polhemus.com/fastrak.htm.

[4] Shapegrabber inc. http://www.shapegrabber.com.

[5] Virtual Sculpting with Haptic Displacement Maps, May 2002.

[6] Cvonline : The on-line compendium of computer vision, April 2004.

http://homepages.inf.ed.ac.uk/rbf/CVonline.

[7] Gnu image manipulation program version 1.2, January 2004. http://www.gimp.org/.

[8] Y.S. Akgul and C. Kambhamettu. Recovery and Tracking of Continuous 3D Surfaces from

Stereo Data Using a Deformable Dual-Mesh. In Proc. of ICCV, volume 2, pages 765–772,

1999.

[9] K. S. Arun, T. S. Huang, and S.D. Blostein. Least-Squares Fitting of Two 3-D Point Sets.

PAMI, PAMI-9(5):698–700, 1987.

[10] J.A. Baerentzen et al. Volume Sculpting Using the Level-Set Method. In Proc. of SMI, pages

175–182, 2002.

[11] R. Bergevin, M. Soucy, H. Gagnon, and D. Laurendeau. Towards a General Multi-View Reg-

istration Technique. PAMI, 18(5):540–547, 1996.

[12] P. J. Besl and N. D. McKay. A Method for Registration of 3-D Shapes. PAMI, 14(2):239–256,

February 1992.

96

BIBLIOGRAPHY 97

[13] M. Blane et al. The 3L algorithm for fitting implicit polynomial curves and surfaces to data.

IEEE Trans. on Pattern Anal. Machine Intell., 22(3):298–313, March 2000.

[14] R. Bolle and D. B. Cooper. On optimally combining pieces of information, with application

to estimating 3-D complex-object position from range data. IEEE Trans. on Pattern Anal.

Machine Intell., 8(5):619–638, September 1986.

[15] Y. Chen and G. Medioni. Object Modeling by Registration of Multiple Range Images. In

Proc. of the International Conference on Robotics and Automation, pages 2724–2729, April

1991.

[16] R. Cippolla, P. Torr, and G. Vogiatzis. Bayesian Stochastic Mesh Optimization for 3D Recon-

struction. In Proc. of Conf. on British Machine Vision, July 2003.

[17] L.D. Cohen. On Active Contour Models and Balloons. Computer Vision, Graphics and Image

Processing : Image Understanding, 53(2):211–218, 1991.

[18] L.D. Cohen, B. Leroy, and I.L. Herlin. Face Identification by Deformation Measure. In Proc.

of ICPR, page 633, 1996.

[19] W. J. Conover. Practical Nonparametric Statistics. John Wiley & Sons, Inc., 1999.

[20] T. Cormen, C. Leiserson, and R. Rivest. Introduction to Algorithms. MIT Press, 1990.

[21] G. Debunne, M. Desbrun, M. P. Cani, and A. H. Barr. Dynamic Real-Time Deformations

using Space & Time Adaptive Sampling. In Proc. of SIGGRAPH, August 2001.

[22] C. Dorai, G. Wang, A. Jain, and C. Mercer. Registration and Integration of Multiple Object

Views for 3D Model Construction. PAMI, 20(1):83–89, 1998.

[23] H. Du and H. Qin. Direct Manipulation and Interactive Sculpting of PDE Surfaces. Computer

Graphics Forum, 19(3), 2000.

[24] R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley & Sons, 2nd edition, 2001.

[25] E. Ferley, M.P. Cani, and J.D. Gascuel. Resolution Adaptive Volume Sculpting. Graphical

Models (GMOD), 63:459–478, 2002.


BIBLIOGRAPHY 98

[26] A. W. Fitzgibbon. Robust Registration of 2D and 3D Point Sets. In Proc. of British Machine

Vision Conference, pages 411–420, 2001.

[27] H. Freeman and L. Garder. A Pictorial Jigsaw Puzzles: The Computer Solution of a Problem

in Pattern Recognition. IEEE Trans. Electron. Comput., 13:118–127, 1964.

[28] Matthew L. Ginsberg and William D. Harvey. Iterative broadening. In National Conference

on Artificial Intelligence, pages 216–220, 1990.

[29] M. Gleicher and A. Witkin. Supporting numerical computations in interactive contexts. In

Proceedings of Graphics Interface, pages 138–146, 1993.

[30] D. Goldberg, C. Malon, and M. Bern. A Global Approach to Automatic Solution of Jigsaw

Puzzles. In Proc. of Conf. on Computational Geometry, pages 82–87, 2002.

[31] R. Goldenberg, R. Kimmel, E. Rivlin, and M. Rudzsky. Cortex Segmentation - A Fast Varia-

tional Geometric Approach. IEEE Trans. on Medical Imaging, 21(2):1544–1551, 2002.

[32] Arthur D. Gregory, Stephen A. Ehmann, and Ming C. Lin. inTouch: Interactive multiresolution

modeling and 3d painting with a haptic interface. In VR, pages 45–53, 2000.

[33] R. M. Haralick, H. Joo, C. N. Lee, X. Zhuang, V. G. Vaidya, and M. B. Kim. Pose Estima-

tion from Corresponding Point Data. IEEE Transactions on Systems, Man, and Cybernetics,

19(6):1426–1446, November/December 1989.

[34] B. Horn. Closed-form solution of absolute orientation using orthonormal matrices. Journal

Optical Society of America, 5(7):1127–1135, 1987.

[35] B. Horn. Closed-form solution of absolute orientation using unit quaternions. Journal Optical

Society of America, 4(4):629–642, 1987.

[36] S. Joshi, U. Grenander, and M. Miller. The Geometry and Shape of Brain Sub-Manifolds. In-

ternational Journal of Pattern Recognition and Artificial Intelligence, 11(8):1317–1343, 1997.

[37] Martha Sharp Joukowsky. Petra Great Temple - Volume 1: Brown University Excavations

1993-1997. Petra Exploration Fund, Providence, RI, USA, 1999. 390 pages.


BIBLIOGRAPHY 99

[38] M. Kampel and R. Sablatnig. Color classification of archaeological fragments. In Proc. of the

15th International Conference on Pattern Recognition, volume 4, pages 771–774, Barcelona,

Spain, 2000.

[39] M. Kampel and R. Sablatnig. Automated segmentation of archaeological profiles for classifi-

cation. In ICPR, volume I, pages pp. 57–60, 2002.

[40] L. Kobbelt, T. Bareuther, and H.P. Seidel. Multiresolution Shape Deformations for Meshes

with Dynamic Vertex Connectivity. In Proc. of EUROGRAPHICS, pages 249–260, 2000.

[41] W. Kong and B. B. Kimia. On solving 2D and 3D puzzles using curve matching. In Proc. of

CVPR, Hawaii, USA, December 2001. IEEE, Computer Society.

[42] R.E. Korf. Depth-First Iterative Deepening An Optimal Admissable Tree Search. Aritificial

Intelligence, 27:97–109, 1985.

[43] Erwin Kreyszig. Differential Geometry, pages 128–129. Dover, 1963.

[44] M. Levoy. Efficient Variants of the ICP Algorithm. In Third International Conference on 3D

Digital Imaging and Modeling (3DIM).

[45] S. Li. Markov Random Field Modeling in Image Analysis, chapter 7. Springer, 2001.

[46] S. Li. Markov Random Field Modeling in Image Analysis, chapter 6, pages 165–194. Springer,

2001.

[47] C. Lu, S. M. Pizer, and J. Sarang. A Markov Random Field Approach to Multi-scale Shape

Analysis. In Proc. of Scale Space 2003, pages 416–431.

[48] R. Malladi, J. Sethian, and B. Vemuri. Shape Modeling with Front Propagation: A Level Set

Approach. PAMI, 17(2):15–175, 1995.

[49] J. McBride and B. Kimia. Archaeological Fragment Re-Assembly Using Curve Matching.

Master’s thesis, Brown University, 2002.

[50] K. McDonnell, H. Qin, and R. Wlodarczyk. Virtual clay: A Real-Time Sculpting System with

Haptic Toolkits. In Proc. of ACM Symposium on Interactive 3D Graphics, pages 179–190,

2001.


BIBLIOGRAPHY 100

[51] T. McInerney and D. Terzopoulos. Deformable Models in Medical Image Analysis. Medical

Image Analysis, 1(2):91–108, 1996.

[52] D. Metaxas, S.K. Goldenstein, and C. Vogler. Statistical Cue Integration in DAG Deformable

Models. PAMI, 25(7):801–813, 2003.

[53] D. Metaxas and I. Kakadiaris. Elastically Adaptive Deformable Models. PAMI, 24(10):1310–

1321, 2002.

[54] J.V. Miller. On GDM’s: Geometrically Deformed Models for the Extraction of Closed Shapes

from Volume Data. PhD thesis, Rensselaer Polytechnic Institute, 1990.

[55] D. Mumford and Y. Cao. Geometric structure estimation of axially symmetric pots from small

fragments. In Proc. of Int. Conf. on Signal Processing, Pattern Recognition, and Applications,

pages 92–97, 2002.

[56] D. Mumford, G. Gordon, A. Yuille, P. Giblin, and P. Hallinan. Two and Three Dimensional

Patterns of the Face. A. K. Peters, 1999.

[57] T. Nakamura, Asada M., and S. Yoshiaki. A qualitative approach to quantitative recovery of

SHGC’s shape and pose from shading and contour. In Proc. IEEE Conf. on Comp. Vision and

Pattern Recognition, pages 116–122, 1993.

[58] R. Nevatia and T. Binford. Description and recognition of curved objects. Artificial Intelli-

gence, 38:77–98, 1977.

[59] R. Nevatia and F. Ulupinar. Shape from contour: Straight homogeneous generalized cylinders

and constant cross section generalized cylinders. IEEE Trans. on Pattern Analysi and Machine

Intelligence, pages 120–135, February 1995.

[60] P. Norvig and S. Russell. Artificial Intelligence: A Modern Approach. Prentice Hall, 2003.

[61] Xavier Orriols. Generative Models for Video Analysis and 3D Range Data Applications. PhD

thesis, Universitat Autónoma de Barcelona, 2004.

[62] G. Papaioannou, E-A. Karabassi, and T. Theoharis. Reconstruction of three-dimensional ob-

jects through matching of their parts. PAMI, 2002.


BIBLIOGRAPHY 101

[63] X. Pennec and J.P. Thirion. A framework for uncertainty and validation of 3-d registration

methods based on points and frames. IJCV, 1997.

[64] S. Pizer, P. Fletcher, A. Styner, G. Gerig, and S. Joshi. Object Models in Multi-Scale Intrinsic

Coordinates via M-reps. Image and Vision Computing, (to Appear).

[65] H. Pottmann, M. Peternell, and B. Ravani. An introduction to line geometry with applications.

Computer-Aided Design, 31:3–16, 1999.

[66] H. Qin and D. Haixia. Interactive Shape Design Using Volumetric Implicit PDEs. In Proceed-

ings ACM symposium on Solid modeling and applications, pages 235–246, June 2003.

[67] H. L. Royden. Real Analysis. Prentice Hall, 1988.

[68] G. Sharp, S. W. Lee, and D. K. Wehe. ICP Registration Using Invariant Features. PAMI,

24(1):90–102, 2002.

[69] J. Stolfi and H.C. Leitão. A multiscale method for the reassembly of two-dimensional frag-

mented objects. PAMI, 2002.

[70] Dirk J. Struik. Lectures on Classical Differential Geometry. Dover, NY, 1950.

[71] R. Szeliski and D. Terzopoulos. From Splines To Fractals. Computer Graphics, 23(3):51–60,

1989.

[72] T. Tasdizen, J. P. Tarel, and D. B. Cooper. Improving the stability of algebraic curves for

applications. IEEE Trans. on Image Proc., 9(3):405–416, March 2000.

[73] G. Taubin. Estimation of planar curves, surfaces and nonplanar space curves defined by im-

plicit equations with applications to edge and range image segmentation. IEEE Trans. on

Pattern Anal. Machine Intell., 13(11):1115–1138, November 1991.

[74] G. Taubin. Estimation of Planar Curves, Surfaces and Nonplanar Space Curves Defined

by Implicit Equations with Applications to Edge and Range Image Segmentation. PAMI,

13(11):1115–1138, 1991.

[75] G. Taubin. Estimating the Tensor Of Curvature Of A Surface From A Polyhedral Approxima-

tion. In Proc. of ICCV, pages 902–907, 1995.


BIBLIOGRAPHY 102

[76] D. Terzopoulos and D. Metaxas. Dynamic 3D Models with Local and Global Deformations

: Deformable Superquadrics. IEEE Tran. on Pattern Analysis and Machine Intelligence,

13(7):703–714, 1991.

[77] D. Terzopoulos and D. Metaxas. Shape and Nonrigid Motion Estimation through Physics-

Based Synthesis. PAMI, (6):580–591, 1993.

[78] D. Terzopoulos, J.Y. Park, T. McInerney, and M. Kim. A non-self-intersecting adaptive de-

formable surface for complex boundary extraction from volumetric images. Computers &

Graphics, 25:421–440, 2001.

[79] D. Terzopoulos, J. Platt, A. Barr, and K. Fleischer. Elastically Deformable Models. Computer

Graphics, 21(4):205–214, 1987.

[80] D. Terzopoulos and H. Qin. Dynamic NURBS with Geometric Constraints for Interactive

Sculpting. ACM Transactions on Graphics, 13(2):103–136, 1994.

[81] S. Tosovic and R. Sablatnig. 3D modeling of archaeological vessels using shape from sil-

houette. In Third International Conference on 3D Digital Imaging and Modeling, Quebec,

Canada, June 2001.

[82] Gokturk Ucoluk and Ismail Hakki Toroslu. Automatic reconstruction of broken 3-D surface

objects. Computers & Graphics, 23(4):573–582, August 1999.

[83] S. Umeyama. Least-Squares Estimation of Transformation Parameters Between Two Point

Patterns. PAMI, 13(4):376–380, 1991.

[84] B. Vemuri and Y. Guo. Snake Pedals: Compact and Versatile Geometric Models with Physics-

Based Control. PAMI, 22(5):446–459, 2000.

[85] B. C. Vemuri and A. Radisavljevic. Multiresolution Stochastic Hybrid Shape Models with

Fractal Priors. ACM Transactions on Graphics, 13(2):177–207, 1994.

[86] W. Welch and A. Witkin. Free Form Shape Design Using Triangulated Surfaces. SIGGRAPH,

28:247–256, 1994.


BIBLIOGRAPHY 103

[87] A. Willis, D. Cooper, et al. Accurately Estimating Sherd 3D Surface Geometry with Applica-

tion to Pot Reconstruction. In CVPR Workshop : ACVA, June 2003.

[88] P. H. Winston. Artificial Intelligence. Addison Wesley, 1977.

[89] H. Wolfson, E. Schonberg, A. Kalvin, and Y. Lambdan. Solving Jigsaw Puzzles Using Com-

puter Vision. Annals of Operations Research, 12:51–64, 1988.

[90] M. Zerroug and R. Nevatia. Part-based 3D descriptions of complex objects from a single

image. IEEE Trans. on Pattern Analysi and Machine Intelligence, 21(9):835–848, September

1999.

[91] M. Zwicker, M. Pauly, O. Knoll, and M. Gross. Pointshop 3D: An Interactive System for

Point-Based Surface Editing. SIGGRAPH, pages 322–329, 2002.


Documents

Stochastic 3D Geometric Models for Classication, Deformation, and Estimationvisionlab.uncc.edu/.../2004_PhDThesis_complete.pdf · 2009-10-07 · Stochastic 3D Geometric Models for