Discrete thin-plate splines - Institute for Mathematical … · · 2017-10-25... ( Mathematical...

Discrete thin-plate splines

Linda Stals

Mathematical Sciences Institute, Australian National University

Linda Stals ( Mathematical Sciences Institute, Australian National University)Discrete thin-plate splines 1 / 65

Thin-Plate Splines

3D Image Recovery

Finger Print Analysis

Image Warping

Medical Image Analysis

Data Mining

Smoothing Splines

Given a set of attributes vectors x = (x1, x2, · · · , xd)T , build a predictivemodel

y = f (x).

y ≈ f (x).

To estimate f by a 2nd-order smoothing spline minimise:

Jα(f ) =1

n∑i=1

(f (x(i))− y (i))2 + α

∑|ν|=2

)(Dν f (x))2dx.

The first term penalises lack of fit, the second penalises roughness.

Radial Basis Functions

The standard approach is to represent f as a linear combination of radialbasis functions

f (x) =M∑k=1

aφk(x) + α

n∑i=1

wiU(x, x(i)),

where φk are monomials of order up to 1 and U are suitable radial basisfunctions.Favoured method as it gives an analytical solution.

Radial Basis Functions

2D Eg:

U(x, x(i)) =−1

16πr2ln(r).

Thin Plate Splines

Requires a solution of a dense system of matrices.

System may be ill-conditioned.

Size increases with the number of data points.

Not practical for large data sets.

Finite Element Approximation

Represent f as a linear combination of linear finite elements.In vector notation f will be of the form

f (x) = b(x)Tc.

Minimise Jα over all f of this form

Jα(f ) =1

n∑i=1

(f (x(i))− y (i))2 + α

∑|ν|=2

)(Dν f (x))2dx.

[S. Roberts, M. Hegland, I. Altas; SINUM 2003]

Non-Conforming Finite elements

The smoothing term (derivatives) is not defined for piecewise multi-linearfunctions.Use non-conforming finite elements.Represent the gradient of f by u = (bTg1, ...,b

Tgd) where∫Ω∇f (x) · ∇v(x) dx =

∫Ωu(x) · ∇v(x) dx,

for all piecewise multi-linear function v .

Non-Conforming Finite elements

∫Ω∇f (x) · ∇v(x) dx =

∫Ωu(x) · ∇v(x) dx,

is equivalent to

Lc =d∑

Gsgs ,

where L is a discrete approximation to the negative Laplace operator and(G1, ...,Gd) is a discrete approximation to the transpose of the gradientoperator.

Finite Element Approximation

2nd-order smoothing spline: minimise

Jα(f ) =1

n∑i=1

(f (x(i))− y (i))2 + α

∑|ν|=2

)(Dν f (x))2dx.

Finite element approximation: minimise

Jα(c, g1, g2, · · · , gd) =1

n∑i=1

(f (x(i))− y (i))2 + α

d∑s=1

gTs Lgs ,

subject to

Lc =d∑

Gsgs .

2D Formulation

2nd-order smoothing spline: minimise

Jα(f ) =1

n∑i=1

(f (x(i))− y (i))2

((∂2

1 f (x))2

+ 2 (∂1∂2f (x))2 +(∂2

2 f (x))2)dx,

Jα(c, g1, g2) =1

n∑i=1

(b(x(i))Tc− y (i))2

∫Ω∇bT (x)g1.∇bT (x)g1 +∇bT (x)g2.∇bT (x)g2 dx

2D Formulation

Minimise:

Jα(c , g1, g2) = cTAc− 2dTc + ‖y‖2/n + α(gT1 Lg1 + gT2 Lg2)

subject toLc = G1g1 + G2g2.

n∑i=1

b(x(i))b(x(i))T ,

n∑i=1

b(x(i))y (i).

Discrete System

Recall :

Minimise:

Jα(c , g1, g2) = cTAc− 2dTc + ‖y‖2/n + α(gT1 Lg1 + gT2 Lg2)

subject toLc = G1g1 + G2g2.

A 0 0 L0 αL 0 −GT

0 0 αL −GT2

L −G1 −G2 0

,w is a Lagrange multiplier.The vectors h1, · · · ,h4 store the boundary information. The boundaryconditions can be Dirichlet or Neumann.

Boundary Condition Examples - Dirichlet

x(1) = (0.25, 0.25), x(2) = (0.75, 0.25), x(3) = (0.25, 0.75),x(4) = (0.75, 0.75); y (1) = 1, y (2) = 0, y (3) = 0 and y (4) = 1

hf (x) = tps fit. hf (x) = 0.

Boundary Condition Examples - Neumann

x(1) = (0.25, 0.25), x(2) = (0.75, 0.25), x(3) = (0.25, 0.75),x(4) = (0.75, 0.75); y (1) = 1, y (2) = 0, y (3) = 0 and y (4) = 1

0.4 0.6

0.8 1 0

0 0.2 0.4 0.6 0.8

1 1.2 1.4 1.6

Boundary Condition Test 3

Neumann

Further Model Problems - Peak

(−50

(0.5− x

(−50

(0.5− x

where ε is a normally distributed random variable with mean 0 and avariance of 0.05

0.4 0.6

Original Data

0.4 0.6

Spline Fit

-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Boundary conditions: hf = fy , hu = ∇fy and hλ = −α∆fy .

Further Model Problems - Sin

sin(4πx1) sin(4πx2)

Finite element grid of size m = 4225 with different values of α.

0 0.1 0.2 0.3 0.4 0.5 0.3 0.4

0.5 0.6

0.7 0.8

Test Problem 6 with alpha = 0.0001

0 0.1 0.2 0.3 0.4 0.5 0.3 0.4

0.5 0.6

0.7 0.8

Test Problem 6 with alpha = 0.000001

Boundary conditions: hf = fy , hu = ∇fy and hλ = −α∆fy .[M. F. Hutchinson; Communications in Statistics 1989]

Sine Example with Holes

y(x , y) = sin(2πx) sin(2πy), such that y(x , y) < 0.

n = 179401, m = 4229 with α = 10−6

Boundary: hf (x) = y(x), hu = ∇hf (x), hλ(x) = −α∆hf .

Use of FEM Grid

(0.25− ‖x(j) − [0.5, 0.5]T‖1)× (0.5− ‖x(j) − [0.5, 0.5]T‖1)

0 0.2 0.4 0.6 0.8 1

FEM grid - Diamond with Hole

Use of FEM Grid

Examples of initial FEM grid, with the centre.

0 0.2 0.4 0.6 0.8 1

FEM grid - Diamond

Sphere Example

Grid - 189 Nodes, α = 10−3.

Sphere Example

Grid - 68705 Nodes, α = 10−3.

Semi Sphere Example

Grid - 68705 Nodes, α = 10−3.

Two Sphere Example

Grid - 68705 Nodes, α = 10−7.

Convergence on Smooth Problem

y (i) = fy (x(i)) where ∇4fy = 0.

f (x) = fy (x) =

∥∥∥∥x +

[0.50.5

]∥∥∥∥2

-10-9-8-7-6-5-4-3-2

log2 of Grid Spacing

L2 Error for Model Problem 3

interpol

-10-9-8-7-6-5-4-3-2

O(h) convergenceO(h2) convergence

f (x) = fy (x) = cosh(2πx1) sin(2πx2)

-10-9-8-7-6-5-4-3-2

cg1g2w

interpol

-10-9-8-7-6-5-4-3-2

O(h) convergenceO(h2) convergence

Solution Process

Recall: A 0 0 L0 αL 0 −GT

0 0 αL −GT2

L −G1 −G2 0

.Scale by 1/α to get

A/α 0 0 L0 L 0 −GT

0 0 L −GT2

L −G1 −G2 0

−h1/αh2/αh3/αh4

.The vectors h1, · · · ,h4 store the boundary information. (Assume Dirichletfor now.)

Data Projection Matrix

The matrix A = BBT , where

BT =1√n

b1(x(1)) · · · bm(x(1))...

......

b1(x(n)) · · · bm(x(n))

,is symmetric, nonnegative and sparse.B : Data points→ FEM.The properties of the A matrix varies with the distribution of the datapoints.

If no data points lie in the support of basis function bi , the entries inthe ith column are zero. In which case A, has a non-trivial null-space.

If any triangles has less than two data points, the system isunder-determined system and thus A has a non-trivial null-space

Let P =[0 Ip

]be a projection onto the region of the domain containing

data points. ( Ip be an identity matrix).The matrix Ap := PAPT is defined on the region of the domain wherethere are data points.Note that A = PTApP, since A is zero in the region where there are nodata points.

Furthermore, Ap may be decomposed as follows

[Boo Bou

] [BToo 0

BTou BT

[AToo Bou

BTou Iu

where Iu is an identity matrix, Aoo = BooBToo + BouB

Tou and

[Io 00 BT

]with Io being an identity matrix.

[AToo Bou

BTou Iu

Then Ap = QTFQ and A = (PTQT )F (QP).Note that F is SPD.

Example Data Distribution

Figure : Example distribution of data in a uniform finite element gridcorresponding to the A matrix

Figure : Example distribution of data in a uniform finite element gridcorresponding to the Ap matrix. The empty triangles have been removed

Figure : Example distribution of data in a uniform finite element gridcorresponding to the F matrix. Work directly on the data points in regions wherethere are not a lot of data points.

SMW formula

Recall:

L −G1 −G2 00 L 0 −GT

0 0 L −GT2

A/α 0 0 L

If P1 =

[0 · · · 0 P

]and P2 =

[P 0 · · · 0

]then M + PT

1 ApP2 =M + PT

1 QTFQP2 is equivalent to above matrix where

L −G1 −G2 00 L 0 −GT

0 0 L −GT2

0 0 0 L

.This is an upper triangular matrix with the Laplacian matrix sitting alongthe main diagonal. As the L matrices are SPD the above system can bereadily inverted by using back substitution.

SMW formula

Recall:

(M + X1FX

)−1= M−1 −M−1X1

[F−1 + XT

2 M−1X1

]−1XT

2 M−1.

According to the to Sherman–Morrison–Woodbury formula

S−1 = M−1 −M−1PT1 QT

[F−1 + QPHPTQT/α

]−1QP2M

where H = L−1(∑d

s=1 GsL−1GT

)L−1.

M is an upper triangular matrix and is easily inverted.

The Laplacian matrix L is SPD and L−1 is applied to a vector byusing an optimal solver such as the multigrid method.

Most of the work goes into solving systems of the form Eαv = αf,where Eα :=

(αF−1 + QPHPTQT

). The matrix Eα is SPD so a PCG

solver is used.

Eα Condition Number

To measure the conditioning we need to determine the eigenvalues of L, Fand Gs .We firstly look at a uniform finite element grid defined on the onedimensional domain [0, 1] with grid spacing h = 1/(m + 1).If the domain contains a lot of data spread throughout,

Eα = αA−1u + L−1

−1GT1

)L−1.

[L. Stals; Journal of Scientific Computing 2013]

Laplacian Matrix

For a uniform finite element grid in one dimension, the stencils for L is

[−1 2 −1

Following Hackbusch, we can show the eigenvalues of L are

λµ(L) =4

(πµh

and the eigenvectors are uµ where uµj = sin (πµjh) for 1 ≤ µ ≤ m and1 ≤ j ≤ m.

Data Matrix

Recall:

n∑j=1

b(x(j))b(x(j))T .

If there are a lot of data points throughout the whole domain, i.e. n islarge,

Ak,l ≈∫

Ωbk(x)bl(x) dx.

We construct a matrix Au, where (Au)k,l =∫

Ω bkbl dx, that is defined bythe stencil

[1 4 1

Consequently, Au can be used as an approximation to A.

Data Matrix

By using the same eigenvectors that we used for L, uµ, we get

λµ (Au) =h

(2 cos2

(πµh

G1L−1GT

We have uµ is also an eigenvector for G1L−1GT

1 with correspondingeigenvalues

λµ(G1L

−1GT1

)= h cos2

(πµh

Theorem

IfEα = αA−1

u + L−1(G1L

−1GT1

)L−1,

λµ(Eα) =3α

2 cos2(πµh

cos2(πµh

(πµh

) ,λµ(AuEα) = α +

(2 cos2

(πµh

cos2(πµh

(πµh

) ,and

λµ(LEαL) =48α

sin4(πµh2 )

2 cos2(πµh

+ h cos2

(πµh

where 1 ≤ µ ≤ m.

Parallel Implementation

Recall :

We want to solve

S−1 = M−1 −M−1PT1 QT

[F−1 + QPHPTQT/α

]−1QP2M

where H = L−1(∑d

s=1 GsL−1GT

)L−1.

Expensive part:

Eα :=(αF−1 + QPHPTQT

(Eg. Eα = αA−1u + L−1

−1GT1

)L−1. )

Saddle Point Formulation

Recall: (yet again)A/α 0 0 L

0 L 0 −GT1

0 0 L −GT2

L −G1 −G2 0

−h1/αh2/αh3/αh4

.Swap the first and last rows to get

L −G1 −G2 00 L 0 −GT

0 0 L −GT2

A/α 0 0 L

h2/αh3/αh1/α

Matrix Properties

Symmetric Indefinite for Dirichlet

Symmetric and Singular for Neumann;

null space =

Matrix Properties

Conditioning = O(h2), but large constant.

Matrix Properties

Use SQMR method as solver directly on saddle point problem (butmust be careful about breakdowns.)

SQMR same as MINRES for SPD preconditioners. Unlike MINRESSQMR allows the use of an indefinite preconditioner. Not so muchtheory for SQMR.

Model Problem

y(x) = exp(−50 (x − 0.5)2

0.0 0.2 0.4 0.6 0.8 1.0x

−0.2

1.0 Example splines for different values of α

α=1.0−2

α=1.0−5

α=1.0−8

Figure : Spline fit to example test problem with α = 10−2, 10−5, 10−8 and gridspacing h = 2−6.

Convergence Rate I (α = 10−2)

2 4 6 8 10 12

of Itera

I Preconditioner (α = 10−2

163264

Dirichlet top Neumann did not converge(Similar for α = 10−5 and α = 10−8.)

Domain Decomposition Method

N∑i=1

RTi SiRi .

The boundary conditions on each subdomain is Dirichlet, so can use SMWapproach to solve problem on each subdomain.I did not find much theory on the use of DD methods with indefiniteproblems, so did testing with Python on 1D model problem before movingto proper parallel code.

Convergence Rate DD (α = 10−2)

2 4 6 8 10 12

of Itera

DD Preconditioner (α = 10−2

163264

Dirichlet top Neumann did not converge

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

Domain Decomposition Method With Coarse GridPreconditioner

P−1coarse = RT

(R0SRT

)−1R0 +

N∑i=1

RTi SiRi .

The coarse grid is designed to move the small eigenvalues away from zero.

Convergence Rate DD+Coarse (α = 10−2)

2 4 6 8 10 12

of Itera

Coarse Preconditioner (α = 10−2

163264

2 4 6 8 10 12

of Itera

163264

Dirichlet top Neumann bottomLinda Stals ( Mathematical Sciences Institute, Australian National University)Discrete thin-plate splines 54 / 65

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

Domain Decomposition Method - PBNN Preconditioner

Q = RT0

(R0SRT

)−1R0.

R = I − QS.

BNN= RTP−1

DDR + Q.

This technique is similar to coarse grid correction, but removesaccumulation of errors.

Convergence Rate DD+Projection (α = 10−2)

2 4 6 8 10 12

of Itera

Projection Preconditioner (α = 10−2

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2 4 6 8 10 12

of Itera

163264

2D Example

0.4 0.6

Original Data Set (641601 data points)

2D Example

FEM fit with 4225 nodes and α = 10−8.

Future Work

Finish looking into parallel solver.

Run some large 2D and 3D codes.

Adapative refinement

Error indicator

Reduce/Remove reading data in from files

Load balancing: take into account connection between FEM grid anddata points.

10 20 30 40 50 60

Thank You

Discrete thin-plate splines - Institute for Mathematical … · · 2017-10-25... ( Mathematical...

Documents

Non-Rigid Range-Scan Alignment Using Thin-Plate SplinesNon-Rigid Range-Scan Alignment Using Thin-Plate Splines Benedict J. Brown Princeton University bjbrown@cs.princeton.edu Szymon

Linear Splines

Cubic Splines

B-Splines Primer

Buckling of thin-walled steel shells with closely spaced, discrete … · 2018. 1. 26. · Buckling of thin-walled steel shells with closely spaced, discrete and ﬂexible anchors

Thin plate regression splines - Nabble · Thin plate regression splines Simon N.Wood University of St Andrews, UK [Received October 2001. Final revision June 2002] Summary. I discuss

Interpolating Splines

Discrete B-Splines and Subdivision Techniques in Computer

Subdivision Schemes for Thin Plate Splinesjwarren/papers/thinplate.pdfWeimer and Warren / Subdivision Schemes for Thin Plate Splines 1.1. Thin Plate Splines Thin plate splines were

Non-rigid Surface Registration Using Spherical Thin-Plate ...jinghua/publication/miccai-07.pdfNon-rigid Surface Registration Using Spherical Thin-Plate Splines Guangyu Zou 1,JingHua,

Silvia Bianconcini A Reproducing Kernel Perspective of ... · splines and vector splines on the sphere, partial splines and interaction splines, variational problems where the data

Subdivision Schemes for Thin Plate Splinesjwarren/papers/thinplate.pdf · Keywords: surface modeling, variational design, subdivision, thin plate splines, optimization, scattered

Growth and Pattern Formation for Epitaxial SurfacesAtomistic Modeling of Strain in Thin Films • Lattice statics for discrete atomistic system, – minimize discrete strain energy

Flexible smoothing with B-splines and Penalties or P-splines

Splines and Extended Splines - teachers.dadeschools.netteachers.dadeschools.net/aalvarez2/Lessons/3DS_Max/Splines.pdfSplines and Extended Splines Learning Objectives After completing

THIN CLIENT DISTRIBUTED SIMULATION - … · Thin Client Distributed Simulation Architecture. ... Thin Client Distributed Simulation of Discrete Event Models Architecture ... HTML5

MaxEnt 2007, Udo v. Toussaint, July 2007, udo.v.toussaint@ipp.mpg.de 1 Udo v. Toussaint, Silvio Gori 2-D Deconvolution using Thin-Plate Splines Astrophysics:

A Discrete Model for Inelastic Deformation of Thin Shells

Curves and Splines

A Discrete Model for Inelastic Deformation of Thin Shellsmrl.nyu.edu/~dzorin/papers/secord2004sds.pdf · A Discrete Model for Inelastic Deformation of Thin Shells ... [COPar] use