1Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Shape reconstruction and inverse problems

Lecture 9

© Alexander & Michael Bronsteintosca.cs.technion.ac.il/book

Numerical geometry of non-rigid shapesStanford University, Winter 2009

2Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

3Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Measurement

Shape space Measurement space

Projection

4Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Reconstruction

?Find Given

Shape space Measurement space

5Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Reconstruction

Shape space Measurement space

6Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Inverse problems

Reconstruct the shape by minimizing the distance between

given measurement and measurement obtained from shape

Given a (possibly noisy) measurement of unknown shape

7Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Inverse problems

8Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Ill-posedness

Many shapes have the

same measurement!

Shape space Measurement space

9Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Prior knowledge

We know that the measurements come from

deformations of the same object!

10Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Regularization

Deformations of the dog shape

Shape space Measurement space

11Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Shape space Measurement space

Prior

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Regularization

12Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Inverse problems with intrinsic prior

Prior is given on the intrinsic geometry of the shape (intrinsic prior)

Error = distance between measurements

Regularization = intrinsic distance from prior shape

Prior shape is a deformation of the shape we need to reconstruct

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Error Regularization

13Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Solution of inverse problems with intrinsic prior

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Optimization variable: shape , represented as a set of

coordinates

Possible initialization: prior shape

Gradients of and w.r.t. are required

Does it sound familiar?

14Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Intrinsic dissimilarity

Ext

rinsi

c di

ssim

ilarit

y

15Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Joint similarity as inverse problem

Measurement space = shape space

Identity projection operator

= intrinsic distance on shape space

= extrinsic distance on measurement space

Prior = the shape itself

16Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Computation of the regularization term

A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

Assume shape = deformation of prior with the same connectivity

Trivial correspondence

Compute L2 distortion of geodesic distances

and gradient

is a fixed (precomputed) matrix of geodesic distances on

depends on the variables (must be updated on every iteration)

17Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Computation of using Dijkstra’s algorithm

A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

Same approach as in joint similarity

Compute and fix the path of the geodesic

is a matrix of Euclidean distances between adjacent vertices

is a linear operator integrating the path length along fixed path

At each iteration, only changes

Computation of is straightforward

18Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Inconsistency of Dijkstra’s algorithm

Number of points

1

1.1

1.05

Dijkstra/Analytic

FMM/Analytic

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

19Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Computation of using Fast Marching

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Standard FMM FMM with derivativepropagation

Distance update Distance update

Distance derivative update

20Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Shape-from-X

Shading Stereo

Silhouette Sparse points Image

21Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Denoising

Measurement space = shape space

Identity projection operator

= intrinsic distance on shape space

= extrinsic distance on measurement space

Noisy measurement

22Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Denoising

Unknown shape

Prior Measurement Reconstruction(without prior)

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

23Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Denoising

Unknown shape

Prior Measurement Reconstruction(with prior)

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

24Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Bundle adjustment

Shape Noisy measurement

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Clean measurement

25Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Measurement space of 2D point clouds

Projection operator

(assuming known correspondence)

Noisy measurement

Bundle adjustment

26Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Unknown shape

Prior Measurement Reconstruction(without prior)

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Bundle adjustment

27Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Unknown shape

Prior Measurement

Devir, Rosman, Bronstein, Bronstein, Kimmel, 2009

Reconstruction(with prior)

Bundle adjustment

28Numerical geometry of non-rigid shapes Shape reconstruction and inverse problems

Shape-to-image matching

Measured image Reconstruction

Salzmann, Pilet, Ilic, Fua, 2007