Surface Reconstruction: Part I - Computer graphicsgraphics.stanford.edu/courses/cs468-12-spring/... · • “Surface reconstruction from unorganized points”, Hoppe et al., ’92

SurfaceSurfaceSurface Surface Reconstruction:Reconstruction:

Part IPart I

1

(Stanford’s) Digital Michelangelo Project(Stanford’s) Digital Michelangelo Project

4G sample points → 8M triangles1G sample points → 8M triangles

2

(Stanford’s) Digital Michelangelo Project(Stanford’s) Digital Michelangelo Project(Stanford s) Digital Michelangelo Project(Stanford s) Digital Michelangelo Project

3mm

1mm

mesh

0.3mm3

Local SightseeingLocal SightseeingLocal SightseeingLocal Sightseeing

4

Surface ReconstructionSurface ReconstructionSurface ReconstructionSurface Reconstruction

ScanningScanning devices

RegistrationImplicit

reconstruction

physicalmodel

acquiredpoint cloud

reconstructedmodelp

5

Range Scanning SystemsRange Scanning SystemsRange Scanning SystemsRange Scanning Systems

Passive: Stereo matchingFind and match features in both images

P bl N d f hProblem: Needs features to match

6

Range Scanning SystemsRange Scanning Systems

Active: Structured lightProject special b/w patterns to identify pixels

Problematic for materials / textures having strong color differences.

7


Active: Structured infrared lightProject infrared dots to identify pixels

Microsoft Kinect

video

Microsoft Kinect

Prone to noise8


Active: Structured infrared lightProject infrared dots to identify pixels

Microsoft Kinect

video

Microsoft Kinect

Prone to noise9


Active: Laser scanning, time-of-flightSend laser pulse, time how long it takes to return

A d d h i lAccuracy depends on how precisely can measure time

Operates on large distances – good for buildings, large scale objectsbuildings, large scale objects

10


Active: Laser scanning triangulationSweep laser, record where pixel intensity is max.

Problematic for difficult reflectance properties (highly specular, hair)properties (highly specular, hair)

Line of sight required to camera and laser

High accuracy

11

Range ScansRange ScansRange ScansRange Scans

• Scanning generatesScanning generates multiple range images

• Each contain 3D points for different parts of the modeldifferent parts of the model in local coordinates of the scannerscanner

• Need registration to oneNeed registration to one point cloud

12

GoalGoal

set of raw scans reconstructed model

13

Range Processing PipelineRange Processing PipelineRange Processing PipelineRange Processing Pipeline

Steps1. Initial registration

2. Pairwise registration

3 Global relaxation to spread3. Global relaxation to spread out error

4 Generate surface4. Generate surface

Slides from: http://www.cs.princeton.edu/~bjbrown/iccv05_course/Slides from: http://www.cs.princeton.edu/~bjbrown/iccv05_course/

14






Slides from: http://www.cs.princeton.edu/~bjbrown/iccvSlides from: http://www.cs.princeton.edu/~bjbrown/iccv0505_course/_course/

15






Slides from: http://www.cs.princeton.eu/~bjbrown/iccv05_course/Slides from: http://www.cs.princeton.eu/~bjbrown/iccv05_course/

16







17







18

Initial RegistrationInitial RegistrationInitial RegistrationInitial Registration

• Options:– Scans are calibrated – scan on a turntable

– Manual feature selection, global coarse alignment g g

– Automatic feature selection and matching

Aligned Scans

Partially Overlapping Scans


19

NonNon--Rigid RegistrationRigid RegistrationNonNon Rigid RegistrationRigid Registration

VideoVideo

20

PairwisePairwise Rigid RegistrationRigid RegistrationPairwisePairwise Rigid RegistrationRigid Registration

Images from: “Geometry and convergence analysis of algorithms for registration of 3D shapes” by Pottman 21

PairwisePairwise Rigid Registration GoalRigid Registration GoalPairwisePairwise Rigid Registration GoalRigid Registration Goal

Ali t ti llAlign two partially-

overlapping point clouds

given initial guess

for relative transformfor relative transform


22

Aligning 3D DataAligning 3D DataAligning 3D DataAligning 3D Data

If correct correspondences are known canIf correct correspondences are known, can find correct relative rotation/translation


23

AligningAligning 33D DataD DataAligning Aligning 33D DataD Data

• How to find correspondences: User input?How to find correspondences: User input? Feature detection? Signatures?

• Alternative: assume closest points correspond• Alternative: assume closest points correspond


24

AligningAligning 33D DataD DataAligning Aligning 33D DataD Data

• and iterate to find alignment… and iterate to find alignment– Iterative Closest Points (ICP) [Besl & McKay 92]

• Converges if starting position “close enough“• Converges if starting position “close enough“


25

Iterative Closest Point (ICP)Iterative Closest Point (ICP)Iterative Closest Point (ICP)Iterative Closest Point (ICP)

Gi t P d Q• Given two scans P and Q. • Iterate:

1. Find some pairs of closest points (pi, qi)2. Find rotation R and translation t to minimize

26

ExampleExampleExampleExample

27

FindingFinding ttFinding Finding tt

• Define barycentered point setsDefine barycentered point sets

• Optimal translation vector t maps barycentersonto each otheronto each other

• Proof [H 88]• Proof [Horn 88]

28

FindingFinding RRFinding Finding RR

• In matrix notation Frobenius normIn matrix notation

• The problem:

• Equivalent to finding rotation matrix R closest to:to: Orthogonal Procrustes Problem

29

FindingFinding RRFinding Finding RR

• Compute matrixCompute matrix

• Singular value decomposition (SVD) extracts rotation from S

[Schonemann ’66 - “A generalized solution of the orthogonal Procrustes problem”] 30

Converges?Converges?Converges?Converges?

• Errors decrease monotonically [Besl & McKay 92]y• Converges to local minimum• Good initial guess Converges to global

minimumminimum

31

ICP VariantsICP VariantsICP VariantsICP Variants

Variants on the following stages of ICPhave been proposed:

1. Selecting source points (from one or both meshes)2 Matching to points in the other mesh2. Matching to points in the other mesh3. Weighting the correspondences4. Rejecting certain (outlier) point pairs5. Assigning an error metric to the current transform6. Minimizing the error metric w.r.t. transformation


32

ICP VariantsICP VariantsICP VariantsICP Variants

Variants on the following stages of ICPhave been proposed:

1. Selecting source points (from one or both meshes)2 Matching to points in the other mesh2. Matching to points in the other mesh3. Weighting the correspondences4. Rejecting certain (outlier) point pairs5. Assigning an error metric to the current transform6. Minimizing the error metric w.r.t. transformation


33

PointPoint--toto--Plane Error MetricPlane Error MetricPointPoint toto Plane Error MetricPlane Error Metric

Using point-to-plane distance instead of point-Using point to plane distance instead of pointto-point lets flat regions slide along each other [Chen & Medioni 91][ ]


34


• Error function:Error function:

• Linearize (assume rotation is small)

Skew-Symmetric Matrix

35


R lt t i d li tResult: over-constrained linear system

36


O t i d li t• Over-constrained linear system

• Solve using least squaresSolve using least squares


37

RealReal--time Exampletime ExampleRealReal time Exampletime Example


38

PhotographPhotographPhotographPhotograph


39

RealReal--Time ResultTime ResultRealReal Time ResultTime Result


40

Global Registration GoalGlobal Registration GoalGlobal Registration GoalGlobal Registration Goal

• Given: n scans around an objectGiven: n scans around an object

• Goal: align them all

i C h h• First attempt: ICP each scan to one other


41

Global Registration GoalGlobal Registration GoalGlobal Registration GoalGlobal Registration Goal

• Want method for distributing accumulatedWant method for distributing accumulated error among all scans


42

Approach #1: Avoid the ProblemApproach #1: Avoid the ProblemApproach #1: Avoid the ProblemApproach #1: Avoid the Problem

• In some cases have 1 (possibly low-In some cases, have 1 (possibly lowresolution) scan that covers most of surface

• Align all other scans to this “anchor” [T k 94]• Align all other scans to this anchor [Turk 94]

• Disadvantage: not always practical to obtain hanchor scan


43

Approach #2: The Greedy SolutionApproach #2: The Greedy SolutionApproach #2: The Greedy SolutionApproach #2: The Greedy Solution

• Align each new scan to all previous scansAlign each new scan to all previous scans [Masuda 96]

• Disadvantages:Disadvantages:– Order dependent

– Doesn’t spread out error– Doesn t spread out error


44

Approach #3: The BruteApproach #3: The Brute--Force Force SolutionSolution

• While not converged:While not converged:– For each scan:

• For each point:For each point:– For every other scan

» Find closest point

– Minimize error w.r.t. transforms of all scans

• Disadvantage:– Solve (6n)×(6n) matrix equation,

where n is number of scans


45

Approach #Approach #33a: Slightly Less Brutea: Slightly Less Brute--ForceForce

• While not converged:While not converged:– For each scan:

• For each point:For each point:– For every other scan

» Find closest point

• Minimize error w.r.t. transform of this scan

• Faster than previous method (matrices are 6×6) [Bergevin 96, Benjemaa 97]


46

Graph MethodsGraph MethodsGraph MethodsGraph Methods

• Speedup previous by creating a graph ofSpeedup previous by creating a graph of pairwise alignments between overlapping scansscans

Scan 5Scan 3

Scan 1 Scan 5

• Find transformations consistent as possible

Scan 4Scan 2 Scan 6

Find transformations consistent as possible with all pairwise ICP


47

Surface ReconstructionSurface ReconstructionSurface ReconstructionSurface Reconstruction

Next time:

Scanning

Next time:

Scanning devices

RegistrationImplicit

reconstruction

physicalmodel

acquiredpoint cloud

reconstructedmodelp

48

ReferencesReferencesReferencesReferences

• “The 3D Model Acquisition Pipeline”, Bernardini et al, ‘02

• “A method for registration of 3-D shapes”, Besl et al., ’92 ICP paper• “Surface reconstruction from unorganized points”, Hoppe et al., ’92

• “Closed-form solution of absolute orientation using orthonormal matrices”, Horn et al., ‘88

• “A general solution of the orthogonal Procrustes problem” Schönemann ’66• A general solution of the orthogonal Procrustes problem , Schönemann 66

49

Documents

Surface Reconstruction: Part I - Computer graphicsgraphics.stanford.edu/courses/cs468-12-spring/... · • “Surface reconstruction from unorganized points”, Hoppe et al., ’92