Surface Reconstruction: Part I - Computer...

SurfaceSurfaceSurface Surface Reconstruction:Reconstruction:

Part IPart I

(Stanford’s) Digital Michelangelo Project(Stanford’s) Digital Michelangelo Project

4G sample points → 8M triangles1G sample points → 8M triangles

(Stanford’s) Digital Michelangelo Project(Stanford’s) Digital Michelangelo Project(Stanford s) Digital Michelangelo Project(Stanford s) Digital Michelangelo Project

0.3mm3

Local SightseeingLocal SightseeingLocal SightseeingLocal Sightseeing

Surface ReconstructionSurface ReconstructionSurface ReconstructionSurface Reconstruction

ScanningScanning devices

RegistrationImplicit

reconstruction

physicalmodel

acquiredpoint cloud

reconstructedmodelp

Range Scanning SystemsRange Scanning SystemsRange Scanning SystemsRange Scanning Systems

Passive: Stereo matchingFind and match features in both images

P bl N d f hProblem: Needs features to match

Range Scanning SystemsRange Scanning Systems

Active: Structured lightProject special b/w patterns to identify pixels

Problematic for materials / textures having strong color differences.

Active: Structured infrared lightProject infrared dots to identify pixels

Microsoft Kinect

Prone to noise8

Active: Structured infrared lightProject infrared dots to identify pixels

Microsoft Kinect

Prone to noise9

Active: Laser scanning, time-of-flightSend laser pulse, time how long it takes to return

A d d h i lAccuracy depends on how precisely can measure time

Operates on large distances – good for buildings, large scale objectsbuildings, large scale objects

Active: Laser scanning triangulationSweep laser, record where pixel intensity is max.

Problematic for difficult reflectance properties (highly specular, hair)properties (highly specular, hair)

Line of sight required to camera and laser

High accuracy

Range ScansRange ScansRange ScansRange Scans

• Scanning generatesScanning generates multiple range images

• Each contain 3D points for different parts of the modeldifferent parts of the model in local coordinates of the scannerscanner

• Need registration to oneNeed registration to one point cloud

GoalGoal

set of raw scans reconstructed model

Range Processing PipelineRange Processing PipelineRange Processing PipelineRange Processing Pipeline

Steps1. Initial registration

2. Pairwise registration

3 Global relaxation to spread3. Global relaxation to spread out error

4 Generate surface4. Generate surface

Slides from: http://www.cs.princeton.edu/~bjbrown/iccv05_course/Slides from: http://www.cs.princeton.edu/~bjbrown/iccv05_course/

Slides from: http://www.cs.princeton.edu/~bjbrown/iccvSlides from: http://www.cs.princeton.edu/~bjbrown/iccv0505_course/_course/

Slides from: http://www.cs.princeton.eu/~bjbrown/iccv05_course/Slides from: http://www.cs.princeton.eu/~bjbrown/iccv05_course/

Initial RegistrationInitial RegistrationInitial RegistrationInitial Registration

• Options:– Scans are calibrated – scan on a turntable

– Manual feature selection, global coarse alignment g g

– Automatic feature selection and matching

Aligned Scans

Partially Overlapping Scans

NonNon--Rigid RegistrationRigid RegistrationNonNon Rigid RegistrationRigid Registration

VideoVideo

PairwisePairwise Rigid RegistrationRigid RegistrationPairwisePairwise Rigid RegistrationRigid Registration

Images from: “Geometry and convergence analysis of algorithms for registration of 3D shapes” by Pottman 21

PairwisePairwise Rigid Registration GoalRigid Registration GoalPairwisePairwise Rigid Registration GoalRigid Registration Goal

Ali t ti llAlign two partially-

overlapping point clouds

given initial guess

for relative transformfor relative transform

Aligning 3D DataAligning 3D DataAligning 3D DataAligning 3D Data

If correct correspondences are known canIf correct correspondences are known, can find correct relative rotation/translation

AligningAligning 33D DataD DataAligning Aligning 33D DataD Data

• How to find correspondences: User input?How to find correspondences: User input? Feature detection? Signatures?

• Alternative: assume closest points correspond• Alternative: assume closest points correspond

AligningAligning 33D DataD DataAligning Aligning 33D DataD Data

• and iterate to find alignment… and iterate to find alignment– Iterative Closest Points (ICP) [Besl & McKay 92]

• Converges if starting position “close enough“• Converges if starting position “close enough“

Iterative Closest Point (ICP)Iterative Closest Point (ICP)Iterative Closest Point (ICP)Iterative Closest Point (ICP)

Gi t P d Q• Given two scans P and Q. • Iterate:

1. Find some pairs of closest points (pi, qi)2. Find rotation R and translation t to minimize

ExampleExampleExampleExample

FindingFinding ttFinding Finding tt

• Define barycentered point setsDefine barycentered point sets

• Optimal translation vector t maps barycentersonto each otheronto each other

• Proof [H 88]• Proof [Horn 88]

FindingFinding RRFinding Finding RR

• In matrix notation Frobenius normIn matrix notation

• The problem:

• Equivalent to finding rotation matrix R closest to:to: Orthogonal Procrustes Problem

FindingFinding RRFinding Finding RR

• Compute matrixCompute matrix

• Singular value decomposition (SVD) extracts rotation from S

[Schonemann ’66 - “A generalized solution of the orthogonal Procrustes problem”] 30

Converges?Converges?Converges?Converges?

• Errors decrease monotonically [Besl & McKay 92]y• Converges to local minimum• Good initial guess Converges to global

minimumminimum

ICP VariantsICP VariantsICP VariantsICP Variants

Variants on the following stages of ICPhave been proposed:

1. Selecting source points (from one or both meshes)2 Matching to points in the other mesh2. Matching to points in the other mesh3. Weighting the correspondences4. Rejecting certain (outlier) point pairs5. Assigning an error metric to the current transform6. Minimizing the error metric w.r.t. transformation

ICP VariantsICP VariantsICP VariantsICP Variants

Variants on the following stages of ICPhave been proposed:

1. Selecting source points (from one or both meshes)2 Matching to points in the other mesh2. Matching to points in the other mesh3. Weighting the correspondences4. Rejecting certain (outlier) point pairs5. Assigning an error metric to the current transform6. Minimizing the error metric w.r.t. transformation

PointPoint--toto--Plane Error MetricPlane Error MetricPointPoint toto Plane Error MetricPlane Error Metric

Using point-to-plane distance instead of point-Using point to plane distance instead of pointto-point lets flat regions slide along each other [Chen & Medioni 91][ ]

• Error function:Error function:

• Linearize (assume rotation is small)

Skew-Symmetric Matrix

R lt t i d li tResult: over-constrained linear system

O t i d li t• Over-constrained linear system

• Solve using least squaresSolve using least squares

RealReal--time Exampletime ExampleRealReal time Exampletime Example

PhotographPhotographPhotographPhotograph

RealReal--Time ResultTime ResultRealReal Time ResultTime Result

Global Registration GoalGlobal Registration GoalGlobal Registration GoalGlobal Registration Goal

• Given: n scans around an objectGiven: n scans around an object

• Goal: align them all

i C h h• First attempt: ICP each scan to one other

Global Registration GoalGlobal Registration GoalGlobal Registration GoalGlobal Registration Goal

• Want method for distributing accumulatedWant method for distributing accumulated error among all scans

Approach #1: Avoid the ProblemApproach #1: Avoid the ProblemApproach #1: Avoid the ProblemApproach #1: Avoid the Problem

• In some cases have 1 (possibly low-In some cases, have 1 (possibly lowresolution) scan that covers most of surface

• Align all other scans to this “anchor” [T k 94]• Align all other scans to this anchor [Turk 94]

• Disadvantage: not always practical to obtain hanchor scan

Approach #2: The Greedy SolutionApproach #2: The Greedy SolutionApproach #2: The Greedy SolutionApproach #2: The Greedy Solution

• Align each new scan to all previous scansAlign each new scan to all previous scans [Masuda 96]

• Disadvantages:Disadvantages:– Order dependent

– Doesn’t spread out error– Doesn t spread out error

Approach #3: The BruteApproach #3: The Brute--Force Force SolutionSolution

• While not converged:While not converged:– For each scan:

• For each point:For each point:– For every other scan

» Find closest point

– Minimize error w.r.t. transforms of all scans

• Disadvantage:– Solve (6n)×(6n) matrix equation,

where n is number of scans

Approach #Approach #33a: Slightly Less Brutea: Slightly Less Brute--ForceForce

• While not converged:While not converged:– For each scan:

• For each point:For each point:– For every other scan

» Find closest point

• Minimize error w.r.t. transform of this scan

• Faster than previous method (matrices are 6×6) [Bergevin 96, Benjemaa 97]

Graph MethodsGraph MethodsGraph MethodsGraph Methods

• Speedup previous by creating a graph ofSpeedup previous by creating a graph of pairwise alignments between overlapping scansscans

Scan 5Scan 3

Scan 1 Scan 5

• Find transformations consistent as possible

Scan 4Scan 2 Scan 6

Find transformations consistent as possible with all pairwise ICP

Surface ReconstructionSurface ReconstructionSurface ReconstructionSurface Reconstruction

Next time:

Scanning

Next time:

Scanning devices

RegistrationImplicit

reconstruction

physicalmodel

acquiredpoint cloud

reconstructedmodelp

ReferencesReferencesReferencesReferences

• “The 3D Model Acquisition Pipeline”, Bernardini et al, ‘02

• “A method for registration of 3-D shapes”, Besl et al., ’92 ICP paper• “Surface reconstruction from unorganized points”, Hoppe et al., ’92

• “Closed-form solution of absolute orientation using orthonormal matrices”, Horn et al., ‘88

• “A general solution of the orthogonal Procrustes problem” Schönemann ’66• A general solution of the orthogonal Procrustes problem , Schönemann 66

Surface Reconstruction: Part I - Computer...

Documents

Atlas-Free Surface Reconstruction of the Cortical Grey ... · Atlas-Free Surface Reconstruction of the Cortical Grey- ... L, et al. (2011) Atlas-Free Surface Reconstruction of the

Surface Reconstruction from Point Clouds - MIT CSAIL · 2007. 9. 18. · Initial Detection (“Pop-out”) Ballooning Surface Reconstruction Surface Segmentation Recognition Multiple

Surface Reconstruction from 3D Line Segments

Piecewise smooth surface reconstruction

Ensembles for Surface Reconstruction - POSTECHcg.postech.ac.kr/papers/Ensembles-for-Surface-Reconstruction.pdf · Ensembles for Surface Reconstruction The input of the surface reconstruction

Road Surface Reconstruction by Stereo Vision

Surface Vibration Reconstruction using Inverse Numerical

Scalable Surface Reconstruction From Point Clouds …openaccess.thecvf.com/.../Mostegel_Scalable_Surface_Reconstruction... · Scalable Surface Reconstruction from Point Clouds with

Surface Reconstruction using Radial Basis Functions

Watertight Surface Reconstruction for Uncertain Data

Surface reconstruction using point cloud

Surface reconstruction from photometric stereo imagesmcl.usc.edu/.../2014/01/1993-05-Surface-reconstruction-from-photometric-stereo-images.pdfSurface reconstruction from photometric

Surface Reconstruction: Part II - Computer graphicsgraphics.stanford.edu/.../LectureSlides/04_Surface_Reconstruction.pdf · Surface Reconstruction: Part II. Surface ReconstructionSurface

Feature-Preserving Surface Reconstruction and Simpli ... · 2.1 Surface Reconstruction A common approach to robust surface reconstruction from defect-laden point sets involves denoising

Surface Reconstruction Algorithms:Review and Comparison

Surface Reconstruction by Voronoi Filtering

Screened Poisson Surface Reconstruction

Delaunay Triangulation Based Surface Reconstruction: Ideas

Surface reconstruction in gold nanowiresScientific REPORTS | (2018)8:9836 I. 1 .nae.comcienificepo Surface reconstruction in gold nanowires Yasuchika Suzuki & Tokushi Kizuka Surface

SURFACE RECONSTRUCTION FOR GENERATING DIGITAL …