RANSAC-Assisted Display Model Reconstruction for Projective Display Patrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs

RANSAC-Assisted Display Model Reconstruction for Projective

DisplayPatrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs

Department of Computer Science

University of North Carolina at Chapel Hill

Emerging Display Technologies 2006 – March 26, 2006

2 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Multi-Projector Display


Perspectively CorrectP


Perspectively-correct Rendering

P matricesIntrinsics and extrinsics of each projector

Viewer’s locationDisplay Surface Model

New Method for Display Surface Estimation!


Rendering Methods

Rendering Algorithm

1-pass (homography)

2-pass (projective tex.)

Display Shape

Planar surface Complex shaped surface

References

Raskar99,Yang01,Sukthankar01,Chen02, Steele02,Wallace04

Raskar98,Yang-Welch01,Brown02,Cotting04


Display Surface

A Primer: 2-Pass Rendering

1st PassGeometry: application definedViewpoint: user’s positionResult: “ideal image”

2nd PassGeometry: screen surface•Use “ideal image” as projected

texture

Viewpoint: projector’s positionResult: projector’s image

Viewer

Projector


Display Surface Estimation

3D Stereo

Reconstruction

Display Surface Model

Generation3D Point Cloud

Tessellated Mesh


Artifacts with Tessellated Meshes

Inaccurate corner representations

Sampling issue

Texture mapping distortion on planar surfaces

Reconstruction errors

Holes in surface model

Meshing errors


Visual ArtifactsP


Rooms Are Piecewise Planar

New display model estimation method

Fit planes to point cloudConvert planes to polygons for rendering

AdvantageCorners can be accurately estimated (intersection of planes)Noiseless models eliminate texture distortion


RANSAC Plane Fitting

Random Sample ConsensusDesigned to work with many outliersFinds largest set of inliersCode from Peter Kovesi, Univ. Western Australia

Hypothesis plane from 3 random pointsFinds plane with most inliers

User specified fit tolerance

Fits least-squares plane to these inliers


Method

FitPlane( C )• Fit one plane P to point cloud C using

RANSAC• Remove P inlier points from cloud C’ = C –

Pinliers

• Recursively loop using outliers from this pass as input for next plane


RANSAC Plane Fitting


Plane To Polygons

Task: What planes intersect?

Reduce complexity to a 2D problemEstablish a “floor” plane orthogonal to the “wall” planesProject wall points onto this planeCasts problem as a line intersection problem


Cast into a 2D Task

Intersecting Line (Plane)

Pairs

Red – Orange

Orange – Green

Green – Cyan

Cyan – Magenta

Magenta – Black

Black – Blue


Results

Less than 2-3% dimensional errorRe-projection: Std. Dev. < 0.5 pixels


Results

Improved corner matching (< 2 pixels)Lack of texture distortion in application scenery

PV


FlightGear DemoP P


Installations

UNC Lab3 walls with columns in each corner

UNC lobby and I/ITSEC 2005

2-wall corner

Naval Research Lab

3 walls with a column in one corner


SUMMARY: New Surface Model Generation MethodRecursively extract planes from 3D point cloud with RANSAC-assisted algorithm

Removes outliers easilyFits noisy data well

Convert Planes to Polygons for 2-pass renderingResults in simple, accurate model

Much less distortion than with tessellated mesh


Future Work

Use optimal triangulation algorithmRather than DLT; less noise in data

Generalized Planes to Polygon solutionExtract other surface functions

Applicability of cylinders, quadrics, etc.

Apply techniques to create larger virtual environmentsContinuous calibration/refinement during operation


Thank You

FundingFundingOffice of Naval Research Award N00014-03-1-0589Office of Naval Research Award N00014-03-1-0589• DARWARS Training Superiority program DARWARS Training Superiority program • VIRTE – Virtual Technologies and Environments programVIRTE – Virtual Technologies and Environments program



Planes to Polygons

1. Compute intersection line between two planes

Align this with Y axis

2. Project down all points to get floorplanLine segments, based on range of inliers

3. Find closest 2D line intersectionsDetermines which planes to intersect

4. Intersect the planes, make height uniform5. Truncate non-intersecting planes (edges)


Room Coordinate System

Motivation Can easily estimate position of viewer

Auto-alignment performedIncorporated into Planes to Polygons algorithm

Chosen to be intersection of planesAlso have tool to change coordinate system

Extends to a tracked user


3D Stereo Reconstruction

1. Display feature set with projector(s)Checkerboard, Gray-coded stripes, Gaussian blobs, etc.

2. Image with stereo camera pair3. Extract, identify, match feature

points4. Triangulate corresponding points

Model represented by 3D point cloud!


Plane 1


Plane 2


Plane 3


Plane 4

Documents

RANSAC-Assisted Display Model Reconstruction for Projective Display Patrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs