Slide 1

Symmetric Architecture Modeling with a Single Image Author: Nianjuan Jiang, Ping Tan, Loong-Fah Cheong Department of Electrical & Computer Engineering, National University of Singapore Presenter: Feilong Yan

Slide 2

Motivation Model architecture from single image is common task in 3D creation due to the lack of the more images. Historic Photo: Internet Photo:

Slide 3

Motivation Single image based modeling is very difficult! Due to the trouble on camera calibration and texture loss The recent methods only can handle simple and planar faade Pascal Mulle et al. Image-based procedural modeling of facades Changchang Wu et al. Repetition-based Dense Single-View Reconstruction

Slide 4

Motivation But what about this one? And If we only have this single photo. Complex and not planar

Slide 5

Motivation Fortunately, the symmetry is very prevalent in the architecture Complex and not planar Symmetry is a breakthrough which magically can generate more images from the input This is reasonable, but exciting to me

Slide 6

Main Idea Bilateral Symmetry

Slide 7

Main Idea Rotational Symmetry

Slide 8

Main Idea 2 even more views Reconstruction

Slide 9

Modeling Pipeline Input Image and Frustum Vertices Calibration and 3D Reconstruction Model Initialization Texture Enhancement Model Refinement 3D Reconstruction Surface Modeling

Slide 10

3D Reconstruction Camera Calibration 3D points Reconstruction

Slide 11

Camera Calibration Previous Methods Calibrate the camera from the vanishing points of 3 mutually orthogonal directions in a single image. But many photos do not have 3 vanishing points, and this method is often numerical unstable HARTLEY, R., AND ZISSERMAN Multiple View Geometry in Computer Vision

Slide 12

Camera Calibration Previous Methods Parallelipiped is used to represent a building block. Under the constraint of parallelipiped, the visible 6 spatial vertices may be estimated. WILCZKOWIAK, M. et. al Using geometric constraints through parallelepipeds for calibration and 3d modeling This method is stable but not very suitable for some architecture If enough(>=6) correspondences between spatial vertices and the image pixels are known, the camera calibration may be immediately computed.

Slide 13

Camera Calibration New Method: Inspired by parallelipiped method, the author found the frustum more general to represent the architecture

Slide 14

Camera Calibration Demo: Frustum is symmetric

Slide 15

Camera Calibration 1 2 4 3 5 6 Coordinate represented in world: Of this example

Slide 16

Camera Calibration 1 2 4 3 5 6 t =t(x, y, z) 15 parameters to estmate

Slide 17

Camera Calibration 1 2 4 3 5 6 Simplification: 11 parameters to estimate, now the calibration is formulated as a non-linear optimization

Slide 18

Camera Calibration Optimization Initialization: The Quadratic : Extend the right multiplication, since the R is unit orthogonal matrix, then we obtain:

Slide 19

3D Points Reconstruction Symmetry-Based Triangulation:

Slide 20

3D Points Reconstruction Symmetry-Based Triangulation:

Slide 21

Surface Modeling Geometry Modeling Model Refinement Texture Mapping User-Interaction Assisted Modeling

Slide 22

Geometry Modeling Planar Structure Roof

Slide 23

Model Refinement

Slide 24

Texture Mapping Single image inevitably lack texture samples due to the foreshortening and occlusion. But to achieve a good texture effect, there are 2 requirements: 1. the final texture should be consistent with the foreshortened image ; 2, the final texture should have consistent weathering pattern. Detect Texture Quality Refine low quality region Texture the occluded region We need to know where is well textured and where not

Slide 25

Texture Quality Detection Back Project Ratio = Triangle.size / imageProjection.size Ratio > Threshold and Ratio is finite: large texture distortion Ratio