Temporally Coherent Completion of Dynamic Shapes

Hao Li, Linjie Luo, Daniel Vlasic, Pieter Peers, Jovan Popović, Mark Pauly, Szymon RusinkiewiczACM Transactions on Graphics 31(1) , ( to be presented at SIGGRAPH 2012)

Presenter: Kangxue YinApr. 20, 2012

outline

• Introduction and Related Work• The proposed Method:

• Pairwise correspondence• Hole filling• Temporal filtering• Detail resynthesis

• Results and Conclusion

Introduction and Related Work

The problem:How to obtain temporally coherent and watertight 3D meshes from scan sequences of dynamic performances?

Related topics：Space-time reconstructionNon-rigid registration

• hole filling techniquesJU, T. 2009. Fixing geometric errors on polygonal models: A survey. Journal of Computer Science and Technology 24, 1, 19–29.

• Non-rigid registration: • template-based method• articulation-based method• consensus skeleton

Introduction and Related Work

fails with drastic topology changes

The proposed Method: overview

Shape completion

The proposed Method: Pairwise Correspondence

A two-scale registration: Coarse-scale and fine-scale.

Coarse-scale Registration Based on Deformation Graphs [Li et al. 2009]:

a non-rigid ICP algorithm

The proposed Method: Pairwise Correspondence

Fine-scale alignment

a non-rigid locally weighted ICP algorithm based on [Brown and Rusinkiewicz 2007]

The proposed Method: Pairwise Correspondence

The proposed Method: Hole filling

visual Hull prior [Vlasic et al. 2009] + weighted Poisson surface reconstruction [Kazhdan et al. 2006]

Fairing: Minimizing bending energy of the patch’s vertices using bi-Laplacian

The proposed Method : Temporal filtering

1. Warp two neighboring frames to current frame based on the pairwise correspondences

2. Combine them using Poisson reconstruction with different weight for different region

Poisson reconstruction

warp to warp to

The proposed Method : Detail resynthesis

1. Acquire normal maps:Daniel Vlasic, Pieter Peers, Ilya Baran, Paul Debevec, Jovan Popović, Szymon Rusinkiewicz, and Wojciech Matusik. Dynamic Shape Capture using Multi-View Photometric Stereo. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 28(5), December 2009.

2. Resynthesize high frequency detail from normal maps: Diego Nehab, Szymon Rusinkiewicz, James Davis, and Ravi Ramamoorthi. Efficiently Combining Positions and Normals for Precise 3D Geometry. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH 2005) 24(3), August 2005.

The proposed Method: summary

Shape completion

The proposed Method: Results and Conclusion

The proposed Method: Results and Conclusion

Contribution: the framework and some little improvements on previous algorithms ( too detailed )

Limitation:1. each frame should cover most part of the object surface ( limited to multi-view scans)2. Topology change problem was not solved fundamentally

Future work:To take physical properties into account

1. registration of dynamic shape with drastic topology changed2. Reconstruction of interacting objects.

My Plan

topology change interacting objects