Structure Recovery by Part Assembly Chao-Hui Shen, Hongbo Fu, Kang Chen and Shi-Min Hu Tsinghua University City University of Hong Kong Presented by: Chenyang Zhu

Outline Background Overview Method Conclusion

Background Consumer level scanning devices Capture both RGB and depth Reconstruction is challenging ◦ Low resolution ◦ Noise ◦ Missing data ◦…◦…

Example-based Scan Completion Global-to-local and top-down [Kraevoy and Sheffer 2005; Pauly et al. 2005] Rely on the availability of suitable template model However … No suitable model! shape retrieval

Solution Recover the Structure by Part Assembly ◦ Structure recovery instead of geometry reconstruction ◦ Do NOT prepare a large database ◦ Retrieve and assemble suitable parts on the fly

Problem setup Input Point cloud + Image (Single view) Pre-segmented Repository Models (Parts + Labels) … Output ……

Algorithm Overview Candidate Parts SelectionStructure CompositionPart Conjoining …

Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Search in a 3D offset window around the part

Geometric contribution score Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Search in a 3D offset window around the part Geometric fidelity score 3D2D edge map 2D 3D

Candidate Parts Employ 3D repository model as a global context ◦ Globally align the models with the input scan first Select top K parts with highest score for each category Seat Back Arm Front leg … …… ………………

Structure Composition Search for promising compositions under constraints … Optimal composition average geometry fidelity of parts total geometry fidelity total geometry contribution Globally Evaluate the compositions

Part Conjoining Problem: the parts are loosely placed together Goal: generate a consistent & complete model

Part Conjoining identity scale i j transformed contact points

Conclusion A bottom-up structure recovery approach ◦ Effectively reuse limited repository models ◦ Automatically compose new structure ◦ Handle single-view inputs by the Kinect system Future work ◦ Multi-view inputs ◦ Include style/functional constraints ◦ Recover Indoor scenes

THANK YOU! Q&A