1. Example Based 3D Shape Completion Mark Pauly 1,2, Niloy J. Mitra 1, Joachim Giesen 2, Markus Gross 2, Leonidas J. Guibas 1 1 Stanford University 2 ETH, Zurich

2. Example Based 3D Shape Completion Shape Completion Incomplete raw scans Imperfect range scanned data Complex objects with occluded regions Misalignment of multiple-views depth image scans Specular highlights Ill-posed problem Use prior knowledge !!

3. Example Based 3D Shape Completion Diffusion (Davis, Marschner, Garr, Levoy, 2002)

4. Example Based 3D Shape Completion Context Based Solution (Sharf, Alexa, Cohen-or, 2004)

5. Example Based 3D Shape Completion Template based Solution (Allen, Curless, Popovic, 2003; Kraevoy and Sheffer, 2005)

6. Example Based 3D Shape Completion Our Solution Use 3D model database to provide geometric priors for shape completion Apply non-rigid transforms on the models More deformation  less likely completion Consistently combine geometric information from multiple context models Final result comes with confidence values

7. Example Based 3D Shape Completion Shape Completion Pipeline

8. Example Based 3D Shape Completion Shape Completion Pipeline

9. Example Based 3D Shape Completion Data Classification Local analysis quality of fit uniformity of sample distribution Scored Point Cloud confidence value assigned to each point High Low

10. Example Based 3D Shape Completion Shape Completion Pipeline

11. Example Based 3D Shape Completion Database Retrieval 1.931.711.461.271.0

12. Example Based 3D Shape Completion Shape Completion Pipeline

13. Example Based 3D Shape Completion Non-rigid Alignment Similar to the approaches proposed by: Allen, Curless and Popovic, 2003. Sumner and Popovic, 2004.

14. Example Based 3D Shape Completion Non-rigid Alignment Deformation Model Piecewise linear. Each vertex of the mesh assigned an independent displacement vector. Optimize for smallest Shape Matching Penalty Distortion Measure Geometric Error Feature Correspondence Derived in the continuous setting to allow consistent comparison between different context models.

15. Example Based 3D Shape Completion Warped Models High Low Context ModelWarped ModelMatching Penalty

16. Example Based 3D Shape Completion Non-rigid Transform

17. Example Based 3D Shape Completion Shape Completion Pipeline

18. Example Based 3D Shape Completion Initial Segmentation Input Data Warped Context Model

19. Example Based 3D Shape Completion Patch Growing Initial SegmentationFinal Segmentation

20. Example Based 3D Shape Completion Video

21. Example Based 3D Shape Completion Shape Completion Pipeline

22. Example Based 3D Shape Completion Blending Final Segments

23. Example Based 3D Shape Completion Result

24. Example Based 3D Shape Completion Shape Completion Pipeline

25. Example Based 3D Shape Completion Shape Completion Pipeline

26. Example Based 3D Shape Completion Segmentation Giraffe Example Context Models Deformed Models

27. Example Based 3D Shape Completion Final Model Giraffe Example Context Models Deformed Models

28. Example Based 3D Shape Completion Evaluation Input DataContext ModelFinal ModelEvaluation

29. Example Based 3D Shape Completion Enriching the Database

30. Example Based 3D Shape Completion Additional Constraints Physical ModelAcquired DataContext Model No Constraints Symmetry Constraints

31. Example Based 3D Shape Completion Galleria dell’ Accademia

32. Example Based 3D Shape Completion Galleria dell’ Accademia

33. Example Based 3D Shape Completion Galleria dell’ Accademia

34. Example Based 3D Shape Completion Galleria dell’ Accademia

35. Example Based 3D Shape Completion Future Directions Improve the retrieval stage. Automatic feature point detection. Use of more semantic information. Apply learning techniques to shape completion. Completion of additional attributes like surface texture, color.

36. Example Based 3D Shape Completion Acknowledgements NSF grants CARGO-0138456, ITR-0205671, FRG-0454543, ARO grant DAAD19-03-1-033. Stanford Graduate fellowship. Mario Botsch, David Koller, Doo Young Kwon, Marc Levoy, Filip Sadlo, Vin da Silva, and Bob Sumner.

37. Example Based 3D Shape Completion Questions

38. Example Based 3D Shape Completion

39. Correspondence High Input Data Warped Model Low Invalid Valid

40. Example Based 3D Shape Completion Video

41. Example Based 3D Shape Completion Example-based image completion [Drori et al. 2003; Jia and tang 2003; Sun et al. 2003] Texture synthesis [Efros and Leung 1999; Efros and Freeman 2001; Wei and Levoy 2000; Ying et al. 2001] Smooth surface completion [Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003] Model-based surface reconstruction [Savchenko et al. 2002] Curve analogies [Hertzmann et al. 2002] Drori et al. 03 Liepa 03 Wei and Levoy 00 Hertzmann et al. 02 Related Works