1. 3D Photography Szymon Rusinkiewicz Fall 2002

2. 3D Photography Obtaining 3D shape (and sometimes color) of real-world objectsObtaining 3D shape (and sometimes color) of real-world objects

3. Industrial Inspection Determine whether manufactured parts are within tolerancesDetermine whether manufactured parts are within tolerances

4. Medicine Plan surgery on computer model, visualize in real timePlan surgery on computer model, visualize in real time

5. Medicine

6. Medicine

7. Medicine

8. Scanning Buildings Quality control during buildingQuality control during building As-build modelsAs-build models

9. Scanning Buildings Quality control during buildingQuality control during building As-build modelsAs-build models

10. Clothing Scan a person, custom-fit clothingScan a person, custom-fit clothing U.S. Army; booths in mallsU.S. Army; booths in malls

11. Graphics Research Availability of complex datasets drives research (you wouldn’t believe how the poor bunny has been treated…)Availability of complex datasets drives research (you wouldn’t believe how the poor bunny has been treated…)

12. Sculpture Scanning The Pietà ProjectThe Pietà Project IBM Research The Digital Michelangelo ProjectThe Digital Michelangelo Project Stanford University The Great Buddha ProjectThe Great Buddha Project University of Tokyo

13. Why Scan Sculptures? Interesting geometryInteresting geometry Introduce scanning to new disciplinesIntroduce scanning to new disciplines – Art: studying working techniques – Art history – Cultural heritage preservation – Archeology High-visibility projectsHigh-visibility projects

14. Why Scan Sculptures? ChallengingChallenging – High detail, large areas – Large data sets – Field conditions – Pushing hardware, software technology But not too challengingBut not too challenging – Simple topology – Possible to scan most of surface

15. Issues Addressed ResolutionResolution CoverageCoverage – Theoretical: limits of scanning technologies – Practical: physical access, time Type of dataType of data – High-res 3D data vs. coarse 3D + normal maps – Influenced by eventual application Intellectual PropertyIntellectual Property

16. The Digital Michelangelo Project

17. Goals Scan 10 sculptures by MichelangeloScan 10 sculptures by Michelangelo High-resolution (“quarter-millimeter”) geometryHigh-resolution (“quarter-millimeter”) geometry Side projects: architectural scanning (Accademia and Medici chapel), scanning fragments of Forma Urbis RomaeSide projects: architectural scanning (Accademia and Medici chapel), scanning fragments of Forma Urbis Romae

18. Why Capture Chisel Marks? Atlas (Accademia) ugnettougnetto ? ?

19. Why Capture Chisel Marks? Day (Medici Chapel) 2 mm

20. Who? Faculty and staff Prof. Brian CurlessJohn Gerth Jelena JovanovicProf. Marc Levoy Lisa PacelleDomi Pitturo Dr. Kari Pulli Graduate students Sean AndersonBarbara Caputo James DavisDave Koller Lucas PereiraSzymon Rusinkiewicz Jonathan ShadeMarco Tarini Daniel Wood Undergraduates Alana ChanKathryn Chinn Jeremy GinsbergMatt Ginzton Unnur GretarsdottirRahul Gupta Wallace HuangDana Katter Ephraim LuftDan Perkel Semira RahemtullaAlex Roetter Joshua SchroederMaisie Tsui David Weekly In Florence Dottssa Cristina Acidini Dottssa Franca Falletti Dottssa Licia BertaniAlessandra Marino Matti Auvinen In Rome Prof. Eugenio La RoccaDottssa Susanna Le Pera Dottssa Anna SomellaDottssa Laura Ferrea In Pisa Roberto Scopigno Sponsors Interval ResearchPaul G. Allen Foundation for the Arts Stanford University Equipment donors CyberwareCyra Technologies Faro TechnologiesIntel Silicon GraphicsSony 3D Scanners

21. Scanner Design 4 motorized axes laser, range camera, white light, and color camera

22. Triangulation Project laser stripe onto objectProject laser stripe onto object Object Laser CameraCamera

23. CameraCamera Triangulation Depth from ray-plane triangulationDepth from ray-plane triangulation Laser (x,y) Object

24. Scanning a Large Object Calibrated motionsCalibrated motions – pitch (yellow) – pan (blue) – horizontal translation (orange) Uncalibrated motions – vertical translation – rolling the gantry – remounting the scan head

25. Single Scan of St. Matthew 1 mm

26. Statistics About the Scan of David 480 individually aimed scans480 individually aimed scans 0.3 mm sample spacing0.3 mm sample spacing 2 billion polygons2 billion polygons 7,000 color images7,000 color images 32 gigabytes32 gigabytes 30 nights of scanning30 nights of scanning 22 people22 people

27. Head of Michelangelo’s David Photograph 1.0 mm computer model

28. Side project: The Forma Urbis Romae

29. Forma Urbis Romae Fragment Forma Urbis Romae Fragment side face

31. IBM’s Pietà Project Michelangelo’s “Florentine Pietà”Michelangelo’s “Florentine Pietà” Late work (1550s)Late work (1550s) Partially destroyed by Michelangelo, recreated by his studentPartially destroyed by Michelangelo, recreated by his student Currently in the Museo dell’Opera del Duomo in FlorenceCurrently in the Museo dell’Opera del Duomo in Florence

32. Who? Dr. Jack Wasserman, professor emeritus of art history at Temple UniversityDr. Jack Wasserman, professor emeritus of art history at Temple University Visual and Geometric Computing group @ IBM Research:Visual and Geometric Computing group @ IBM Research: Fausto Bernardini Holly Rushmeier Ioana Martin Joshua Mittleman Gabriel Taubin Andre Gueziec Claudio Silva

33. Scanner Visual Interface “Virtuoso”Visual Interface “Virtuoso” Active multibaseline stereoActive multibaseline stereo Projector (stripe pattern), 6 B&W cameras, 1 color cameraProjector (stripe pattern), 6 B&W cameras, 1 color camera Augmented with 5 extra “point” light sources for photometric stereo (active shape from shading)Augmented with 5 extra “point” light sources for photometric stereo (active shape from shading)

34. Data Range data has 2 mm spacing, 0.1mm noiseRange data has 2 mm spacing, 0.1mm noise Each range image: 10,000 points, 20  20 cmEach range image: 10,000 points, 20  20 cm Color data: 5 images with controlled lighting, 1280  960, 0.5 mm resolutionColor data: 5 images with controlled lighting, 1280  960, 0.5 mm resolution Total of 770 scans, 7.2 million pointsTotal of 770 scans, 7.2 million points

35. Scanning Final scan June 1998, completed July 1999Final scan June 1998, completed July 1999 Total scanning time: 90 hours over 14 days (includes equipment setup time)Total scanning time: 90 hours over 14 days (includes equipment setup time)

36. Results

37. The Great Buddha Project Great Buddha of KamakuraGreat Buddha of Kamakura Original made of wood, completed 1243Original made of wood, completed 1243 Covered in bronze and gold leaf, 1267Covered in bronze and gold leaf, 1267 Approx. 15 m tallApprox. 15 m tall Goal: preservation of cultural heritageGoal: preservation of cultural heritage

38. Who? Institute of Industrial Science, University of TokyoInstitute of Industrial Science, University of Tokyo Daisuke Miyazaki Takeshi Ooishi Taku Nishikawa Ryusuke Sagawa Ko Nishino Takashi Tomomatsu Yutaka Takase Katsushi Ikeuchi

39. Scanner Cyrax range scanner by Cyra TechnologiesCyrax range scanner by Cyra Technologies Laser pulse time-of-flightLaser pulse time-of-flight

40. Pulsed Time of Flight Send out pulse of light (usually laser), time how long it takes to returnSend out pulse of light (usually laser), time how long it takes to return

41. Pulsed Time of Flight Advantages:Advantages: – Large working volume (up to 100 m.) Disadvantages:Disadvantages: – Not-so-great accuracy (at best ~5 mm.) Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds Does not scale with working volumeDoes not scale with working volume Often used for scanning buildings, rooms, archeological sites, etc.Often used for scanning buildings, rooms, archeological sites, etc.

42. Data 20 range images (a few million points)20 range images (a few million points) 4 mm accuracy4 mm accuracy Several nights of scanningSeveral nights of scanning

43. Results

44. Course Meets Mondays 11-12, Wednesdays 11-12:20Meets Mondays 11-12, Wednesdays 11-12:20 Class project – build a 3D photography system from the ground upClass project – build a 3D photography system from the ground up http://www.cs.princeton.edu/courses/597B/http://www.cs.princeton.edu/courses/597B/