1. 3D photography Marc Pollefeys Fall 2008 http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/

2. 3D Photography Obtaining 3D shape (and appearance) of real-world objects

3. Motivation Applications in many different areas A few examples …

4. Surgery - simulation simulate results of surgery allows preoperative planning

5. Surgery - teaching Capture models of surgery for interactive learning

6. Clothing Scan a person, custom-fit clothing

7. Forensics Crime scene recording and analysis

8. Forensics

9. 3D urban modeling UNC/UKY UrbanScape project

10. Industrial inspection Verify specifications Compare measured model with CAD

11. Scanning industrial sites as-build 3D model of off-shore oil platform

12. Robot navigation small tethered rover pan/tilt stereo head ESA project our task: Calibration + Terrain modelling + Visualization

13. Architecture Survey Stability analysis Plan renovations

14. Architecture Survey Stability analysis Plan renovations

15. Cultural heritage Virtual Monticello Allow virtual visits

16. Cultural heritage Stanford’s Digital Michelangelo Digital archive Art historic studies

17. Archaeology accuracy ~1/500 from DV video (i.e. 140kb jpegs 576x720)

18. Record different excavation layers Archaeology Generate & verify construction hypothesis Layer 1 Layer 2 Generate 4D excavation record

19. Computer games

20. Course objectives To understand the concepts that allow to recover 3D shape from images Explore the state of the art in 3D photography Implement a 3D photography system/algorithm

21. Material Slides and more http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/ Also check out on-line “shape-from-video” tutorial: http://www.cs.unc.edu/~marc/tutorial.pdf http://www.cs.unc.edu/~marc/tutorial/ Other interesting stuff: Book by Hartley & Zisserman, Multiple View Geometry Peter Allen’s 3D photography class webpage http://www1.cs.columbia.edu/~allen/F07/schedule.html http://www1.cs.columbia.edu/~allen/F07/schedule.html

22. Learning approach Lectures: cover main 3D photography concepts and approaches. Exercice sessions: Assignments (1 st half semester) Paper presentations and project updates (2 nd half semester) 3D photography project: Choose topic, define scope (1 st half semster) Implement algorithm/system (2 nd half semester) Grade distribution Assignments & paper presentation: 50% Course project: 50%

23. Content camera model and calibration single-view metrology triangulation epipolar geometry, stereo and rectification structured-light, active techniques feature tracking and matching structure-from-motion shape-from-silhouettes space-carving …

24. 3D photography course schedule (tentative) LectureExercise Sept 17Introduction- Sept 24Geometry & Camera modelCamera calibration Oct. 1Single View MetrologyMeasuring in images Oct. 8Feature Tracking/matching (Jan-Michael Frahm?) Correspondence computation Oct. 15Epipolar Geometry (David Gallup?) F-matrix computation Oct. 22Shape-from-SilhouettesVisual-hull computation Oct. 29Stereo matchingProject proposals Nov. 5Structured light and active range sensing (TBD) Papers Nov. 12Structure from motionPapers Nov. 19Multi-view geometry and self-calibration Papers Nov. 26Shape-from-XPapers Dec. 33D modeling and registrationPapers Dec. 10Appearance modeling and image-based rendering (TBD) Papers Dec. 17Final project presentations

25. Fast Forward! Quick overview of what is coming…

26. Camera models and geometry Pinhole camera Geometric transformations in 2D and 3D or

27. Camera Calibration Know 2D/3D correspondences, compute projection matrix also radial distortion (non-linear)

28. Single View Metrology

29. Antonio Criminisi

30. Feature tracking and matching Harris corners, KLT features, SIFT features key concepts: invariance of extraction, descriptors to viewpoint, exposure and illumination changes

31. l2l2 3D from images C1C1 m1m1 M? L1L1 m2m2 L2L2 M C2C2 Triangulation - calibration - correspondences

32. Epipolar Geometry Fundamental matrix Essential matrix Also how to robustly compute from images

33. Stereo and rectification Warp images to simplify epipolar geometry Compute correspondences for all pixels

34. Structured-light Projector = camera Use specific patterns to obtain correspondences

35. Initialize Motion (P 1,P 2 compatibel with F) Initialize Structure (minimize reprojection error) Extend motion (compute pose through matches seen in 2 or more previous views) Extend structure (Initialize new structure, refine existing structure) Structure from motion

36. ** ** projection constraints Auto-calibration

37. Shape-from-Silhouette

38. Space-carving Only keep photo-consistent voxels

39. Shape-from-X Shape-from-focus Shape-from-texture time-of-flight

40. 3D modeling and texturing Multiple depth imagesSurface model Texture integration patchwork texture map

41. Papers and discussion Will cover recent state of the art Each student will present a paper, discussion Select paper related to project

42. Course project: Build your own 3D scanner! Example: Bouguet ICCV’98 Students can work in group or alone

43. 3D photography course team Marc Pollefeys CAB F66 marc.pollefeys@inf.ethz.edu Tel. 044 63 23105 David Gallup CAB F64.2 gallup@cs.unc.edu Tel. 044 23061