1. Copyright  Philipp Slusallek Cs448.98.fall IBR: Model-based Methods Philipp Slusallek

2. Copyright  Philipp Slusallek Cs448.98.fall Modeling and Rendering Traditional Pipeline: Modeling is hard Geometry: measurements, plans, user input Appearance: BRDF, texture Rendering is hard Complexity, reflection, lighting User inputModel: Geometry + MaterialImages ModelingRendering

3. Copyright  Philipp Slusallek Cs448.98.fall Modeling and Computer Vision Computer Vision: Modeling from images Images contain geometric and appearance information Model Reconstruction is hard ImagesImage-based Model Calibration & Registration Accurate Model Images Model Reconstruction Rendering Image-based Rendering

4. Copyright  Philipp Slusallek Cs448.98.fall Model Representation Representations: Geometry & Material Geometry & Textures Images with Depth (Range images, LDIs) Lightfield/Lumigraph Panorama Image-basedGeometry-based

5. Copyright  Philipp Slusallek Cs448.98.fall Importance of Geometry

6. Copyright  Philipp Slusallek Cs448.98.fall Image-based Rendering Advantages: Any geometry Photo-realistic: appearance is available Lower complexity Rendering is faster (?) Disadvantages: Sampled representation Visibility Data size Instability of CV algorithms

7. Copyright  Philipp Slusallek Cs448.98.fall Model-based IBR Basic Idea: Sparse set of images [Debevec’97, Pulli’96] Overview: Approximate Modeling Photogrammetric modeling Triangulated depth maps View-dependent Texture Mapping Weighting Hardware accelerated rendering Model-based Stereo Details from stereo algorithms

8. Copyright  Philipp Slusallek Cs448.98.fall Hybrid Approach Courtesy: P. Debevec

9. Copyright  Philipp Slusallek Cs448.98.fall Approximate Modeling User-assisted photogrammetry [Debevec ‘97]: Based on “Structure from Motion” Use constraints in architectural models Approach: Simple block model Constraints reduce DOF Matching based on lines Non-linear optimization Initial Camera Positions

10. Copyright  Philipp Slusallek Cs448.98.fall Approximate Modeling: Block Model Courtesy: P. Debevec

11. Copyright  Philipp Slusallek Cs448.98.fall Approximate Modeling Active Light: Calibrated camera and projector Plane of light and triangulation Registration of multiple views Triangulation of point cloud Projector Camera

12. Copyright  Philipp Slusallek Cs448.98.fall Approximate Modeling

13. Copyright  Philipp Slusallek Cs448.98.fall Projecting Images Technique: Known camera positions Projective texture mapping Shadow buffer for occlusions Blending between textures Filling in

14. Copyright  Philipp Slusallek Cs448.98.fall Visibility

15. Copyright  Philipp Slusallek Cs448.98.fall Projecting Images

16. Copyright  Philipp Slusallek Cs448.98.fall Projecting Images Simple compositing vs. blending Blending: Select “best” image closeness to viewing direction distance to border sampling density [Pulli] deletion of features Some computation in HW Smooth transition between pixels and frames Alpha blending, soft Z-buffer, confidence

17. Copyright  Philipp Slusallek Cs448.98.fall Projecting Images Closeness to viewing direction: Triangulate the Hemisphere Delaunay triangulation of viewing directions Regular triangulation: label each vertex with best view Interpolate based on barycentric coordinates

18. Copyright  Philipp Slusallek Cs448.98.fall Blending of Textures

19. Copyright  Philipp Slusallek Cs448.98.fall Model-Based Stereo Problems with conventional stereo algorithms: Correspondences are difficult to find Large disparities Foreshortening, projective distortions Approach: Use approximate geometry to reproject one image Compute disparity of warped image Significant smaller disparity and foreshortening

20. Copyright  Philipp Slusallek Cs448.98.fall Model-Based Stereo

21. Copyright  Philipp Slusallek Cs448.98.fall Model-Based Stereo

22. Copyright  Philipp Slusallek Cs448.98.fall Model-Based Stereo

23. Copyright  Philipp Slusallek Cs448.98.fall Demos