1. Copyright  Philipp Slusallek IBR: View Interpolation Philipp Slusallek

2. Copyright  Philipp Slusallek Image-Based Rendering Geometry-basedImage-based Panorama / Environment Map Sprite / Imposter Imposter / Sprite with Depth LDI Geometry View-dependent Texture Mapping Lightfield / Lumigraph

3. Copyright  Philipp Slusallek View Interpolation 1. Interpolating between range images - Chen and Williams, View Interpolation, 1993 - Shade et al., Layered Depth Images, 1998 2. Correspondences and epipolar analysis - Laveau and Faugeras, Collection of Images, 1994 - McMillan and Bishop, Plenoptic Modeling, 1995 3. Combination of view synthesis and morphing - Chen and Williams, 1993 - Seitz and Dyer, 1995  Require depths or correspondences

4. Copyright  Philipp Slusallek Drawing RGBZ Images Various techniques: 1. Priority sorted splatting (Gauss, small rectangles, or points) - Chen and Williams 2. Priority sorted height field - McMillan and Bishop 3. Micropolygon mesh - Marks et al. 4. Inverse ray-tracing of height field - McMillan 5. Decimate and draw as a polygon mesh

5. Copyright  Philipp Slusallek View Interpolation Issues: Visibility/Occlusion Holes (Anti-)aliasing Shading (diffuse)

6. Copyright  Philipp Slusallek Directly from correspondences Laveau and Faugeras Algorithm Existing View1 Existing View2 New View 3 p p1p1 p2p2 l 13 l 23 p3p3

7. Copyright  Philipp Slusallek Merging RGBZ Images Problems: Overlaps - Oversampling - Occlusion Gaps or holes - Undersampling - Expose unseen areas Chen and Williams 1993

8. Copyright  Philipp Slusallek View Interpolation [Chen/Williams’93] Preprocessing: Correct correspondences for pixel (morph map) Division into blocks (hierarchical) Obtain priorities form Z filtering Display: Linear interpolation of morph map Back-to-front rendering of blocks Fill holes from adjacent pixels

9. Copyright  Philipp Slusallek Sprites / Imposters Purpose: Cache complex rendered geometry Decouple rendering updates from image updates Drawing primitives Issues: Placement of polygon Warping of texture (perspective, affine) Validity of cache Hierarchy Continuity with environment

10. Copyright  Philipp Slusallek Sprites / Imposters

11. Copyright  Philipp Slusallek Imposters

12. Copyright  Philipp Slusallek Sprites/Imposters with Depth Better image warping: Wider range of reuse Backward mapping only with homograph New mapping: Stored depth map Forward map depth map (approximate geometry) Backward mapping of color using depth information d d’

13. Copyright  Philipp Slusallek Mapping with Depth Forward Mapping: Holes and aliasing I 1 d 1 (I 2 ) I 2 

14. Copyright  Philipp Slusallek Mapping with Depth Backward Mapping: What is d? I 1 (I 2 ) d 2 I 2 

15. Copyright  Philipp Slusallek Mapping with Depth Solution: Forward map depth Reconstruct approximate geometry Backward map color I 1 (I 2 ) d 2 I 2 

16. Copyright  Philipp Slusallek Layered Depth Images Idea: Handle disocclusion Store invisible geometry in depth images Data structure: Per pixel list of depth samples Per depth sample: RGBA Z Encoded: Normal direction, distance Pack into cache lines

17. Copyright  Philipp Slusallek Layered Depth Images Computation: Incremental warping computation Implicit ordering information Process in up to four quadrant Splat size computation Table lookup Fixed splat templates Clipping of LDIs

18. Copyright  Philipp Slusallek Layered Depth Images

19. Copyright  Philipp Slusallek Micro-Polygon rendering Goal: Fill Holes by assuming smooth geometry Approach: Connect neighboring pixels with polygons Delete polygons for very large steps Render micro-polygons  Heavy load on graphics system

20. Copyright  Philipp Slusallek Some more Application: Stereo Example courtesy L. McMillan, MIT

21. Copyright  Philipp Slusallek Some More Applications Holography Generating new views of models Decoupling of rendering and image updates Z is available Avoid artifacts Motion Capture Matching of model to image data Morphing for flexible models Video Rewrite / Talking Head

22. Copyright  Philipp Slusallek From Michael Cohen

23. Copyright  Philipp Slusallek From Michael Cohen

24. Copyright  Philipp Slusallek From Michael Cohen

25. Copyright  Philipp Slusallek Image-Based Rendering Limitations Static scene geometry Fixed lighting Constrained look-from or look-at point Possibilities: Complete modeling and rendering system? Integration of Computer Vision and Computer Graphics?