1. NVIDIA Lecture 10 Copyright  Pat Hanrahan Image-Based Rendering: 1st Wave Definition: Using images to enhance the realism of 3D graphics Brute Force in Image-Space, L. Williams Texture maps for surface detail Mipmapping (antialiasing) Zbuffers for hidden surfaces Shadow mattes and maps Object tags

2. NVIDIA Lecture 10 Copyright  Pat Hanrahan Image-Based Rendering: 2nd Wave Represent (model and render) environments without using traditional geometric representations (polygons) Examples QuickTimeVR and panoramic image representations Light fields or virtual holograms Color + Range images Coupling viewing and morphing Advantages Cost independent of scene complexity Imagery from real or virtual scenes Modest computation compared to classical C.G.

3. NVIDIA Lecture 10 Copyright  Pat Hanrahan Applications Capture 3D environments MIT’s ASPEN MovieMap, Cyan’s MYST and RIVEN Museums, real estate, products, sets, planets Interactive 3D Graphics and Games Representations of complex environments (lod) Background environments Textured portals 3D sprites, billboards and imposters Post rendering warps Frame interpolation, priority rendering (Talisman) Latency compentation: HMDs, networked telepresence Stereo from single RGBZ image

4. NVIDIA Lecture 10 Copyright  Pat Hanrahan Apple’s QuickTime VR OutwardInward

5. NVIDIA Lecture 10 Copyright  Pat Hanrahan History of Panoramas Robert Barker’s Panorama (1792) 50’ high, 400’ circumference, platform for viewing Raoul Brimoin-Sanson’s Cineorama (1897) 10 synchronized projectors, 300’ circumference Disney’s CircleVision 9 35mm projectors Mirrored panoramic camera Modern IMAX and OMNIMAX theaters

6. NVIDIA Lecture 10 Copyright  Pat Hanrahan Panoramic Cameras Rotating Cameras Kodak Cirkut Globuscope Stationary Cameras Be Here

7. NVIDIA Lecture 10 Copyright  Pat Hanrahan Mars Pathfinder Panorama

8. NVIDIA Lecture 10 Copyright  Pat Hanrahan Creating a Cylindrical Panorama From www.quicktimevr.apple.com

9. NVIDIA Lecture 10 Copyright  Pat Hanrahan Stitching Prewarping Lens Distortion, Radiometry, Cylindrical projection Image Registration Feature alignment, Pixel difference minimization, Fourier correlation Global Alignment Gap closing Mutual image aligment Compositing Blending, Nearest Neighbor Example courtesy James Davis

10. NVIDIA Lecture 10 Copyright  Pat Hanrahan Commercial Products QuickTime VR, LivePicture, IBM (Panoramix) VideoBrush IPIX (PhotoBubbles), Be Here, etc.

11. NVIDIA Lecture 10 Copyright  Pat Hanrahan Future Directions Geometry + panoramas Hot spots Hierarchical panoramas Stereo / depth-mapped panoramas Video panoramas

12. NVIDIA Lecture 10 Copyright  Pat Hanrahan Light Field Key Ideas: 4D function - Valid outside convex hull 2D slice = image - Insert to create - Extract to display Inward or outward

13. NVIDIA Lecture 10 Copyright  Pat Hanrahan Two Plane Parametrization Object Focal plane (st) Camera plane (uv)

14. NVIDIA Lecture 10 Copyright  Pat Hanrahan

15. NVIDIA Lecture 10 Copyright  Pat Hanrahan Reconstruction

16. NVIDIA Lecture 10 Copyright  Pat Hanrahan Original prototype gantry Issues: focal plane, aperture, support, lighting

17. NVIDIA Lecture 10 Copyright  Pat Hanrahan New light field gantry Applications: digitizing light fields measuring BRDFs range scanning Designed by Marc Levoy et al.

18. NVIDIA Lecture 10 Copyright  Pat Hanrahan Light Field Camera prototype (32 z 2) long-term goal (32 z 32) light field imaging chip compression chip Concept: Pat Hanrahan Mark Horowitz Marc Levoy

19. NVIDIA Lecture 10 Copyright  Pat Hanrahan Dayton Taylor’s Multiple Camera Rig