1. Image Based Rendering

2. Light Field Gershun in 1936 –An illuminated objects fills the surrounding space with light reflected of its surface, establishing the object ’ s “ light field ” Adelson and Bergen 1991 –Plenoptic Function

3. Plenoptic Funtions An observer at any point in space (x,y,z) and any moment in time (t) can look in any direction(θ,ǿ) and record the incoming light intensity Φ at all visible wavelength λ Φ= Φ(x,y,z,t, θ,ǿ, λ) Φ is the radiation’s spectral intensity, a scalar value denotes the incoming light’s power per unit per area per solid angle per unit wavelength interval

4. Image Based Rendering If Φis known, any view of the scene can be reconstructed IBR an approach to computer-generated 2D-views of natural 3D-scenes Approximation of the plenoptic functions from a number of conventional images

5. IBR Topics View-Dependent Texture Mapping Hybrid Geometry and Image Based Rendering Delta Tree Representation Multiple-Centre-of-Projection Images Layered Depth Images Sprites with Depth Plenoptic Modeling and Environment Maps Light Field Rendering and Lumigraphs

6. Light Field Rendering M. Levoy A ray is parameterized by its intersection coordinates (u,v,s,t) The set of all rays that can be parameterized by two grid planes represents a light field slab

7. Light Field Rendering

8. Sampling Requirements S X T = 256 x 256 U X V = 121000 images

9. IBR with Controllable Illumination For a static scene, 3 or more images with “ orthogonal ” lighting can be used to render all possible lighting conditions

10. Set of all possible images Original 6 Photos with different lighting conditions Three basis images

11. Set of all possible images a

12. IBR and Video Scene with Camera Motion A video scene with camera motion contains a sequence of images of different viewing positions Applying IBR or IBM, develop algorithm to do video object segmentation Object elimination inside a video sequence, like the one in “ Rising Sun ” - Sean Connery Any help to video understanding ? Evolution of Vision ?