1. Proxy Plane Fitting for Line Light Field Rendering Presented by Luv Kohli COMP238 December 17, 2002

2. Group Teleimmersion Project in UNC’s Office of the Future group Goal: create a system for many-to- many video teleconferencing –Attempt to provide best approximate view for all participants while maintaining sense of presence

3. Approach Use light field rendering techniques to create novel views of participants Participant’s viewpoint tends to be limited to eye level with small lateral motions –Motivation for using a 1D linear array of cameras instead of a full 2D setup

4. Line Light Field Less data required, so real-time capture, transmission, and rendering possible

5. Multiple focal planes Current system only allows one focal plane –Background out of focus –Participants out of focus if they move away from focal plane Use plane fitting and multiple focal planes instead

7. Plane fitting Determine a focal plane for each participant using stereo correlation of silhouette points Epipolar constraint: (u,v) C1C1 C2C2

8. Plane fitting (2) Segment foreground objects out Find silhouette points Use stereo correlation of silhouette points to find approximate plane –Silhouette point in one image corresponds to epipolar line in second image

9. Multiple focal planes Use techniques described in Dynamically Reparameterized Light Fields (Isaksen, McMillan, Gortler) paper Conceptually, shoot a ray through light field and determine which focal planes it intersects Choose focal plane closest to geometry in scene – use scoring function σ

10. Multiple focal planes: scoring

11. Focal plane on participants

12. Focal plane on background

13. Focal plane on front objects

14. Multiple focal planes

17. Issues and future work Implementation issues Will plane-fitting and multiple focal planes be viable in a real-time networking environment? Work can be done in parallel Will this enhance the feeling of immersion?

18. References "Creating Adaptive Views for Group Video Teleconferencing -- An Image-Based Approach " Ruigang Yang, Celso Kurashima, Andrew Nashel, Herman Towles, Anselmo Lastra, Henry Fuchs. Presented at International Workshop on Immersive Telepresence, December 6, 2002, Juan Les Pins, France. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen. The Lumigraph. In Proceedings of SIGGRAPH 1996, pages 43-54, New Orleans, August 1996.

19. References M. Levoy and P. Hanrahan. Light Field Rendering. In Proceedings of SIGGRAPH 1996, pages 31-42, New Orleans, August 1996. A. Isaksen, L. McMillan, and S. J. Gortler. Dynamically reparameterized light fields. Technical Report LCS-TR- 778, Massachusetts Institute of Technology, May 1999.

20. References C. Kurashima, R. Yang, A. Lastra. Combining Approximate Geometry with View-Dependent Texture Mapping – A Hybrid Approach to 3D Video Teleconferencing. Presented at SIBGRAPI 2002, XV Brazilian Symposium on Computer Graphics and Image Processing (07 – 10 October 2002, Fortaleza, CE, Brazil). L. McMillan and Gary Bishop. Plenoptic Modeling: An Image-Based Rendering System. In Proceedings of SIGGRAPH 1995, pages 39- 46, 1995.