Creating Adaptive Views for Group Video Teleconferencing – An Image-Based Approach Creating Adaptive Views for Group Video Teleconferencing – An Image-Based Approach Ruigang Yang Celso Kurashima Andrew Nashel Herman Towles Anselmo Lastra Henry Fuchs University of North Carolina at Chapel Hill University of North Carolina at Chapel Hill

International Workshop on Immersive Telepresence 2002Slide 2 Current Teleconferencing Capture Transport Display ?

International Workshop on Immersive Telepresence 2002Slide 3 The Office of the Future

International Workshop on Immersive Telepresence 2002Slide 4 Group Teleconferencing Multiple persons (3-4) at each siteMultiple persons (3-4) at each site Life-size, monoscopic displayLife-size, monoscopic display High-resolution seamless imageryHigh-resolution seamless imagery Active view controlActive view control Cameras

International Workshop on Immersive Telepresence 2002Slide 5 Active View Control Provide the best approximating view

International Workshop on Immersive Telepresence 2002Slide 6 Active View Control A view synthesis problem Extract 3D geometry from a few camerasExtract 3D geometry from a few cameras –Less expensive –Hard to get good results Image-based method: capture many imagesImage-based method: capture many images –Looks really good on every scene –Need many images A view synthesis problem Extract 3D geometry from a few camerasExtract 3D geometry from a few cameras –Less expensive –Hard to get good results Image-based method: capture many imagesImage-based method: capture many images –Looks really good on every scene –Need many images

International Workshop on Immersive Telepresence 2002Slide 7 Our Image-based Approach Observation: Eye level remains relatively the same during a conference sessionObservation: Eye level remains relatively the same during a conference session A compact Light Field representationA compact Light Field representation –Parameterized by a 3D function (s, u, v) Observation: Eye level remains relatively the same during a conference sessionObservation: Eye level remains relatively the same during a conference session A compact Light Field representationA compact Light Field representation –Parameterized by a 3D function (s, u, v) u v s t Focal Plane

International Workshop on Immersive Telepresence 2002Slide 8 Linear Light Field

International Workshop on Immersive Telepresence 2002Slide 9 LLF Rendering Projective Texture mapping and blendingProjective Texture mapping and blending –Tessellate the focal plane –Project input images onto the focal plane –View-dependent blending Projective Texture mapping and blendingProjective Texture mapping and blending –Tessellate the focal plane –Project input images onto the focal plane –View-dependent blending New view Base image Focal Plane

International Workshop on Immersive Telepresence 2002Slide 10 Blending Function Focal Plane

International Workshop on Immersive Telepresence 2002Slide 11 Samples Images Perspective Projection Orthogonal Projection (extreme case)

International Workshop on Immersive Telepresence 2002Slide 12 Sampling Analysis Configuration parameters – –Focal plane depth D – –Camera’s FOV – –Camera’s horizontal resolution W – –Inter-camera distance d Error term: pixel drift (e) Configuration parameters – –Focal plane depth D – –Camera’s FOV – –Camera’s horizontal resolution W – –Inter-camera distance d Error term: pixel drift (e) Given the configuration parameters, and a desired error tolerance e, what is the maximum depth deviation  D from the optimal depth D.

International Workshop on Immersive Telepresence 2002Slide 13 Sampling Analysis – Result

International Workshop on Immersive Telepresence 2002Slide 14 More results Distributed SystemDistributed System –11 cameras –6 capture PC ( 640x480) ROI encodedROI encoded JPEG compressionJPEG compression –One rendering PC Roughly 1000 x 480 outputRoughly 1000 x 480 output 4-7 frames per second4-7 frames per second Distributed SystemDistributed System –11 cameras –6 capture PC ( 640x480) ROI encodedROI encoded JPEG compressionJPEG compression –One rendering PC Roughly 1000 x 480 outputRoughly 1000 x 480 output 4-7 frames per second4-7 frames per second

International Workshop on Immersive Telepresence 2002Slide 15 Conclusions We presented a novel system designed specifically for group video- teleconferencing. Best approximate view for the groupBest approximate view for the group Photo-realistic results at interactive ratePhoto-realistic results at interactive rate Flexible and scaleableFlexible and scaleable We presented a novel system designed specifically for group video- teleconferencing. Best approximate view for the groupBest approximate view for the group Photo-realistic results at interactive ratePhoto-realistic results at interactive rate Flexible and scaleableFlexible and scaleable

International Workshop on Immersive Telepresence 2002Slide 16 Acknowledgements Funding support from The Department of Energy's ASCI VIEWS Program Sandia National Laboratories USA Collaborators from Sandia Phil Heermann Christine Yang Corbin Stewart Funding support from The Department of Energy's ASCI VIEWS Program Sandia National Laboratories USA Collaborators from Sandia Phil Heermann Christine Yang Corbin Stewart

International Workshop on Immersive Telepresence 2002Slide 17 The End Thank You