Rendering with Concentric Mosaics Heung – Yeung Shum and Li – Wei He Presentation By: Jonathan A. Bockelman

Agenda 1)A general description of concentric mosaics 2)Rendering concentric mosaics 3)Capturing concentric mosaics 4)Some examples 5)Issues that still need to be resolved and future plans 6)A brief demo

Rendering Made Easy... Sort of Boo! Yay!  Problems with traditional rendering schemes  The appeal of image-based modeling and rendering  The plenoptic function

History of Plenoptic Functions

What is a Concentric Mosaic?  “A manifold mosaic”  A 3D plenoptic: radius, rotation angle, and vertical elevation  A 3D image built from a series of 360° slit images

Rendering a Novel View  Any point within the outermost circle can be the viewpoint  Rays tangent to the camera paths are used  Bilinear interpolation between neighboring mosaics can also be used

The Problem of Non-Planar Rays  Rays off the plane need to be approximated  Objects assumed to have an infinite depth  Vertical distortion is created

The Need for Depth Correction  Depth correction can fix the vertical distortion  3 types of depth correction exist

Full Perspective Correction  Individual corrections are made for each pixel  Exact depths of objects are necessary  Hole-filling problems are a complication  Excellent results are seen in synthetic scenes

Weak Perspective Correction  Corrections are made for each vertical line  Estimated depths are calculated  Vertical distortions can occur

Constant Depth Approximation  A constant depth is used  Users can control the assumed depth  Vertical distortions are produced if the wrong depth is given

Consequences of a 3D function  Vertical parallax is not captured  Much smaller data sets are required  Users can move in a circular region

Synthetic Mosaics  3D Studio Max can be used  Images are cut into slits  Depth values for each pixel can be found  Sampling is a bit tricky

How NOT to Do Real World Scenes  A series of single-slit cameras on a rotating beam  A single camera that can slide along a beam

The Lone Camera  A single off-centered camera sits on a rotary table  Regular images are taken  Multiple concentric mosaics can be recreated from one image

Ideal Solution  A single camera can produce distortion  A few tangential cameras along a beam can correct the problem

How the Pros Do It  An single ordinary digital video camera is used with a rotary table  The camera faces radially outward  1351 frames are captured in 90 seconds  The system is incredibly simple and efficient

The Lobby Scene 3 concentric mosaics from a lobby scene

Occlusion Occlusion is captured.

Horizontal Parallax Horizontal parallax is simulated quite well.

Lighting and Glare Spectacular lighting effects are easy to create.

Constant Depth Correction Revisited  Aspect ratios are maintained at the chosen depth  Objects at other depths are distorted

Point vs. Bilinear Sampling  Point sampling is twice as fast, but image quality is lower  Bilinear sampling is slower, but images are much smoother

Compression  Since adjacent frames are very similar, a majority of the data can be compressed.  Vector quantization and entropy coding allow the 415Mb original video to be shrunk to 16Mb.  MPEG4 compression can reduce the data size to 640k, but blocky artifacts are created.

Why Use Concentric Mosaics?  Quick and easy image capture  Parallax and specular highlights are preserved  Much smaller data sets than Lumigraphs  No messy geometry and lighting  User interaction is automatically incorporated

Future Endeavors  Correcting vertical distortion  Increasing the region of motion  Improving compression ratios

One Last Example

Demo

Mathematical Madness