A Lumigraph Camera for Image Based Rendering Jason C. Yang Prof. Leonard McMillan May 10, 1999

Overview Image Based Rendering Video Demo System Design Obstacles

Image Based Rendering Motivation - Geometry is hard. - Textures are easy. Light Field Rendering: Generate novel views using a database of “rays” from a 2D array of images.

State-of-the-art in CG Model Model + Shading Model + Shading + Textures At what point do things start looking real?

Rendering Process new viewpoint

Demo System

Goals Real Time Camera System Off the shelf components Desired frame rate: 30 frames per second 640 x 480 resolution

Interface Solution (PCI or Cardbus) Sensor Pod - A BAB C A C D B D C A C D B D Motherboard FIFO Address Data Overall Design

Sensor Pod Aramis Random Access CMOS Sensor (on board A/D) FPGA Logic CLK VRAM (for FPN) Address Data Address Data

Sensor Pod CMOS Sensor (on board A/D) FPGA Logic CLK VRAM Address Data Address Data

Major Hurdle Ideal frame rate is not achievable -Best Frame rate: 7fps Bottlenecks: - PCI bus - Turnaround time for one pixel Potential Solution: - Interleaving - FIFOs

Actual Frame Rate To reach desired 30fps we need: 37MBs Maximum Random Access on PCI: 33MBs Turn Around DataWait Address CLK Address/ Data 30ns Turn around time (time to access pixel from camera) is not one clock cycle!

Virtual vs. Physical Pixels Green Blue Red Green Blue GreenRedGreen 0,01,0 0,11,1 2,0 2,1 0,21,2 When the host requests a pixel it is actually a virtual pixel as indicated by the circles. The actual pixels requested are the color photocells on the imager. Each virtual pixel request will return a four byte group of color values.

Time to Access a Pixel Y0X0X1Y1X2X3 A/D (Y0,X0)(Y0,X1)(Y1,X2)(Y1,X3) 30ns CLK Address A/D Data 480ns 4 byte Pixel ready

Actual Transfer Rate Turn Around DataWait Address CLK Address/ Data 30ns ns450ns30ns = 540ns 540ns/pixel => 640x

Mux/ Logic Motherboard - Interleaving Address Sensor Pod - A Sensor Pod - B Mux/ Logic Data 32 bit Optimizations

Use PCI Burst Mode PCI burst mode can achieve 133MBs Turn Around Data Address CLK Address/ Data 30 ns Data Idea: Use FIFOs to store Addresses and Data

Conclusion Tradeoffs Technological Needs - Random Access CMOS imagers - Faster imagers - Faster bus protocols