Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective

Perspective Projection Humans evolved with perspective eyes Capture light along rays that converge at a single point

Cameras also evolved with perspective optics Optimized for humans, not computers!

A non-perspective image

The Blue Marble, NASA satellite image composite

Cyberware Scanner

Print Gallery, by M.C. Escher, 1956

Panorama from Disney’s 1940 film Pinocchio (from Wood et al., SIGGRAPH 1997)

What’s an Image? An image is any 2D subset of rays in space Actually, the light energy flowing along these rays Any 2D “slice” of the plenoptic function perspective image general image

Non-perspective Imaging Issues: What other types of images are possible? Which images are useful? How can we capture these images?

Path Images

x y t

x y t

Pushbroom images satellite Bolles et al. [87] EPI Tsuji et al. [92] omni-directional image Peleg et al. [97] manifold mosaic Radamacher & Bishop [99] MCOP... x y t

Linear Path input pushbroom images Video Cube Demo application by Michael Cohen et al., Microsoft

Circular Path panorama (“concentric mosaic”) circular EPI input image

Circular Path inputcyclographsvideo cube

What are these images good for? Applications to computer graphics, computer vision?

Circular Stereo [Ishiguro, Yamamoto, Tsuji, 92] [Peleg and Ben-Ezra, 99] [Shum, Kalai, Seitz, 99] [Nayar and Karmarkar, 00] I y x  x 0 +n x 0 -n x0x0

Stereo Panorama Disparity map result

Stereo Panorama Dark--close, light--far

Circular Stereo Advantages 360 degree scene reconstruction Uniform accuracy, optimal

Pushbroom Stereo y x t x 0 +n x 0 -n x0x0

Stereo Cyclographs y x  x 0 +n x 0 -n x0x0

Stereo Cyclograph Reconstruction Computed from two cyclograph images Using unmodified stereo matcher [Zitnick & Kanade]

Stereo Path Images Do these all produce stereo pairs? two images with horizontal parallax Yes

Stereo Path Images How about this path? No Yes if the camera path is a conic –line, circle, ellipse, parabola, hyperbola Must capture rays lying on doubly-ruled quadrics –[Padja 2001], [Seitz 2001]

Stereo Parabolic Panoramas perspective image parabolic panorama

Beyond Perspective Cameras for humans, not machines! Need to rethink cameras Image should suit the task Future: cameras will evolve like CPU’s First: task-specific cameras Then: programmable cameras –FPGA  programmable camera arrays Thanks to Jiwon Kim, Michael Cohen