Image-Based Rendering Produce a new image from real images. Combining images Interpolation More exotic methods.

Slides:

Advertisements

Similar presentations

3D Modeling from a photograph

Advertisements

Light Fields PROPERTIES AND APPLICATIONS. Outline  What are light fields  Acquisition of light fields  from a 3D scene  from a real world scene 

Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the.

Acquiring the Reflectance Field of a Human Face Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, Mark Sagar Haarm-Pieter Duiker,

Light Field Rendering Shijin Kong Lijie Heng.

Week 9 - Wednesday.  What did we talk about last time?  Fresnel reflection  Snell's Law  Microgeometry effects  Implementing BRDFs  Image based.

Computer Vision Optical Flow

Dana Cobzas-PhD thesis Image-Based Models with Applications in Robot Navigation Dana Cobzas Supervisor: Hong Zhang.

Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi

Announcements Quiz Thursday Quiz Review Tomorrow: AV Williams 4424, 4pm. Practice Quiz handout.

View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.

 Marc Levoy History of computer graphics CS Introduction to Computer Graphics Autumn quarter, 2006 Slides for September 26 lecture.

CS 223b 1 More on stereo and correspondence. CS 223b 2 =?f g Mostpopular For each window, match to closest window on epipolar line in other image. (slides.

Image Stitching and Panoramas

Copyright  Philipp Slusallek IBR: View Interpolation Philipp Slusallek.

Image or Object? Michael F. Cohen Microsoft Research.

 Marc Levoy History of computer graphics CS Introduction to Computer Graphics Autumn quarter, 2003 Slides for September 25 lecture.

Siggraph’2000, July 27, 2000 Jin-Xiang Chai Xin Tong Shing-Chow Chan Heung-Yeung Shum Microsoft Research, China Plenoptic Sampling SIGGRAPH’2000.

Computational Photography Light Field Rendering Jinxiang Chai.

University of New Mexico

Object recognition under varying illumination. Lighting changes objects appearance.

Matching Compare region of image to region of image. –We talked about this for stereo. –Important for motion. Epipolar constraint unknown. But motion small.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

Texture Mapping Mohan Sridharan Based on slides created by Edward Angel 1 CS4395: Computer Graphics.

CS 563 Advanced Topics in Computer Graphics View Interpolation and Image Warping by Brad Goodwin Images in this presentation are used WITHOUT permission.

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

 Marc Levoy History of computer graphics CS Introduction to Computer Graphics Autumn quarter, 2001 Slides for September 27 lecture.

CS 563 Advanced Topics in Computer Graphics Introduction To IBR By Cliff Lindsay Slide Show ’99 Siggraph[6]

View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.

Computer Graphics Inf4/MSc Computer Graphics Lecture 11 Texture Mapping.

Light Field. Modeling a desktop Image Based Rendering  Fast Realistic Rendering without 3D models.

The Uncanny Valley in 3D Modeling Steve Seitz and Rick Szeliski BIRS Workshop on Computer Vision and the Internet August 31, 2009.

1 Texturing. 2 What is Texturing? 3 Texture Mapping Definition: mapping a function onto a surface; function can be:  1, 2, or 3D  sampled (image) or.

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am.

Texture Mapping. 2 The Limits of Geometric Modeling Although graphics cards can render over 10 million polygons per second, that number is insufficient.

Computer Graphics Inf4/MSc Computer Graphics Lecture 7 Texture Mapping, Bump-mapping, Transparency.

Image-Based Rendering. 3D Scene = Shape + Shading Source: Leonard mcMillan, UNC-CH.

Video Textures Arno Schödl Richard Szeliski David Salesin Irfan Essa Microsoft Research, Georgia Tech.

Dynamically Reparameterized Light Fields Aaron Isaksen, Leonard McMillan (MIT), Steven Gortler (Harvard) Siggraph 2000 Presented by Orion Sky Lawlor cs497yzy.

Computational photography CS4670: Computer Vision Noah Snavely.

Proxy Plane Fitting for Line Light Field Rendering Presented by Luv Kohli COMP238 December 17, 2002.

Y. Moses 11 Combining Photometric and Geometric Constraints Yael Moses IDC, Herzliya Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion.

Image-based rendering Michael F. Cohen Microsoft Research.

CS 638, Fall 2001 Today Project Stage 0.5 Environment mapping Light Mapping.

Image-based Rendering. © 2002 James K. Hahn2 Image-based Rendering Usually based on 2-D imagesUsually based on 2-D images Pre-calculationPre-calculation.

Stereo Many slides adapted from Steve Seitz.

Lighting affects appearance. How do we represent light? (1) Ideal distant point source: - No cast shadows - Light distant - Three parameters - Example:

Image Based Rendering. Light Field Gershun in 1936 –An illuminated objects fills the surrounding space with light reflected of its surface, establishing.

Computing & Information Sciences Kansas State University Lecture 10 of 42CIS 636/736: (Introduction to) Computer Graphics CIS 636/736 Computer Graphics.

CSL 859: Advanced Computer Graphics Dept of Computer Sc. & Engg. IIT Delhi.

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm.

The Reason Tone Curves Are The Way They Are. Tone Curves in a common imaging chain.

112/5/ :54 Graphics II Image Based Rendering Session 11.

Image-Based Rendering Geometry and light interaction may be difficult and expensive to model –Think of how hard radiosity is –Imagine the complexity of.

CS559: Computer Graphics Lecture 36: Raytracing Li Zhang Spring 2008 Many Slides are from Hua Zhong at CUM, Paul Debevec at USC.

Video Textures Arno Schödl Richard Szeliski David Salesin Irfan Essa Microsoft Research, Georgia Tech.

Texturing CMSC 435/ What is Texturing? 2 Texture Mapping Definition: mapping a function onto a surface; function can be: – 1, 2, or 3D – sampled.

1© 2009 Autodesk Hardware Shade – Presenting Your Designs Hardware and Software Shading HW Shade Workflow Tessellation Quality Settings Lighting Settings.

University of New Mexico

Buffers and texture mapping

Computer Graphics Texture Mapping

CS 4501: Introduction to Computer Vision Sparse Feature Detectors: Harris Corner, Difference of Gaussian Connelly Barnes Slides from Jason Lawrence, Fei.

Image-Based Rendering

Merle Norman Cosmetics, Los Angeles

© 2005 University of Wisconsin

VOLUMETRIC VIDEO // PLENOPTIC LIGHTFIELD // MULTI CAMERA METHODOLOGIES

CS5500 Computer Graphics May 29, 2006

Texture Mapping Ed Angel Professor Emeritus of Computer Science

Introduction to Ray Tracing

Presentation transcript:

Image-Based Rendering Produce a new image from real images. Combining images Interpolation More exotic methods

Why Image-Based Rendering? What’s the most realistic image? A photograph. But photographs lack flexibility. Can’t change viewpoint. Can’t change lighting.

The need for correspondence Image-based rendering is mostly combining images to get a new image. Correspondences needed to sensibly combine images. If viewpoint has changed this can be hard. If not, it’s trivial.

How to get correspondences By hand: Works if few correspondences needed By matching intensities This is really ~ ½ of computer vision.

Matching Simplest: SSD with windows. Windows needed because pixels not informative enough. Compare windows =?f g Mostpopular Search for windows that match well

Mosaics Take multiple images and construct one big image. Represented as image, cylinder or sphere. Allows panning and zooming. Simplest kind of motion.

Fixed focal point. Correspondence needed to align images. Image rectification

(Images in paper by Szeliski and Shum, linked to on web page)

Other mosaicing issues Pixel interpolation needed. Mosaicing can provide more information at each pixel. Higher resolution images possible. Higher dynamic range.

Morphing What happens if you interpolate images? Need corresponding points.

Morphing Corresponding points needed. Often done by hand. Interpolate each point. Position and intensity. Also use interpolation for more correspondences.

Linear Interpolation of Position

Other Interpolation Also interpolate intensities. Interpolate to find other point correspondes.

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Interpolation We have possibly non-uniform samples of a 4D space. Must interpolate to fill in. Worry about aliasing

Light Field Rendering (Levoy and Hanrahan; paper and slides) (Images from Marc Levoy’s slides:

Linear basis for lighting Z Y X Surface normal = (X,Y,Z) albedo = Directional source = (Lx,Ly,Lz) I = (Lx,Ly,Lz)(X,Y,Z) = Lx* X + Ly* Y + Lz* Z Take Max of this and 0

Using Linear Basis for Rendering Render three images Take linear combinations. Why can’t we do this with three real images?

Reflectance smooths lighting

Basis from diffuse lighting Z Y X XZ YZ XY

Note, this can also be done with 9 real images, because this is a basis that contains real images In 3D, real images aren’t in the 3D space, we have to take the max with 0 to get real images.

Non-Photorealistic Rendering Take a photo and turn it into a different kind of image.

De Carlo and Santella Video

Image Analogies Given A, A’ and B, generate B’ A bit like Efros and Leung texture synthesis. (Pictures from image analogies paper linked to on class web page).