Image or Object? Michael F. Cohen Microsoft Research.

Slides:

Advertisements

Similar presentations

Parameterized Environment Maps

Advertisements

Computer graphics & visualization. Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Our GPU Photon Tracing Pipeline.

Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.

Technische Universität München Computer Graphics SS 2014 Graphics Effects Rüdiger Westermann Lehrstuhl für Computer Graphik und Visualisierung.

Graphics Pipeline.

An Introduction to Light Fields Mel Slater. Outline Introduction Rendering Representing Light Fields Practical Issues Conclusions.

Interactive Deformation of Light Fields Billy Chen, Marc Levoy Stanford University Eyal Ofek, Heung-Yeung Shum Microsoft Research Asia To appear in the.

Introduction to Image-Based Rendering Jian Huang, CS 594, Spring 2002 A part of this set of slides reference slides used at Standford by Prof. Pat Hanrahan.

Light Fields PROPERTIES AND APPLICATIONS. Outline  What are light fields  Acquisition of light fields  from a 3D scene  from a real world scene 

Project 3 Results

Computational Photography

 Engineering Graphics & Introductory Design 3D Graphics and Rendering REU Modeling Course – June 13 th 2014.

Light Field Rendering Shijin Kong Lijie Heng.

Lightfields, Lumigraphs, and Image-based Rendering.

Unstructured Lumigraph Rendering

Advanced Computer Graphics CSE 190 [Spring 2015], Lecture 14 Ravi Ramamoorthi

Modeling Light : Rendering and Image Processing Alexei Efros.

Advanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi

Image-Based Modeling and Rendering CS 6998 Lecture 6.

Image-Based Rendering Computer Vision CSE576, Spring 2005 Richard Szeliski.

Advanced Computer Graphics (Fall 2010) CS 283, Lecture 10: Global Illumination Ravi Ramamoorthi Some images courtesy.

View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL An Incremental Weighted Least Squares Approach To Surface Light Fields Greg Coombe Anselmo Lastra.

Copyright  Philipp Slusallek IBR: View Interpolation Philipp Slusallek.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Siggraph’2000, July 27, 2000 Jin-Xiang Chai Xin Tong Shing-Chow Chan Heung-Yeung Shum Microsoft Research, China Plenoptic Sampling SIGGRAPH’2000.

Computational Photography Light Field Rendering Jinxiang Chai.

Intromission Theory: Plato, Euclid, Ptolemy, da Vinci. Plato for instance, wrote in the fourth century B. C. that light emanated from the eye, seizing.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

1 Angel: Interactive Computer Graphics 4E © Addison-Wesley 2005 Models and Architectures Ed Angel Professor of Computer Science, Electrical and Computer.

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

CS 563 Advanced Topics in Computer Graphics Introduction To IBR By Cliff Lindsay Slide Show ’99 Siggraph[6]

The Story So Far The algorithms presented so far exploit: –Sparse sets of images (some data may not be available) –User help with correspondences (time.

Computer Graphics Inf4/MSc Computer Graphics Lecture 11 Texture Mapping.

Real-Time High Quality Rendering CSE 291 [Winter 2015], Lecture 6 Image-Based Rendering and Light Fields

The Uncanny Valley in 3D Modeling Steve Seitz and Rick Szeliski BIRS Workshop on Computer Vision and the Internet August 31, 2009.

Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.

Virtual Environments with Light Fields and Lumigraphs CS April 2002 Presented by Mike Pilat.

Advanced Computer Graphics (Spring 2013) CS 283, Lecture 15: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Image-Based Rendering. 3D Scene = Shape + Shading Source: Leonard mcMillan, UNC-CH.

Dynamically Reparameterized Light Fields Aaron Isaksen, Leonard McMillan (MIT), Steven Gortler (Harvard) Siggraph 2000 Presented by Orion Sky Lawlor cs497yzy.

Image-based rendering Michael F. Cohen Microsoft Research.

03/12/03© 2003 University of Wisconsin Last Time NPR Assignment Projects High-Dynamic Range Capture Image Based Rendering Intro.

CSC 461: Lecture 3 1 CSC461 Lecture 3: Models and Architectures  Objectives –Learn the basic design of a graphics system –Introduce pipeline architecture.

Lightfields, Lumigraphs, and Other Image-Based Methods.

Image-based Rendering. © 2002 James K. Hahn2 Image-based Rendering Usually based on 2-D imagesUsually based on 2-D images Pre-calculationPre-calculation.

03/24/03© 2003 University of Wisconsin Last Time Image Based Rendering from Sparse Data.

1Computer Graphics Lecture 4 - Models and Architectures John Shearer Culture Lab – space 2

1 Plenoptic Imaging Chong Chen Dan Schonfeld Department of Electrical and Computer Engineering University of Illinois at Chicago May

Image Based Rendering. Light Field Gershun in 1936 –An illuminated objects fills the surrounding space with light reflected of its surface, establishing.

Modeling Light cs129: Computational Photography

03/09/05© 2005 University of Wisconsin Last Time HDR Image Capture Image Based Rendering –Improved textures –Quicktime VR –View Morphing NPR Papers: Just.

CSL 859: Advanced Computer Graphics Dept of Computer Sc. & Engg. IIT Delhi.

Binary Space Partitioning Trees Ray Casting Depth Buffering

Panorama artifacts online –send your votes to Li Announcements.

Interreflections : The Inverse Problem Lecture #12 Thanks to Shree Nayar, Seitz et al, Levoy et al, David Kriegman.

112/5/ :54 Graphics II Image Based Rendering Session 11.

Graphics Graphics Korea University cgvr.korea.ac.kr 1 Surface Rendering Methods 고려대학교 컴퓨터 그래픽스 연구실.

CS 691B Computational Photography Instructor: Gianfranco Doretto Modeling Light.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Image-Based Rendering Geometry and light interaction may be difficult and expensive to model –Think of how hard radiosity is –Imagine the complexity of.

Sub-Surface Scattering Real-time Rendering Sub-Surface Scattering CSE 781 Prof. Roger Crawfis.

Presented by 翁丞世  View Interpolation  Layered Depth Images  Light Fields and Lumigraphs  Environment Mattes  Video-Based.

Advanced Computer Graphics

- Introduction - Graphics Pipeline

Steps Towards the Convergence of Graphics, Vision, and Video

Image-Based Rendering

Real-time Global Illumination with precomputed probe

Presentation transcript:

Image or Object? Michael F. Cohen Microsoft Research

Is this real? Photo by Patrick Jennings Copyright 1995, 96, 97 Whistler B. C. Canada

A mental model Photo by Patrick Jennings Copyright 1995, 96, 97 Whistler B. C. Canada ?

Sue Vision

Is this real? Michael F. Cohen, Shenchang Eric Chen, John R. Wallace, and Donald P. Greenberg 1988, A Progressive Refinement Approach to Fast Radiosity Image Generation.

Bob Graphics

Computer Graphics Image Output Model Synthetic Camera

Real Scene Computer Vision Real Cameras Model Output

Computer Graphics Meets Computer Vision

Combined Model Real Scene Real Cameras Image Output Synthetic Camera

But, vision technology falls short Model Real Scene Real Cameras Image Output Synthetic Camera

… and so does graphics. Model Real Scene Real Cameras Image Output Synthetic Camera

Image Based Rendering Real Scene Real Cameras -or- Expensive Image Synthesis Images+Model Image Output Synthetic Camera

Ray q Constant radiance time is fixed q 5D 3D position 2D direction

All Rays q Plenoptic Function all possible images too much stuff!

Line q Infinite line q 4D 2D direction 2D position

Ray q Discretize q Distance between 2 rays Which is closer together?

Image q What is an image? q All rays through a point Panorama?

Image q 2D position of rays has been fixed direction remains

Image q Image plane q 2D position

q Image plane q 2D position Image

q Light leaving towards “eye” q 2D just dual of image Object

q All light leaving object

Object q 4D 2D position 2D direction

Object q All images

Object q Computer Vision?

Plenoptic Function q 5D q Rays

Lumigraph / Lightfield q Outside convex space q 4D Stuff Empty

Lumigraph q Inside convex space q 4D Empty Stuff

Lumigraph q How to organize capture render

Lumigraph - Organization q 2D position q 2D direction s 

Lumigraph - Organization q 2D position q 2 plane parameterization s u

Lumigraph - Organization q 2D position q 2 plane parameterization u s t s,t u,v v s,t u,v

Lumigraph - Organization  Hold s,t constant  Let u,v vary q An image s,t u,v

Lumigraph - Organization q Discretization higher res near object if diffuse captures texture lower res away captures directions s,t u,v

Lumigraph - Capture q Idea 1 Move camera carefully over s,t plane Gantry see Lightfield paper s,t u,v

Lumigraph - Capture q Idea 2 Move camera anywhere Rebinning see Lumigraph paper s,t u,v

Lumigraph - Rendering q For each output pixel determine s,t,u,v either find closest discrete RGB interpolate near values s,t u,v

Lumigraph - Rendering q For each output pixel determine s,t,u,v either use closest discrete RGB interpolate near values s u

Lumigraph - Rendering q Nearest closest s closest u draw it q Blend 16 nearest quadrilinear interpolation s u

Lumigraph - Rendering q Depth Correction closest s intersection with “object” best u closest u s u

Lumigraph - Rendering q Depth Correction quadralinear interpolation new “closest” like focus s u

Lumigraph - Rendering q Fast s,t,u,v finding scanline interpolate texture mapping shear warp s u

Lumigraph - Rendering q Fast s,t,u,v finding scanline interpolate texture mapping shear warp s u s1,u1 s2,u2

Lumigraph - Rendering q Fast s,t,u,v finding scanline interpolate texture mapping shear warp s u

Lumigraph - Rendering q Fast s,t,u,v finding scanline interpolate texture mapping shear warp s u

Lumigraph - Demo q Lumigraph Lion, Fruit Bowl, Visible Woman, Path Tracing

Lightfield - Demo q Digital Michelangelo Project Marc Levoy, Stanford University Lightfield (“night”) assembled by Jon Shade

Surface Lightfields q Turn 4D parameterization around q Leverage coherence

Surface Lightfields q Wood et al, SIGGRAPH 2000

3D Representations q Image is 2D q Lumigraph is 4D q What happened to 3D? 3D Lumigraph subset Concentric mosaics

3D Lumigraph q One row of s,t plane i.e., hold t constant s,t u,v

3D Lumigraph q One row of s,t plane i.e., hold t constant thus s,u,v a “row of images” s u,v

Concentric Mosaics q Replace “row” with “circle” of images

Concentric Mosaics

q From above

Concentric Mosaics q From above

Concentric Mosaics q Panorama

2.5D Representations q Image is 2D q Lumigraph is 4D q 3D 3D Lumigraph subset Concentric mosaics q 2.5D Layered Depth Images View Dependent Surfaces

Layered Depth Image Image Depth Layered 2.5 D ?

Layered Depth Image q Rendering from LDI Incremental in LDI X and Y Guaranteed to be in back-to-front order

Layered Depth Image q Rendering from LDI Incremental in LDI X and Y Guaranteed to be in back-to-front order

Layered Depth Image (LDI)

Tiled LDIs q Multiresolution q Directionally dependent

View Dependent Surfaces

Summary q 5D: Plenoptic Function (Ray) q 4D: Lumigraph / Lightfield q 3D: Lumigraph Subset q 3D: Concentric Mosaics q 2.5D: Layered Depth Image q 2.5D: View Dependent Models q 2D: Image

Thanks q Peter-Pike Sloan (Lumigraph) q Jonathan Shade (Lightfield, LDI) q Marc Levoy, Stanford University Michaelangelo data set