Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Slides:



Advertisements
Similar presentations
Spherical Convolution in Computer Graphics and Vision Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University SIAM Imaging Science Conference:
Advertisements

Light Fields PROPERTIES AND APPLICATIONS. Outline  What are light fields  Acquisition of light fields  from a 3D scene  from a real world scene 
5/13/2015CAM Talk G.Kamberova Computer Vision Introduction Gerda Kamberova Department of Computer Science Hofstra University.
Automatic scene inference for 3D object compositing Kevin Karsch (UIUC), Sunkavalli, K. Hadap, S.; Carr, N.; Jin, H.; Fonte, R.; Sittig, M., David Forsyth.
Extracting Objects from Range and Radiance Images Computer Science Division University of California at Berkeley Computer Science Division University of.
Structure and Motion from Line Segments in Multiple Images Camillo J. Taylor, David J. Kriegman Presented by David Lariviere.
Dana Cobzas-PhD thesis Image-Based Models with Applications in Robot Navigation Dana Cobzas Supervisor: Hong Zhang.
Image-Based Modeling, Rendering, and Lighting
Advanced Graphics, Overview Advanced Computer Graphics Overview.
A new approach for modeling and rendering existing architectural scenes from a sparse set of still photographs Combines both geometry-based and image.
Advanced Computer Graphics CSE 190 [Spring 2015], Lecture 14 Ravi Ramamoorthi
Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.
Advanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light Fields Ravi Ramamoorthi
Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi
Measurement, Inverse Rendering COMS , Lecture 4.
Representations of Visual Appearance COMS 6160 [Spring 2007], Lecture 4 Image-Based Modeling and Rendering
Image-Based Modeling and Rendering CS 6998 Lecture 6.
Image-Based Lighting : Computational Photography Alexei Efros, CMU, Fall 2005 © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec.
View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.
High-Quality Video View Interpolation
Reflectance and Texture of Real-World Surfaces KRISTIN J. DANA Columbia University BRAM VAN GINNEKEN Utrecht University SHREE K. NAYAR Columbia University.
Computing With Images: Outlook and applications
CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.
Computational Vision Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.
Surface Light Fields for 3D Photography Daniel Wood Daniel Azuma Wyvern Aldinger Brian Curless Tom Duchamp David Salesin Werner Stuetzle.
CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.
A New Correspondence Algorithm Jitendra Malik Computer Science Division University of California, Berkeley Joint work with Serge Belongie, Jan Puzicha,
View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.
The Hilbert Problems of Computer Vision
Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting.
Computer graphics & visualization Introduction. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization.
Bridging the Gap to the Real Wojciech Matusik Adobe Systems, Inc.
Real-Time High Quality Rendering CSE 291 [Winter 2015], Lecture 6 Image-Based Rendering and Light Fields
Digital Face Cloning David Bennett Christophe HerySteve Sullivan George Borshukov J.P. Lewis Lance Williams Paul Debevec Fred Pighin Li Zhang.
My Research Experience Cheng Qian. Outline 3D Reconstruction Based on Range Images Color Engineering Thermal Image Restoration.
The Three R’s of Vision Jitendra Malik.
Advanced Computer Graphics (Spring 2013) CS 283, Lecture 15: Image-Based Rendering and Light Fields Ravi Ramamoorthi
Image-Based Rendering. 3D Scene = Shape + Shading Source: Leonard mcMillan, UNC-CH.
Image-Based Rendering from a Single Image Kim Sang Hoon Samuel Boivin – Andre Gagalowicz INRIA.
Interactive Virtual Relighting and Remodelling of Real Scenes C. Loscos 1, MC. Frasson 1,2,G. Drettakis 1, B. Walter 1, X. Granier 1, P. Poulin 2 (1) iMAGIS*
Image Based Rendering(IBR) Jiao-ying Shi State Key laboratory of Computer Aided Design and Graphics Zhejiang University, Hangzhou, China
3DDI Visualization MURI UC Berkeley and MIT. UC- MIT 3DDI: Overview Project pipeline: 3D capture: Modeling, simulation Rendering3D Display Applications:
03/10/03© 2003 University of Wisconsin Last Time Tone Reproduction and Perceptual Issues Assignment 2 all done (almost)
Xiaoguang Han Department of Computer Science Probation talk – D Human Reconstruction from Sparse Uncalibrated Views.
Sebastian Enrique Columbia University Relighting Framework COMS 6160 – Real-Time High Quality Rendering Nov 3 rd, 2004.
Image-based rendering Michael F. Cohen Microsoft Research.
Interactively Modeling with Photogrammetry Pierre Poulin Mathieu Ouimet Marie-Claude Frasson Dép. Informatique et recherche opérationnelle Université de.
Conceptual and Experimental Vision Introduction R.Bajcsy, S.Sastry and A.Yang Fall 2006.
Vision-based human motion analysis: An overview Computer Vision and Image Understanding(2007)
Spring 2015 CSc 83020: 3D Photography Prof. Ioannis Stamos Mondays 4:15 – 6:15
4.1. R ENDERING Aspects of Game Rendering. From Wikipedia: Rendering is the process of generating an image from a model. The model is a description.
Image-Based Lighting © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec CS194: Image Manipulation & Computational Photography Alexei Efros,
Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography.
Image-Based Rendering of Diffuse, Specular and Glossy Surfaces from a Single Image Samuel Boivin and André Gagalowicz MIRAGES Project.
Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs Computer Science Division University of California at Berkeley.
Subject Name: Computer Graphics Subject Code: Textbook: “Computer Graphics”, C Version By Hearn and Baker Credits: 6 1.
Yizhou Yu Texture-Mapping Real Scenes from Photographs Yizhou Yu Computer Science Division University of California at Berkeley Yizhou Yu Computer Science.
CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.
Image-Based Rendering Geometry and light interaction may be difficult and expensive to model –Think of how hard radiosity is –Imagine the complexity of.
Radiometry of Image Formation Jitendra Malik. A camera creates an image … The image I(x,y) measures how much light is captured at pixel (x,y) We want.
MASKS © 2004 Invitation to 3D vision. MASKS © 2004 Invitation to 3D vision Lecture 1 Overview and Introduction.
Radiometry of Image Formation Jitendra Malik. What is in an image? The image is an array of brightness values (three arrays for RGB images)
Advanced Computer Graphics
Advanced Computer Graphics
Journal of Vision. 2003;3(5):3. doi: /3.5.3 Figure Legend:
Image-Based Rendering
Interactive Computer Graphics
© 2005 University of Wisconsin
Image Based Modeling and Rendering (PI: Malik)
Computer Vision Computer vision attempts to construct meaningful and explicit descriptions of the world depicted in an image Using machines to Interpret!!!
Presentation transcript:

Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley

Overview 3D capture: Modeling, simulation RenderingDisplay Applications: Simulation Virtual Reality Remote collaboration

Graphics and Vision Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image. Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination. Computer graphics is the forward problem: given scene geometry, reflectances and lighting, synthesize an image. Computer vision must address the inverse problem: given an image/multiple images, reconstruct the scene geometry, reflectacnes and illumination.

Image-based Modeling –Vary viewpoint –Vary lighting –Vary scene configuration –Vary viewpoint –Vary lighting –Vary scene configuration Recover Models of Real World Scenes and Make Possible Various Visual Interactions

Image-based Modeling 1st Generation---- vary viewpoint but not lighting –Acquire photographs –Recover geometry (explicit or implicit) –Texture map 1st Generation---- vary viewpoint but not lighting –Acquire photographs –Recover geometry (explicit or implicit) –Texture map

Recovering geometry Historical roots in photogrammetry and analysis of 3D cues in human vision Single images adequate given knowledge of object class Multiple images make the problem easier, but not trivial as corresponding points must be identified. Historical roots in photogrammetry and analysis of 3D cues in human vision Single images adequate given knowledge of object class Multiple images make the problem easier, but not trivial as corresponding points must be identified.

Arc de Triomphe

The Taj Mahal Taj Mahal modeled from one photograph by G. Borshukov

Campus Model of UC Berkeley Campanile + 40 Buildings (Debevec et al)

Image-based Modeling 2nd Generation---- vary viewpoint and lighting –Recover geometry & reflectance properties –Render using light transport simulation or local shading 2nd Generation---- vary viewpoint and lighting –Recover geometry & reflectance properties –Render using light transport simulation or local shading Original Lighting & ViewpointNovel Lighting & Viewpoint

Inverse Global Illumination (Yu et al) Reflectance Properties Radiance Maps Geometry Light Source s

Real vs. Synthetic

Image-based Modeling 3 rd Generation--Vary spatial configurations in addition to viewpoint and lighting Novel ViewpointNovel Viewpoint & Configuration

Our Framework Input –Multiple range scans of a scene –Multiple photographs of the same scene Input –Multiple range scans of a scene –Multiple photographs of the same scene Output –Geometric meshes of each object in the scene –Registered texture maps for objects

Segmentation: From images to objects

Segmentation Results [Yu, Ferencz and Malik ’00]

Models of Individual Objects

Texture-Mapping and Object Manipulation

Image Based modeling for motion capture Body Suits, Markers Video Motion Capture

Eadweard Muybridge [Bregler and Malik ’98]

Continuing Challenges Finding correspondences automatically Optimal estimation of structure from n views under perspective projection Models of reflectance and texture for natural materials and objects Finding correspondences automatically Optimal estimation of structure from n views under perspective projection Models of reflectance and texture for natural materials and objects