Analysis of Lighting Effects Outline: The problem Lighting models Shape from shading Photometric stereo Harmonic analysis of lighting.

Slides:



Advertisements
Similar presentations
Computer Vision Radiometry. Bahadir K. Gunturk2 Radiometry Radiometry is the part of image formation concerned with the relation among the amounts of.
Advertisements

Announcements Project 2 due today Project 3 out today –demo session at the end of class.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #12.
Lecture 35: Photometric stereo CS4670 / 5670 : Computer Vision Noah Snavely.
3D Modeling from a photograph
Capturing light Source: A. Efros. Image formation How bright is the image of a scene point?
Computer Vision CS 776 Spring 2014 Photometric Stereo and Illumination Cones Prof. Alex Berg (Slide credits to many folks on individual slides)
Lighting affects appearance. What is the Question ? (based on work of Basri and Jacobs, ICCV 2001) Given an object described by its normal at each.
1. What is Lighting? 2 Example 1. Find the cubic polynomial or that passes through the four points and satisfies 1.As a photon Metal Insulator.
Based on slides created by Edward Angel
1 Angel: Interactive Computer Graphics 5E © Addison-Wesley 2009 Shading I.
Lecture 23: Photometric Stereo CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein.
AA II z f Surface Radiance and Image Irradiance Same solid angle Pinhole Camera Model.
May 2004SFS1 Shape from shading Surface brightness and Surface Orientation --> Reflectance map READING: Nalwa Chapter 5. BKP Horn, Chapter 10.
Analysis of Lighting Effects Outline: The problem Lighting models Shape from shading Photometric stereo Harmonic analysis of lighting.
© 2002 by Davi GeigerComputer Vision January 2002 L1.1 Image Formation Light can change the image (and appearances). What is the relation between pixel.
Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows Kalyan Sunkavalli, Harvard University Joint work with Todd Zickler and Hanspeter Pfister.
Computer Graphics (Fall 2008) COMS 4160, Lecture 19: Illumination and Shading 2
© 2004 by Davi GeigerComputer Vision February 2004 L1.1 Image Formation Light can change the image and appearances (images from D. Jacobs) What is the.
Computer Vision Sources, shading and photometric stereo Marc Pollefeys COMP 256.
Objectives Learn to shade objects so their images appear three- dimensional Learn to shade objects so their images appear three- dimensional Introduce.
Previously Two view geometry: epipolar geometry Stereo vision: 3D reconstruction epipolar lines Baseline O O’ epipolar plane.
© 2003 by Davi GeigerComputer Vision September 2003 L1.1 Image Formation Light can change the image and appearances (images from D. Jacobs) What is the.
Basic Principles of Surface Reflectance Lecture #3 Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan.
Computer Graphics (Fall 2004) COMS 4160, Lecture 16: Illumination and Shading 2 Lecture includes number of slides from.
Photometric Stereo Merle Norman Cosmetics, Los Angeles Readings R. Woodham, Photometric Method for Determining Surface Orientation from Multiple Images.
Photometric Stereo & Shape from Shading
1 Angel: Interactive Computer Graphics 4E © Addison-Wesley 2005 Shading I Ed Angel Professor of Computer Science, Electrical and Computer Engineering,
Lighting affects appearance. Light Source emits photons Photons travel in a straight line When they hit an object they: bounce off in a new direction.
CS 480/680 Computer Graphics Shading I Dr. Frederick C Harris, Jr.
Light and shading Source: A. Efros.
Computer Vision Spring ,-685 Instructor: S. Narasimhan PH A18B T-R 10:30am – 11:50am Lecture #13.
CS 445 / 645: Introductory Computer Graphics
Reflectance Map: Photometric Stereo and Shape from Shading
Course 12 Calibration. 1.Introduction In theoretic discussions, we have assumed: Camera is located at the origin of coordinate system of scene.
Geometric Models & Camera Calibration
Y. Moses 11 Combining Photometric and Geometric Constraints Yael Moses IDC, Herzliya Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion.
CSC418 Computer Graphics n Illumination n Lights n Lightinging models.
Shape from Shading and Texture. Lambertian Reflectance Model Diffuse surfaces appear equally bright from all directionsDiffuse surfaces appear equally.
Capturing light Source: A. Efros.
Sources, Shadows, and Shading
Understanding the effect of lighting in images Ronen Basri.
Course 10 Shading. 1. Basic Concepts: Light Source: Radiance: the light energy radiated from a unit area of light source (or surface) in a unit solid.
Announcements Office hours today 2:30-3:30 Graded midterms will be returned at the end of the class.
Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012 Material Representation K. H. Ko School of Mechatronics Gwangju Institute.
CSCE 641 Computer Graphics: Reflection Models Jinxiang Chai.
November 4, THE REFLECTANCE MAP AND SHAPE-FROM-SHADING.
Shape from Shading Course web page: vision.cis.udel.edu/cv February 26, 2003  Lecture 6.
Announcements Final Exam Friday, May 16 th 8am Review Session here, Thursday 11am.
Local Illumination and Shading
1Ellen L. Walker 3D Vision Why? The world is 3D Not all useful information is readily available in 2D Why so hard? “Inverse problem”: one image = many.
Lighting affects appearance. Lightness Digression from boundary detection Vision is about recovery of properties of scenes: lightness is about recovering.
Reflectance Function Estimation and Shape Recovery from Image Sequence of a Rotating object Jiping Lu, Jim Little UBC Computer Science ICCV ’ 95.
Tal Amir Advanced Topics in Computer Vision May 29 th, 2015 COUPLED MOTION- LIGHTING ANALYSIS.
COMPUTER GRAPHICS CS 482 – FALL 2015 OCTOBER 27, 2015 SCATTERING LIGHT SCATTERING PHYSICALLY BASED SCATTERING SUBSURFACE SCATTERING AMBIENT OCCLUSION.
OpenGL Shading. 2 Objectives Learn to shade objects so their images appear three-dimensional Introduce the types of light-material interactions Build.
CS552: Computer Graphics Lecture 33: Illumination and Shading.
Understanding the effect of lighting in images Ronen Basri.
EECS 274 Computer Vision Sources, Shadows, and Shading.
Announcements Project 3a due today Project 3b due next Friday.
1 Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization Neil G. Alldrin Satya P. Mallick David J. Kriegman University of California, San.
MAN-522 Computer Vision Spring
7. Illumination Phong Illumination Diffuse, Specular and Ambient
Merle Norman Cosmetics, Los Angeles
Announcements Project 3b due Friday.
Depthmap Reconstruction Based on Monocular cues
Announcements Today: evals Monday: project presentations (8 min talks)
Analysis of Lighting Effects
Shape from Shading and Texture
Physical Problem of Simultaneous Linear Equations in Image Processing
Presentation transcript:

Analysis of Lighting Effects Outline: The problem Lighting models Shape from shading Photometric stereo Harmonic analysis of lighting

Applications Modeling the effect of lighting can be used for Recognition – particularly face recognition Shape reconstruction Motion estimation Re-rendering …

Lighting is Complex Lighting can come from any direction and at any strength Infinite degree of freedom

Issues in Lighting Single light source (point, extended) vs. multiple light sources Far light vs. near light Matt surfaces vs. specular surfaces Cast shadows Inter-reflections

Lighting From a source – travels in straight lines Energy decreases with r 2 (r – distance from source) When light rays reach an object  Part of the energy is absorbed  Part is reflected (possibly different amounts in different directions)  Part may continue traveling through the object, if object is transparent / translucent

Specular Reflectance When a surface is smooth light reflects in the opposite direction of the surface normal

Specular Reflectance When a surface is slightly rough the reflected light will fall off around the specular direction

Lambertian Reflectance When the surface is very rough light may be reflected equally in all directions

Lambertian Reflectance When the surface is very rough light may be reflected equally in all directions

Lambertian Reflectance

Lambert Law or 

BRDF A general description of how opaque objects reflect light is given by the Bidirectional Reflectance Distribution Function (BRDF) BRDF specifies for a unit of incoming light in a direction (θ i,Φ i ) how much light will be reflected in a direction (θ e,Φ e ). BRDF is a function of 4 variables f(θ i,Φ i ;θ e,Φ e ). (0,0) denotes the direction of the surface normal. Most surfaces are isotropic, i.e., reflectance in any direction depends on the relative direction with respect to the incoming direction (leaving 3 parameters)

Why BRDF is Needed? Light from front Light from back

Most Existing Algorithms Assume a single, distant point source All normals visible to the source (θ<90°) Plus, maybe, ambient light (constant lighting from all directions)

Shape from Shading Input: a single image Output: 3D shape Problem is ill-posed, many different shapes can give rise to same image Common assumptions:  Lighting is known  Reflectance properties are completely known – For Lambertian surfaces albedo is known (usually uniform) First solutions: Horn, 1977

convex concave

convex concave

HVS Assumes Light from Above

Lambertian Shape from Shading (SFS) Image irradiance equation Image intensity depends on surface orientation It also depends on lighting and albedo, but in SFS those assumed to be known

Surface Normal A surface z(x,y) A point on the surface: (x,y,z(x,y)) T Tangent directions t x =(1,0,p) T, t y =(0,1,q) T with p=z x, q=z y

Lambertian SFS We obtain Proportionality – because albedo is known up to scale For each point one differential equation in two unknowns, p and q But both come from an integrable surface z(x,y) Thus, p y = q x (z xy =z yx ). Therefore, one differential equation in one unknowns (Horn, 1977)

Lambertian SFS

SFS with Fast Marching Suppose lighting coincides with viewing direction l =(0,0,1) T, then Therefore For general l we can rotate the camera

Distance Transform is called Eikonal equation Consider d(x) s.t. |d x |=1 Assume x 0 =0 x0x0 d x

Distance Transform is called Eikonal equation Consider d(x) s.t. |d x |=1 Assume both x 0 =0 and x 1 =0 Minimum at every point (shortest distance) x0x0 d x x1x1

SFS with Fast Marching - Some places are more difficult to walk than others Solution to Eikonal equations –using a variation of Dijkstra’s algorithm Initial condition: we need to know z at extrema Starting from lowest points, we propagate a wave front, where we gradually compute new values of z from old ones (Kimmel and Sethian, 2001)

Results

Photometric Stereo Fewer assumptions are needed if we have several images of the same object under different lightings In this case we can solve for both lighting, albedo, and shape This can be done by Factorization Recall that Ignore the case θ>90° (Woodham 1980)

Photometric Stereo - Factorization Goal: given M, find L and S What should rank(M) be? (Hayakawa, 1994)

Photometric Stereo - Factorization Use SVD to find a rank 3 approximation Define So Factorization is not unique, since, A invertible To reduce ambiguity we impose integrability

Reducing Ambiguity Assume We want to enforce integrability Notice that Denote by the three rows of A, then From which we obtain (Belhumeur, Kriegman, Yuille, 1999)

Reducing Ambiguity Transforming a surface linearly maintains integrability It can be shown that this is the only transformation that maintains integrability Such transformations are called “generalized bas relief transformations” (GBR) Thus, by imposing integrability the surface is reconstructed up to GBR

Relief Sculptures

Summary Lighting effects are complex Algorithms for SFS and photometric stereo for Lambertian object illuminated by a single light source Harmonic analysis extends this to multiple light sources Handling specularities, shadows, and inter- reflections is difficult