EECS 274 Computer Vision Sources, Shadows, and Shading.

Slides:



Advertisements
Similar presentations
Photometric Image Formation
Advertisements

Computer Vision Radiometry. Bahadir K. Gunturk2 Radiometry Radiometry is the part of image formation concerned with the relation among the amounts of.
Announcements Project 2 due today Project 3 out today –demo session at the end of class.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #12.
Lecture 35: Photometric stereo CS4670 / 5670 : Computer Vision Noah Snavely.
3D Modeling from a photograph
Capturing light Source: A. Efros. Image formation How bright is the image of a scene point?
Computer Vision CS 776 Spring 2014 Photometric Stereo and Illumination Cones Prof. Alex Berg (Slide credits to many folks on individual slides)
Eyes for Relighting Extracting environment maps for use in integrating and relighting scenes (Noshino and Nayar)
Virtual Realism LIGHTING AND SHADING. Lighting & Shading Approximate physical reality Ray tracing: Follow light rays through a scene Accurate, but expensive.
16421: Vision Sensors Lecture 6: Radiometry and Radiometric Calibration Instructor: S. Narasimhan Wean 5312, T-R 1:30pm – 2:50pm.
Basic Principles of Surface Reflectance
Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012 Material Representation K. H. Ko School of Mechatronics Gwangju Institute.
Ray Tracing & Radiosity Dr. Amy H. Zhang. Outline  Ray tracing  Radiosity.
Computer Vision - A Modern Approach Set: Radiometry Slides by D.A. Forsyth Radiometry Questions: –how “bright” will surfaces be? –what is “brightness”?
Shape-from-X Class 11 Some slides from Shree Nayar. others.
Lecture 23: Photometric Stereo CS4670/5760: Computer Vision Kavita Bala Scott Wehrwein.
AA II z f Surface Radiance and Image Irradiance Same solid angle Pinhole Camera Model.
Capturing light Source: A. Efros. Review Pinhole projection models What are vanishing points and vanishing lines? What is orthographic projection? How.
May 2004SFS1 Shape from shading Surface brightness and Surface Orientation --> Reflectance map READING: Nalwa Chapter 5. BKP Horn, Chapter 10.
Introduction to Computer Vision CS / ECE 181B Tues, May 18, 2004 Ack: Matthew Turk (slides)
Basic Principles of Surface Reflectance
Computer Vision Sources, shading and photometric stereo Marc Pollefeys COMP 256.
Objectives Learn to shade objects so their images appear three- dimensional Learn to shade objects so their images appear three- dimensional Introduce.
Lecture 32: Photometric stereo, Part 2 CS4670: Computer Vision Noah Snavely.
CSCE 641 Computer Graphics: Radiosity Jinxiang Chai.
Basic Principles of Surface Reflectance Lecture #3 Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan.
Foundations of Computer Graphics (Spring 2010) CS 184, Lecture 21: Radiosity
CSCE 641 Computer Graphics: Radiosity Jinxiang Chai.
1 Dr. Scott Schaefer Radiosity. 2/38 Radiosity 3/38 Radiosity Physically based model for light interaction View independent lighting Accounts for indirect.
Photometric Stereo Merle Norman Cosmetics, Los Angeles Readings R. Woodham, Photometric Method for Determining Surface Orientation from Multiple Images.
CSCE 441 Computer Graphics: Radiosity Jinxiang Chai.
Light and shading Source: A. Efros.
Computer Vision Spring ,-685 Instructor: S. Narasimhan PH A18B T-R 10:30am – 11:50am Lecture #13.
EECS 274 Computer Vision Light and Shading. Radiometry – measuring light Relationship between light source, surface geometry, surface properties, and.
Reflectance Map: Photometric Stereo and Shape from Shading
Today More raytracing stuff –Soft shadows and anti-aliasing More rendering methods –The text book is good on this –I’ll be using images from the CDROM.
Computer Vision - A Modern Approach Set: Sources, shadows and shading Slides by D.A. Forsyth Sources and shading How bright (or what colour) are objects?
Analysis of Lighting Effects Outline: The problem Lighting models Shape from shading Photometric stereo Harmonic analysis of lighting.
Y. Moses 11 Combining Photometric and Geometric Constraints Yael Moses IDC, Herzliya Joint work with Ilan Shimshoni and Michael Lindenbaum, the Technion.
Shape from Shading and Texture. Lambertian Reflectance Model Diffuse surfaces appear equally bright from all directionsDiffuse surfaces appear equally.
02/16/05© 2005 University of Wisconsin Last Time Re-using paths –Irradiance Caching –Photon Mapping.
Capturing light Source: A. Efros.
Sources, Shadows, and Shading
EECS 274 Computer Vision Sources, Shadows, and Shading.
Graphics Lecture 13: Slide 1 Interactive Computer Graphics Lecture 13: Radiosity - Principles.
Announcements Office hours today 2:30-3:30 Graded midterms will be returned at the end of the class.
Illumination and Shading
1 Chapter 5: Sources, Shadows, Shading Light source: Anything emits light that is internally generated Exitance: The internally generated power per unit.
Shape from Shading Course web page: vision.cis.udel.edu/cv February 26, 2003  Lecture 6.
1Ellen L. Walker 3D Vision Why? The world is 3D Not all useful information is readily available in 2D Why so hard? “Inverse problem”: one image = many.
Lighting affects appearance. Lightness Digression from boundary detection Vision is about recovery of properties of scenes: lightness is about recovering.
Tal Amir Advanced Topics in Computer Vision May 29 th, 2015 COUPLED MOTION- LIGHTING ANALYSIS.
01/27/03© 2002 University of Wisconsin Last Time Radiometry A lot of confusion about Irradiance and BRDFs –Clarrified (I hope) today Radiance.
CS552: Computer Graphics Lecture 33: Illumination and Shading.
Announcements Project 3a due today Project 3b due next Friday.
1 Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization Neil G. Alldrin Satya P. Mallick David J. Kriegman University of California, San.
MAN-522 Computer Vision Spring
Radiometry (Chapter 4).
Merle Norman Cosmetics, Los Angeles
Capturing light Source: A. Efros.
Radiosity Dr. Scott Schaefer.
CSCE 441 Computer Graphics: Radiosity
Announcements Project 3b due Friday.
Announcements Today: evals Monday: project presentations (8 min talks)
Lecture 28: Photometric Stereo
CS5500 Computer Graphics May 29, 2006
Announcements Project 3 out today demo session at the end of class.
CS 480/680 Computer Graphics Shading.
Shape from Shading and Texture
Presentation transcript:

EECS 274 Computer Vision Sources, Shadows, and Shading

Surface brightness Depends on local surface properties (albedo), surface shape (normal), and illumination Shading model: a model of how brightness of a surface is obtained Can interpret pixel values to reconstruct its shape and albedo Reading: FP Chapter 2, H Chapter 11

Radiometric properties How bright (or what color) are objects? One more definition: Exitance of a light source is –the internally generated power (not reflected) radiated per unit area on the radiating surface Similar to radiosity: a source can have both –radiosity, because it reflects –exitance, because it emits Independent of its exit angle Internally generated energy radiated per unit time, per unit area But what aspects of the incoming radiance will we model? –Point, line, area source –Simple geometry

Radiosity due to a point sources

Radiosity due to a point source As r is increased, the rays leaving the surface patch and striking the sphere move closer evenly, and the collection changes only slightly, i.e., diffusive reflectance, or albedo Radiosity due to source

Nearby point source model The angle term, can be written in terms of N and S N: surface normal ρ d : diffuse albedo S: source vector - a vector from P to the source, whose length is the intensity term, ε 2 E –works because a dot-product is basically a cosine

Point source at infinity Issue: nearby point source gets bigger if one gets closer –the sun doesn’t for any reasonable assumption Assume that all points in the model are close to each other with respect to the distance to the source Then the source vector doesn’t vary much, and the distance doesn’t vary much either, and we can roll the constants together to get:

Line sources radiosity due to line source varies with inverse distance, if the source is long enough Infinitely long narrow cylinder with constant exitance

Area sources Examples: diffuser boxes, white walls The radiosity at a point due to an area source is obtained by adding up the contribution over the section of view hemisphere subtended by the source –change variables and add up over the source

Radiosity due to an area source ρ d is albedo E is exitance r is distance between points Q and P Q is a coordinate on the source

Shading models Local shading model –Surface has radiosity due only to sources visible at each point –Advantages: often easy to manipulate, expressions easy supports quite simple theories of how shape information can be extracted from shading Global shading model –Surface radiosity is due to radiance reflected from other surfaces as well as from surfaces –Advantages: usually very accurate –Disadvantage: extremely difficult to infer anything from shading values

Local shading models For point sources at infinity: For point sources not at infinity

Shadows cast by a point source A point that can’t see the source is in shadow (self cast shadow) For point sources, the geometry is simple (i.e., the relationship between shape and shading is simple) Radiosity is a measurement of one component of the surface normal Analogous to the geometry of viewing in a perspective camera

Area source shadows Are sources do not produce dark shadows with crisp boundaries 1.Out of shadow 2.Penumbra (“almost shadow”) 3.Umbra (“shadow”)

Photometric stereo Assume: –A local shading model –A set of point sources that are infinitely distant –A set of pictures of an object, obtained in exactly the same camera/object configuration but using different sources –A Lambertian object (or the specular component has been identified and removed)

Projection model for surface recovery - Monge patch In computer vision, it is often known as height map, depth map, or dense depth map Monge patch

Image model For each point source, we know the source vector (by assumption) We assume we know the scaling constant of the linear camera (i.e., intensity value is linear in the surface radiosity) Fold the normal and the reflectance into one vector g, and the scaling constant and source vector into another Vj Out of shadow: g(x,y): describes the surface Vj: property of the illumination and of the camera In shadow:

From many views From n sources, for each of which V i is known For each image point, stack the measurements Solve least squares problem to obtain g One linear system per point

Dealing with shadows Known Unknown

Recovering normal and reflectance Given sufficient sources, we can solve the previous equation (e.g., least squares solution) for g(x, y) Recall that g(x, y) =  (x,y) N(x, y), and N(x, y) is the unit normal This means that alberdo  x,y) =||g(x, y)|| This yields a check –If the magnitude of g(x, y) is greater than 1, there’s a problem And N(x, y) = g(x, y) /  x,y)

Five synthetic images Generated from a sphere in a orthographic view from the same viewing position

Recovered reflectance ||g(x,y)||=ρ(x,y): the alberdo value should be in the range of 0 and 1

Recovered normal field For viewing purpose, vector field is shown for every 16 th pixel in each direction

Parametric surface tangents u v

Shape from normals Recall the surface is written as Parametric surface This means the normal has the form: If we write the known vector g as Then we obtain values for the partial derivatives of the surface:

Shape from normals Recall that mixed second partials are equal --- this gives us a check. We must have: (or they should be similar, at least) Known as integrability test We can now recover the surface height at any point by integration along some path, e.g.

Recovered surface by integration

The illumination cone N-dimensional Image Space x 1 x 2 What is the set of n-pixel images of an object under all possible lighting conditions (at fixed pose)? (Belhuemuer and Kriegman IJCV 99) x n Single light source image

The illumination cone N-dimensional Image Space x 1 x 2 What is the set of n-pixel images of an object under all possible lighting conditions (but fixed pose)? Illumination Cone Proposition: Due to the superposition of images, the set of images is a convex polyhedral cone in the image space. x n Single light source images: Extreme rays of cone 2-light source image

For Lambertian surfaces, the illumination cone is determined by the 3D linear subspace B(x,y), where When no shadows, then Use least-squares to find 3D linear subspace, subject to the constraint f xy =f yx (Georghiades, Belhumeur, Kriegman, PAMI, June, 2001) Original (Training) Images  x,y  f x (x,y) f y (x,y) albedo (surface normals) Surface. f (x,y) (albedo textured mapped on surface ) 3D linear subspace Generating the illumination cone

Single Light SourceFace Movie Image-based rendering: Cast shadows

Yale face database B 10 Individuals 64 Lighting Conditions 9 Poses => 5,760 Images Variable lighting

Limitation Local shading model is a poor description of physical processes that give rise to images –because surfaces reflect light onto one another This is a major nuisance; the distribution of light (in principle) depends on the configuration of every radiator; big distant ones are as important as small nearby ones (solid angle) The effects are easy to model It appears to be hard to extract information from these models

Interreflections - a global shading model Other surfaces are now area sources - this yields: Vis(x, u) is 1 if they can see each other, 0 if they can’t

What do we do about this? Attempt to build approximations –Ambient illumination Study qualitative effects –reflexes –decreased dynamic range –smoothing Try to use other information to control errors

Ambient illumination Two forms –Add a constant to the radiosity at every point in the scene to account for brighter shadows than predicted by point source model Advantages: simple, easily managed (e.g. how would you change photometric stereo?) Disadvantages: poor approximation (compare black and white rooms –Add a term at each point that depends on the size of the clear viewing hemisphere at each point Advantages: appears to be quite a good approximation, but jury is out Disadvantages: difficult to work with