1. I hope that you: Looked at book & websiteLooked at book & website Want to stay in this class (change before Friday!)Want to stay in this class (change before Friday!) Participate! Ask Questions! Get Inspired …Participate! Ask Questions! Get Inspired … Overview—Eight Big Ideas CS395/495: Advanced Computer Graphics Overview—Eight Big Ideas Jack Tumblin jet@cs.northwestern.edu

2. 8 Topics: You Choose 2 Class Presentation on a selected paperClass Presentation on a selected paper –Must choose topic area by Monday –1-3 brave volunteers needed for next week Write Broad Survey Paper --OR-- Write, Demo Programming ProjectWrite Broad Survey Paper --OR-- Write, Demo Programming Project –Feb 4:reference list due (survey paper) –or-- 1-2 page proposal due (projects) –Mar 13: Paper or Project Demo due. (Graduate) two topic surveysCS495: (Graduate) two topic surveys –Feb 18: 1 st survey paper due –Mar 13: 2 nd survey (or project) due

3. 8 Topics SCHEDULED ORDER STILL UNKNOWN! Shape RepresentationShape Representation 2D Image-Space Techniques2D Image-Space Techniques Surface MappingsSurface Mappings Global Illumination and Light TransportGlobal Illumination and Light Transport Shape ManipulationShape Manipulation Light Fields and ApproximationsLight Fields and Approximations Non-Photorealistic Rendering (NPR)Non-Photorealistic Rendering (NPR) Model Acquisition and IBMRModel Acquisition and IBMR

4. #1 SCHEDULED ORDER STILL UNKNOWN! Shape RepresentationShape Representation 2D Image-Space Techniques Surface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

5. Shape representation The problem: –We need to make pictures of things. –What data structures describe shape? (some are tough for polygons—hair, smoke, water, cloth, peach, fuzz, The solutions…

6. Parametric surfaces The idea: –The scene is a collection of patches. –Patch = curved, continuous. polynomial of u,v parameters (u,v) vary across surface point on surface: (x(u,v), y(u,v), z(u,v)) –Manipulate the patches w/ control pts.Critique: –Nice user control, no faceting, succinct. –Accurate rendering was slow (less true now) – LOD control is tough...

7. Parametrics: Utah, Erlangen, Pixar

8. Points The idea: –A whole bunch of 3D points only **No connectivity, no edges, no faces** –Each with associated colors, normals. –We fill in the gaps with some “blurring”.Critique: –With hierarchy, can render big things. –What about surface mappings?

9. Points: QSplat Stanford: Rusinkiewicz & Levoy

10. Volumes The idea: –The scene is a 3D image. –Pixels become voxels.Critique: –Nice for medical data, transparency. –Data size huge: 100 3 = one million!

11. Volumes: Univ. Stuttgart

12. Many Others… Implicit surfaces Nice for fluids, topology-changing Constructive solid geometry Used in engineering: cut(), union(), diff()... Procedural modeling shape is result of a computed function Natural things: clouds, fire, plants

13. #2 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques2D Image-Space Techniques Surface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

14. Image-Space Techniques ‘Image’==A 2D map of light intensities from a lens‘Image’==A 2D map of light intensities from a lens ‘Digital Image’==a 2D grid of numbers (pixels)‘Digital Image’==a 2D grid of numbers (pixels) PROBLEMS:PROBLEMS: –What is all those points? –What is between all those points? –How can I use images to make other images GOAL: –More Flexibility; let me do more than just display!

15. ‘Digital’ Images: 2D Grid of Numbers NO intrinsic meaning, but...NO intrinsic meaning, but... Widely assumed to representWidely assumed to represent –Point Samples of a “smoothed” 2D intensity surface –Uniform sampling pattern (but not always) x y (!weasel-word!)

16. Image-Space Techniques GOAL: More flexibility! Compositing /Matte: cut-and-paste, transparency, the ‘digital optical bench’ …Compositing /Matte: cut-and-paste, transparency, the ‘digital optical bench’ … Warp: image as a ‘rubber sheet’, you can cut, stretch, and change at willWarp: image as a ‘rubber sheet’, you can cut, stretch, and change at will Compositing / Environment Matte (Pixar`84) (Salesin99…)Compositing / Environment Matte (Pixar`84) (Salesin99…) Texture Synthesis/elaboration Video Texture(Schodl 2000)Texture Synthesis/elaboration Video Texture(Schodl 2000) Polynomial Texture Map (Malzburg2001)Polynomial Texture Map (Malzburg2001)

17. Image-Space Techniques GOAL: More flexibility! Compositing / Environment Matte (Pixar`84) (Salesin99…)Compositing / Environment Matte (Pixar`84) (Salesin99…)

18. Image-Space Techniques GOAL: More flexibility! Texture Synthesis/elaboration Efros98, Wei/Levoy99, Ashikhmin2000, Video Texture(Schodl 2000),Texture Synthesis/elaboration Efros98, Wei/Levoy99, Ashikhmin2000, Video Texture(Schodl 2000), Input Sample Synthesized Result

19. Image-Space Techniques Warp: image as a ‘rubber sheet’, you can cut, stretch, and change at willWarp: image as a ‘rubber sheet’, you can cut, stretch, and change at will.http://www.cs.uu.nl/~martijn/Applets/warp.htmlhttp://www.cs.uu.nl/~martijn/Applets/warp.html http://www.cs.rochester.edu/ u/kyros/Courses/ ECE102/lecture-3-web/sld018.htm

20. #3 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques Surface MappingsSurface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

21. Surface mappings The problem: –We need more detail on our shapes. –But our shapes are already complex. The solutions…

22. Procedural mappings The idea: –Like texture mapping, except: –The RGB value is f(u,v), not an image.Critique: –Very succinct, new possibilities. –Until recently, no hardware support. –Hard to “model” procedures.

23. Procedural mappings: Perlin

24. Bump mapping The idea: –A kind of procedural mapping –Adds fine “bumpiness” to the surfaceCritique: –Works as advertised. –Until recently, no hardware support.

25. Bump mapping: NVidia

26. Many Others… Environment mapping Nice for mirrored surfaces Light mapping Cheat to get global illumination Displacement Maps Add little bits of geometry to surface

27. #4 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques sSurface Mappings Global Illumination and Light TransportGlobal Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

28. Global Illumination Or ‘Physically Accurate Rendering’ PROBLEM:PROBLEM: –‘CG looks a bit weird, & needs lots of tweaking… –Not Predictive: can’t compute light meter #s Solutions:Solutions: –‘Pixel Drawing’  Rendering from 1 st principles; –Science –Science: fundamental optical principles, invariants –Serious Math & Algorithms work...

29. Global Illumination ??Basic Physics!? Why isn’t this solved already? ??Basic Physics!? Why isn’t this solved already? –Endlessly Bouncing Light: Radiance Field is 5-D The Rendering Equation (+ the math makes it look more scary than it is) –REAL Materials are complex, and largely unmeasured BRDF alone is 4D, ‘spiky’, interesting But Light scatters WITHIN materials too! –Computational Complexity—cleverness req’d… (naïve solutions are exponential time: O(e N ))

30. Global Illumination Real surfaces are much more complicated! Specular effects, self-shadowing, refraction, interreflection, BRDF... Surface Properties: Why Phong shading is not enough:

32. Global Illumination Seminal Papers:Seminal Papers: Kajiya, Goral, Hanrahan, Cohen, Rushmeier, … Excellent Freeware Solution:Excellent Freeware Solution: Greg Wards’ RADIANCE 1 st graphics sub-area to struggle for relevance1 st graphics sub-area to struggle for relevance –(Why bother when light maps/ multitexturing on my $200 video card looks great in real time?) Opinion: soon other areas will hit tough scrutinyOpinion: soon other areas will hit tough scrutiny Aside

33. #5 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques sSurface Mappings Global Illumination and Light Transport Shape ManipulationShape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

34. Shape manipulation The problem: –Want adjust our shapes; wiggle, twist, simplify. BUT –Don’t want to redo all our modeling again. The solutions…

35. Model simplification The idea: –It should look the same, but… –It should have fewer primitives.Critique: –In general, it works! –Only limited user control. –Surface mappings poorly handled.

36. Simplification: Watson

37. Surface fitting Surface fitting: Find surface from a cloud of points Hoppe ‘94

38. 3D Morphing Change shape A to shape B (Turk/O’Brien99)

39. #6 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques sSurface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and ApproximationsLight Fields and Approximations Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

40. PROBLEM: –Prompt rendering  dull, inaccurate result. –Poor tradeoff of interaction vs. rendering realism. Can’t we do more than movies?GOAL: –Capture ALL the light leaving an object –Fast sort, re-display allows interaction. Light Fields & Approximations

41. An Inspired Observation: (at an informal ‘bull session’ at SIGGRAPH`94: Levoy, Hanrahan, Rushmeier, Cohen, others…) ‘Sphere of Cameras’ records all rays‘Sphere of Cameras’ records all rays Any image outside bubble == subset of raysAny image outside bubble == subset of rays …

42. Light Fields & Approximations ….

43. .http://graphics.stanford.edu/projects/lightfield/http://www.intel.com/research/mrl/research/lfm/

44. Recent: Try to separate illumination from surface properties in the light-field data. Univ. of Washington SIGGRAPH 2000

45. #7 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques sSurface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR)Non-Photorealistic Rendering (NPR) Model Acquisition and IBMR

46. Non-photorealism The problem: –Reality / Photos are too narrow skips drawing, painting, illustrations, etc. –Let’s make broader classes of artificial renderings! The solutions…

47. Painterly approaches The idea: –Make the image look like it’s painted.Critique: –Works well from image or model. –What about Picasso? –How interactive is it?

48. Painterly NPR

49. Sketching approaches The idea: –Make the image look like it’s drawn.Critique: –Works well from image or model. –Can we add imperfection?

50. Sketching NPR

51. Toon approaches The idea: –Make the image look like it’s a cartoon.Critique: –For games, hardware support is key. –Is it there?

52. Toon NPR

53. #8 SCHEDULED ORDER STILL UNKNOWN! Shape Representation 2D Image-Space Techniques sSurface Mappings Global Illumination and Light Transport Shape Manipulation Light Fields and Approximations Non-Photorealistic Rendering (NPR)Non-Photorealistic Rendering (NPR) Model Acquisition and IBMRModel Acquisition and IBMR

54. Model Acquisition & IBMR PROBLEM: Geometric modeling is tedious, slow, expensive Most modeled objects exist already; 3D laser scanning, etc. is expensive, finicky, difficult Geom. Models almost never look as good as photos Meanwhile: prices fall, speed & capacity rise for Digital Cameras, Displays, Memory, Desktop PCs… ! Lets use cameras to eliminate the tedium !

55. Model Acquisition & IBMR Panorama Stitching...

56. Model Acquisition & IBMR How does photo change with lighting direction? (Malzbender,2000: Polynomial Texture Maps ))

57. Model Acquisition & IBMR How does photo change with lighting direction? (Malzbender,2000: Polynomial Texture Maps) Ordinary Photo Computed Specular Highlight PTM Reconstruction PTM + Highlight

58. Model Acquisition & IBMR GOAL 1: Generalize Photography:GOAL 1: Generalize Photography: –Get MORE than light; –Estimate shape, texture, reflectance, lighting, … –A Re-thinking of pointwise 3-D scanning… GOAL 2: Generalize Image Viewing:GOAL 2: Generalize Image Viewing: –Interactive, movable viewpoint –‘Reprojection: re-use already-rendered image parts