1. Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 15: Ray Tracing/Global Illumination Ravi Ramamoorthi http://www.cs.columbia.edu/~cs4162 Slides adapted from unit 5 (lectures 21-22) of COMS 4160

2. To Do Enjoy spring break!!  If you want, you can work on raytracer/project during break, but this is not required

3. Course Outline  3D Graphics Pipeline Rendering (Creating, shading images from geometry, lighting, materials) Modeling (Creating 3D Geometry)

4. Course Outline  3D Graphics Pipeline Rendering (Creating, shading images from geometry, lighting, materials) Modeling (Creating 3D Geometry) Unit 1: Foundations of Signal and Image Processing Understanding the way 2D images are formed and displayed, the important concepts and algorithms, and to build an image processing utility like Photoshop Weeks 1 – 3. Assignment 1 due Feb 15 Unit 2: Meshes, Modeling Weeks 4 – 6. Ass 2 due Mar 10 Unit 3: Ray Tracing Weeks 7 – 9. Ass 3 due Apr 12

5. This lecture  High-level overview and summary of ray tracing / global illumination (from 4160)  After spring break: technical details of writing a ray tracer for assignment 3 and more advanced topics Chapters 15.10 and 16 are useful background, although they don’t follow treatment here

6. Effects needed for Realism  (Soft) Shadows  Reflections (Mirrors and Glossy)  Transparency (Water, Glass)  Interreflections (Color Bleeding)  Complex Illumination (Natural, Area Light)  Realistic Materials (Velvet, Paints, Glass)  And many more

7. Image courtesy Paul Heckbert 1983

8. Outline  Ray Tracing  Global Illumination and Rendering Equation

9. Ray Tracing: History  Appel 68  Whitted 80 [recursive ray tracing]  Landmark in computer graphics  Lots of work on various geometric primitives  Lots of work on accelerations  Current Research  Real-Time raytracing (historically, slow technique)  Ray tracing architecture

10. Ray Tracing  Different Approach to Image Synthesis as compared to Hardware pipeline (OpenGL)  Pixel by Pixel instead of Object by Object  Easy to compute shadows/transparency/etc Demo applet: http://www.cs.berkeley.edu/~efros/java/tracer/tracer.htmlhttp://www.cs.berkeley.edu/~efros/java/tracer/tracer.html

11. Ray Casting Virtual Viewpoint Virtual ScreenObjects Ray misses all objects: Pixel colored blackRay intersects object: shade using color, lights, materialsMultiple intersections: Use closest one (as does OpenGL)

12. Shadows Virtual Viewpoint Virtual ScreenObjects Light Source Shadow ray to light is unblocked: object visibleShadow ray to light is blocked: object in shadow

13. Shadows: Numerical Issues Numerical inaccuracy may cause intersection to be below surface (effect exaggerated in figure) Causing surface to incorrectly shadow itself Move a little towards light before shooting shadow ray

14. Mirror Reflections/Refractions Virtual Viewpoint Virtual ScreenObjects Generate reflected ray in mirror direction, Get reflections and refractions of objects

15. Recursive Ray Tracing For each pixel  Trace Primary Eye Ray, find intersection  Trace Secondary Shadow Ray(s) to all light(s)  Color = Visible ? Illumination Model : 0 ;  Trace Reflected Ray  Color += reflectivity * Color of reflected ray

16. Problems with Recursion  Reflection rays may be traced forever  Generally, set maximum recursion depth  Same for transmitted rays (take refraction into account)

17. Turner Whitted 1980

18. Effects needed for Realism (Soft) Shadows Reflections (Mirrors and Glossy) Transparency (Water, Glass) Interreflections (Color Bleeding) Complex Illumination (Natural, Area Light) Realistic Materials (Velvet, Paints, Glass) Discussed in this lecture so far Not discussed but possible with distribution ray tracing Hard (but not impossible) with ray tracing; radiosity methods

19. Ray/Object Intersections  Heart of Ray Tracer  One of the main initial research areas  Optimized routines for wide variety of primitives  Various types of info  Shadow rays: Intersection/No Intersection  Primary rays: Point of intersection, material, normals  Texture coordinates  Work out examples in next lecture  Triangle, sphere, polygon, general implicit surface

20. Ray-Tracing Transformed Objects We have an optimized ray-sphere test  But we want to ray trace an ellipsoid… Solution: Ellipsoid transforms sphere  Apply inverse transform to ray, use ray-sphere  Allows for instancing (traffic jam of cars) Mathematical details worked out in class

21. Acceleration Testing each object for each ray is slow  Fewer Rays Adaptive sampling, depth control  Generalized Rays Beam tracing, cone tracing, pencil tracing etc.  Faster Intersections  Optimized Ray-Object Intersections  Fewer Intersections

22. Acceleration Structures Bounding boxes (possibly hierarchical) If no intersection bounding box, needn’t check objects Bounding Box Ray Spatial Hierarchies (Oct-trees, kd trees, BSP trees)

23. Acceleration Structures: Grids

24. Interactive Raytracing  Ray tracing historically slow  Now viable alternative for complex scenes  Key is sublinear complexity with acceleration; need not process all triangles in scene  Allows many effects hard in hardware  OpenRT project real-time ray tracing (http://www.openrt.de)

25. Raytracing on Graphics Hardware  Modern Programmable Hardware general streaming architecture  Can map various elements of ray tracing  Kernels like eye rays, intersect etc.  In vertex or fragment programs  Convergence between hardware, ray tracing [Purcell et al. 2002, 2003] http://graphics.stanford.edu/papers/photongfx

27. Outline  Ray Tracing  Global Illumination and Rendering Equation

28. Global Illumination Diffuse interreflection, color bleeding [Cornell Box]

29. Global Illumination Caustics: Focusing through specular surface Major research effort in 80s, 90s till today

30. Reflectance Equation (review) Reflected Light (Output Image) Emission Incident Light (from light source) BRDF Cosine of Incident angle

31. Reflectance Equation (review) Reflected Light (Output Image) Emission Incident Light (from light source) BRDF Cosine of Incident angle Sum over all light sources

32. Reflectance Equation (review) Reflected Light (Output Image) Emission Incident Light (from light source) BRDF Cosine of Incident angle Replace sum with integral

33. Global Illumination Reflected Light (Output Image) Emission Reflected Light (from surface) BRDF Cosine of Incident angle Surfaces (interreflection)

34. Rendering Equation Reflected Light (Output Image) Emission Reflected Light BRDF Cosine of Incident angle Surfaces (interreflection) UNKNOWN KNOWN

35. Rendering Equation (Kajiya 86)

36. Rendering Equation as Integral Equation Reflected Light (Output Image) Emission Reflected Light BRDF Cosine of Incident angle UNKNOWN KNOWN Is a Fredholm Integral Equation of second kind [extensively studied numerically] with canonical form Kernel of equation

37. Linear Operator Equation Kernel of equation Light Transport Operator Can be discretized to a simple matrix equation [or system of simultaneous linear equations] (L, E are vectors, K is the light transport matrix)

38. Ray Tracing and extensions –General class numerical Monte Carlo methods –Approximate set of all paths of light in scene Binomial Theorem

39. Ray Tracing Emission directly From light sources Direct Illumination on surfaces Global Illumination (One bounce indirect) [Mirrors, Refraction] (Two bounce indirect) [Caustics etc]

40. Ray Tracing Emission directly From light sources Direct Illumination on surfaces Global Illumination (One bounce indirect) [Mirrors, Refraction] (Two bounce indirect) [Caustics etc] OpenGL Shading

41. Rendering Equation Reflected Light (Output Image) Emission Reflected Light BRDF Cosine of Incident angle Surfaces (interreflection) UNKNOWN KNOWN

42. Change of Variables Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables)

43. Change of Variables Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables)

44. Rendering Equation: Standard Form Integral over angles sometimes insufficient. Write integral in terms of surface radiance only (change of variables) Domain integral awkward. Introduce binary visibility fn V Same as equation 2.52 Cohen Wallace. It swaps primed And unprimed, omits angular args of BRDF, - sign. Same as equation above 16.3 in Shirley, except he has no emission, slightly diff. notation