1 Illumination and Shading Day 6, 2013 © Jeff Parker

2 Outline Gallery Goal all Semester: Realistic images Review lighting in the real world Types of illumination Shading Strategies Shading Algorithms Limitations of shading algorithms Alternatives

3 Gallery

4 Dan Chris

Fix function perspective( fovy, aspect, near, far ) { // Polina's fix var f = 1.0 / Math.tan( radians(fovy) / 2 ); // var d = far - near; var d = -(far - near); var result = mat4(); result[0][0] = f / aspect; result[1][1] = f; result[2][2] = (near + far) / d; result[2][3] = 2 * near * far / d; result[3][2] = -1; result[3][3] = 0.0; return result; } The on-line version has been revised

6 Outline Goal in Computer Graphics is to make something good enough to use For design work For entertainment For instruction … Nothing we have done so far looks "real" Tonight we will take a big step forward - lighting

7 Mike

8 Derrick

9 Dan

10 Outline Gallery Goal all Semester: Realistic images Review lighting in the real world Types of illumination Shading Strategies Shading Algorithms Limitations of shading algorithms Alternatives

11 Examples We start by looking at light in some photos What do you see?

12 Examples

13 Examples

14 Examples

Luxo Jr Pixar's first movie

Spectrum Light is visible electromagnetic radiation

17 Observations Light is generated by a source Sun, Light bulb, phosphorescence,… Light travels through space, air, glass, water… Travels in straight line – may bend at boundaries When light strikes object some wavelengths are absorbed, and some are reflected The reflected light travels through air, glass, … Some of the reflected light reaches our eyes

18 Lambert's Law Different types of finish Matt vs. glossy Smooth, bumpy, brushed Most surfaces are matt: many man made surfaces are smooth Lambertian – luminance is isotropic Looks the same from every viewpoint Luminance depends on angle of incidence Sunlight is strong in summer, weak in winter

19 Snell's law Snell's law decribes how light bends moving from one medium to another

20 Fresnel's law Fresnel's law extends Snell's law Refraction is often accompanied by reflection Fresnel also invented the Fresnel lens Only uses Snell's Law

21 Milestones Hooke (1665) – Light is rapid vibration (wave) Newton (1666) – Light is a corpuscle (particle) Young (1801) – Double-Slit experiment - wave Maxwell (1860) – Electromagnetic disturbance Einstein (1905) – Photoelectric effect - Light comes in discrete quanta (photons) – 1921 Nobel Prize Quantum Mechanics – Light is both particle and wave

22 Wave or Particle? Experimental evidence for the two theories PhenomenonWaveParticle Reflectionyes Refractionyes Interferenceyesno Diffractionyesno Polarizationyesno Photoelectric Effectnoyes

23 Human sensitivity The cones sense different wavelengths The original version of the Retinylidene protein is thought to have been tuned for green light

24 Spectrum of sources Compare incandescent to sunlight

25 Halogen Bulb Fresnell's law extends Snell's law Refraction is often accompanied by reflection

26 Blacklight Compare incandescent to sunlight

27 Fluorescent Bulb Fresnell's law extends Snell's law Refraction is often accompanied by reflection

28 Takeaway Different light sources emit different spectra The same scene looks different under different light OpenGL gives you control over your light source's spectrum

29 Light Sources As well as direct light, there are many indirect sources We call this ambient light Other points get direct illumination

30 Point source Some lights are from a radial point source Light bulb: rays radiate Some sources are directional – all the rays are ~ parallel This is an ideal model – best approximation for distant sources

31 Other Light Sources Spot Lights Area light source A 2D area emits light Soft shadows Extended light source Spherical Light source (3D) Soft shadows

32 Hard and Soft Shadows Hard shadows are created by point sources

33 Shadows Shadows are important in creating atmosphere

34 Putting it together The illumination of a scene is made of two parts The light sources Emitted spectrum Geometry (position and direction) Attenuation – energy decreases with distance Surface Reflectance spectrum Geometry Absorption

35 Shading options We will look at alternative forms of shading

36 Shading options We will look at alternative forms of shading

37 Shading options The Quest for Realism (Jim Blinn, 1980) A brief history of lighting in Computer Graphics

38 Reflectance Models We will break down light into three components Ambient Light Diffuse light – point light source, matt finish Specular – point light source, glossy finish Each component will have three sub-components Red, Green, Blue – 9 values in all

39 Diffuse Light: Matt surface A Lambertian surface (e.g. unfinished wood) has the same brightness, whatever the viewer's angle. Not all patches of a matt surface have the same brightness: it depends upon position of light source The Diffuse Intensity (I D ) is equal to the cosine of the angle between the Normal vector N and the vector towards the light, L, multiplied by the Intensity of the light, I L. L and N are assumed to be of unit length (normalized)

40 Diffuse Light: Matt surface This is a step forward – can tell where the light is But we see the edges of the patches

41 Gouraud Henri Gouraud, 1971, PhD University of Utah Interpolate shading over a polygon Good results without requiring small polygons Start with normal vector for each polygon At each vertex, average incident normals to get per-vertex "normal" Use "vertex normals" to compute color for each vertex Interpolate colors across each face Sylvie Gouraud

42 Phong Bui Tuong Phong, 1973, PhD University of Utah Interpolate shading over a polygon Start with normal vector for each polygon At each vertex, average incident normals to get per-vertex "normal" Interpolate normals across faces Use the interpolated normal to compute the color Gouraud shading: Compute normals at each vertex, compute color Then interpolate color for each fragment Phong shading: have a unique normal at each fragment Compute normal at each vertex, as before Interpolate normals across face Fresh computation of color from normal at each fragment Thus a per fragment computation

43 Specular Reflections Bui Tuong Phong, 1973, PhD University of Utah Approximate the specular highlights on a surface Bi-Directional Reflectance Distribution Function (BRDF)

44 BRDF sampling Special purpose hardware to sample BRDF

45 BRDF sampling Sample values

46 LightPosition

47 LightMaterial

48 Specular Highlights For specular component, we use 4 vectors at each point l Light: points towards light v Viewer: points towards viewer n Normal to the point r Reflector: Where light would bounce

49 Specular Highlight Phong added a specular term Strong as vector v approaches reflected vector r, drops off as they differ (angle phi) I r = Reflected Intensity k s = Absorption coef I = Incoming Intensity alpha = shininess coef 

50 Computing Reflector r The angle of reflection must equal the angle of incidence The three vectors should be co-planar

51 Shininess Coef Metals have small specular reflection – large alpha ( ) Plastics have wide specular reflection – smaller alpha (5-10)

52 Blinn-Phong Jim Blinn proposed alternative to Phong model Approximate the angle phi Uses h, the "halfway" vector midway between l and v The new angle psi is half phi if all vectors are coplanar. Given alpha, we find matching power beta. OpenGL uses Flat or Gouraud with Blinn-Phong shading 

53 Why is this any better? If we are using Phong shading, the normal n varies as we interpolate over each polygon. As n changes, we must recompute the vector r Given direct light and a distant eye, l and v do not change, or do not change as quickly. We can quickly compute a dot product on the fly as n varies

54 Distance Term Light from distant source is inversely proportional to the distance squared We can add a term of the form 1/(a + bd + cd 2 )

55 Light Sources We add together the effects of the three forms of light (Ambient, Diffuse, Specular) over the three colors This gives us 9 terms {d, s, a} x {r, g, b}

56 Material Properties Each material can be configured to react distinctly to each type of light, and to each color Gives us 9 absorption coefficients and one shineness coefficient alpha

57 Phong Illumination Model For each light source, and for each color component, the Phong model gives the following equation For each color, we add up the contributions from all lights

58 Compare Phong vs Blinn-Phong

Example 59 teapots.c from The Redbook Single light variety of material parameters

Example 60 material.c from The Redbook Single light variety of material parameters See next page…

61 * The spheres in the first row have materials with no ambient reflection. * The second row has materials with significant ambient reflection. * The third row has materials with colored ambient reflection. * The first column has materials with blue, diffuse reflection only. * The second column has blue diffuse reflection, as well as specular * reflection with a low shininess exponent. * The third column has blue diffuse reflection, as well as specular * reflection with a high shininess exponent (a more concentrated highlight). * The fourth column has materials which also include an emissive component. * glTranslatef() is used to move spheres to their appropriate locations.

Light // Initialize shader lighting parameters in.js var lightPosition = vec4(1.0, 1.0, 1.0, 0.0 ); var lightAmbient = vec4(0.2, 0.2, 0.2, 1.0 ); var lightDiffuse = vec4( 1.0, 1.0, 1.0, 1.0 ); var lightSpecular = vec4( 1.0, 1.0, 1.0, 1.0 ); var materialAmbient = vec4( 1.0, 0.0, 1.0, 1.0 ); var materialDiffuse = vec4( 1.0, 0.8, 0.0, 1.0 ); var materialSpecular = vec4( 1.0, 0.8, 0.0, 1.0 ); var materialShininess = 100.0; 62

Gloss // Point source: RHS is a point var lightPosition = vec4(1.0, 1.0, 1.0, 1.0 ); // Directional source: RHS is a vector var lightPosition = vec4(1.0, 1.0, 1.0, 0.0 ); 63

Light ambientProduct = mult(lightAmbient, materialAmbient); diffuseProduct = mult(lightDiffuse, materialDiffuse); specularProduct = mult(lightSpecular, materialSpecular); gl.uniform4fv(gl.getUniformLocation(program, "ambientProduct"),flatten(ambientProduct)); gl.uniform4fv(gl.getUniformLocation(program, "diffuseProduct"),flatten(diffuseProduct) ); gl.uniform4fv(gl.getUniformLocation(program, "specularProduct"),flatten(specularProduct) ); gl.uniform4fv(gl.getUniformLocation(program, "lightPosition"),flatten(lightPosition) ); gl.uniform1f(gl.getUniformLocation(program, "shininess"), materialShininess); gl.uniformMatrix4fv( gl.getUniformLocation(program, "projectionMatrix"), false, flatten(projection));

Vertex Shader for Sphere 1 attribute vec4 vPosition; attribute vec4 vNormal; varying vec4 fColor; uniform vec4 ambientProduct, diffuseProduct, specularProduct; uniform mat4 modelViewMatrix; uniform mat4 projectionMatrix; uniform vec4 lightPosition; uniform float shininess; void main() { vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz; vec3 L = normalize( light - pos ); vec3 E = normalize( -pos ); vec3 H = normalize( L + E ); // Transform vertex normal into eye coordinates vec3 N = normalize( (modelViewMatrix*vNormal).xyz); // Compute terms in the illumination equation vec4 ambient = ambientProduct; float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*diffuseProduct; float Ks = pow( max(dot(N, H), 0.0), shininess ); vec4 specular = Ks * specularProduct; if( dot(L, N) < 0.0 ) { specular = vec4(0.0, 0.0, 0.0, 1.0); } gl_Position = projectionMatrix * modelViewMatrix * vPosition; fColor = ambient + diffuse +specular; fColor.a = 1.0; } 65

Fragment Shader for Sphere 1 precision mediump float; varying vec4 fColor; void main() { gl_FragColor = fColor; }

Vertex Shader for Sphere 2 attribute vec4 vPosition; attribute vec4 vNormal; varying vec3 N, L, E; uniform mat4 modelViewMatrix; uniform mat4 projectionMatrix; uniform vec4 lightPosition; void main() { vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz; L = normalize( light - pos ); E = -pos; N = normalize( (modelViewMatrix*vNormal).xyz); gl_Position = projectionMatrix * modelViewMatrix * vPosition; } 67

Fragment Shader for Sphere 2 precision mediump float; uniform vec4 ambientProduct; uniform vec4 diffuseProduct; uniform vec4 specularProduct; uniform float shininess; varying vec3 N, L, E; void main() { vec4 fColor; vec3 H = normalize( L + E ); vec4 ambient = ambientProduct; float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*diffuseProduct; float Ks = pow( max(dot(N, H), 0.0), shininess ); vec4 specular = Ks * specularProduct; if( dot(L, N) < 0.0 ) { specular = vec4(0.0, 0.0, 0.0, 1.0); } fColor = ambient + diffuse +specular; fColor.a = 1.0; gl_FragColor = fColor; }

Vertex Shader for 1 attribute vec4 vPosition; attribute vec4 vNormal; varying vec4 fColor; uniform vec4 ambientProduct, diffuseProduct, specularProduct; uniform mat4 modelViewMatrix; uniform mat4 projectionMatrix; uniform vec4 lightPosition; uniform float shininess;

Vertex Shader for 1 void main() { vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz; vec3 L = normalize( light - pos ); vec3 E = normalize( -pos ); // Eye vec3 H = normalize( L + E ); // Transform vertex normal into eye coordinates vec3 N = normalize( (modelViewMatrix*vNormal).xyz); // Compute terms in the illumination equation vec4 ambient = ambientProduct;

Vertex Shader for 1 float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*diffuseProduct; float Ks = pow( max(dot(N, H), 0.0), shininess ); vec4 specular = Ks * specularProduct; if ( dot(L, N) < 0.0 ) { specular = vec4(0.0, 0.0, 0.0, 1.0); } gl_Position = projectionMatrix * modelViewMatrix * vPosition; fColor = ambient + diffuse + specular; fColor.a = 1.0; }

Fragment Shader for Sphere 1 precision mediump float; varying vec4 fColor; void main() { gl_FragColor = fColor; }

Vertex Shader for Sphere 2... void main() { vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz; L = normalize( light - pos ); E = -pos; N = normalize( (modelViewMatrix*vNormal).xyz); gl_Position = projectionMatrix * modelViewMatrix * vPosition; } 73

Fragment Shader for 2 precision mediump float; uniform vec4 ambientProduct; uniform vec4 diffuseProduct; uniform vec4 specularProduct; uniform float shininess; varying vec3 N, L, E; void main() { vec4 fColor; vec3 H = normalize( L + E ); vec4 ambient = ambientProduct;

Fragment Shader for 2 float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*diffuseProduct; float Ks = pow( max(dot(N, H), 0.0), shininess ); vec4 specular = Ks * specularProduct; if ( dot(L, N) < 0.0 ) { specular = vec4(0.0, 0.0, 0.0, 1.0); } fColor = ambient + diffuse + specular; fColor.a = 1.0; gl_FragColor = fColor; }

Difference in JavaScript 1 & 2 function triangle(a, b, c) { normalsArray.push(a); normalsArray.push(b); normalsArray.push(c); pointsArray.push(a); pointsArray.push(b); pointsArray.push(c); index += 3; } The essential difference is how we compute normals special case: p for sphere at origin is p

Difference in JavaScript 3 & 4 function triangle(a, b, c) { var t1 = subtract(b, a); var t2 = subtract(c, a); var normal = normalize(cross(t1, t2)); normal = vec4(normal); normalsArray.push(normal);... p0p0 p1p1 p2p2 n p

Early Fragment Test Don't perform full computation for fragments we will cull

Early Fragment Test Don't perform full computation for fragments we will cull Two passes over fragments First pass computes the depth Second pass checks the depth: if this fragment is live, it computes the color Don't need to write to depth buffer in second pass

Homework Cube Rotations Lost World – check out Karl Sim's Evolved Virtual Life 80

Conclusion Lighting is difficult There are techniques that model reality The implementations often simplify reality to achieve speed We have looked at the most common model Purely local: for example, does not create shadows, refraction, reflections, caustics, … For other effects, need fancier shaders or Ray Tracing 81