1. computer graphics & visualization Shadows / Transparency

2. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Inter-Object realism Covers different kinds of interactions between objects Covers different kinds of interactions between objects – Increasing realism in the scene – Relationships between objects easier to understand Shadows, Reflections, Transparency Shadows, Reflections, Transparency

3. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow techniques In illuminated scenes objects cast shadows on other objects – Important visual cue for object positioning – Scene without shadows doesn’t match reality

4. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow techniques Classification of shadow techniques – How to account for interaction between light, shadow casters and shadowed objects Projective Shadows Shadow Volumes Shadow Maps

5. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows – Shadows are projected copies of the shadowing objects onto shadowed surface – Good for complex objects casting shadows onto simple surfaces Objects Objects & projected objects Shadows

6. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Projection of shadow casters onto planes – Shadow projection is a projective mapping of the object onto the shadowed plane Directional or positional lights can be handled Directional or positional lights can be handled – Shadow projection transform S is added to the end of the modelview matrix S: render the scene from the light source position with the shadowed plane as the image plane S: render the scene from the light source position with the shadowed plane as the image plane Ax+By+Cz+D=0 L+t(P-L)=(x,y,z) T P L

7. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Multi-pass algorithm – Draw the surface to be shadowed – Render the object twice Render the object with original matrix, colors and textures as specified for that object Render the object with original matrix, colors and textures as specified for that object Disable texturing and lighting Disable texturing and lighting Render the object as its shadow with appended matrix S and the color of the shadow Render the object as its shadow with appended matrix S and the color of the shadow

8. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Problems of projective shadows – Projection must be clipped to shadowed surface – Solution: clip the projected polygon to the surface using the stencil buffer Draw polygon setting the stencil buffer Draw shadows where stencil is set Use stencil as mask

9. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Problems of projective shadows – Shadow color modulates receivers color Receiver should not just become darker Receiver should not just become darker – Solution: Render unlit surface again where stencil is set due to shadow projection Render unlit surface again where stencil is set due to shadow projection – Shadow shows color of textured surface

10. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Problems of projective shadows Problems of projective shadows – Z-Fighting: Projected object has „same“ depth values as shadowed plane z-fighting correct

11. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Problems of projective shadows Problems of projective shadows – In complex scenes multiple shadow projections are needed #Matrices depends on #ShadowedSurfaces and #Lights #Matrices depends on #ShadowedSurfaces and #Lights Objects have to be rendered multiple times Objects have to be rendered multiple times – #PotentiallyShadowedSurfaces – Need to find surfaces that are in shadow – Only works well on planar surfaces – For complex objects clipping has to be performed against every face

12. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Projective shadows Projective shadows in complex scenes Shadow clipped to multiple surfaces Culling of non- shadowed surfaces Modulate surface with shadow color

13. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes – Relies on a geometric representation of the spatial region that is shadowed by an object – Shadow polyhedra is generated in advance Project shilouette edges of the object onto shadowed surface and connect points Project shilouette edges of the object onto shadowed surface and connect points Finding shilouettes not a simple task Finding shilouettes not a simple task – Shadow algorithm now involves the stencil test and the depth test

14. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes When is a point in shadow? When is a point in shadow? Shadow volume algorithm [Crow 1977] Shadow volume algorithm [Crow 1977] – Intersect ray from eye with shadow volume – Start with 0, invert at every intersection point – If 0, point not in shadow – If eye in shadow, start with 1

15. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes Shadow volume geometry Shadow volume geometry Without shadows Shadow volume Final scene

16. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes Algorithm Algorithm – Render the unlit scene and clear the stencil buffer – Disable depth write and color write but leave depth test enabled – Render front-facing polygons of the shadow volume and increment the stencil where they pass the depth test – Render back-facing polygons of the shadow volume and decrement the stencil where they pass the depth test Result Result – Stencil value classifies fragments as shadowed or unshadowed Shadowed pixels end up with non-zero stencil Shadowed pixels end up with non-zero stencil

17. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadowing object Light source Eye position zero zero +1 +1 +2 +2 +3 Unshadowed object + - - - + + Shadow Volume Count = +1+1+1-1-1-1 = 0 Illuminated (Behind Shadow Volumes)

18. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadowing object Light source Eye position zero zero +1 +1 +2 +2 +3 Shadowed object + - + + Shadow Volume Count = +1+1+1-1 = 2 Shadowed (inside Shadow Volumes)

19. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadowing object Light source Eye position zero zero +1 +1 +2 +2 +3 Shadowed object Shadow Volume Count = 0 Illuminated (before Shadow Volumes)

20. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group zero zero +1 +1 +2 +2 +3 Near clip plane Far clip plane Missed shadow volume intersection due to near clip plane clipping; leads to mistaken count Problems

21. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes ZFail Algorithm Algorithm – Render the unlit scene and clear the stencil buffer – Disable depth write and color write but leave depth test enabled – Render back-facing polygons of the shadow volume and increment the stencil where they fail the depth test – Render front-facing polygons of the shadow volume and decrement the stencil where they fail the depth test Result Result – Stencil value classifies fragments as shadowed or unshadowed Shadowed pixels end up with non-zero stencil Shadowed pixels end up with non-zero stencil

22. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes OpenGL implementation (Zfail) glDisable(GL_LIGHTING); glEnable(GL_DEPTH_TEST); glDepthFunc(GL_LESS); glDepthMask(0); glColorMask(0,0,0,0); glStencilMask(255); glEnable(GL_CULL_FACES); glCullFace(GL_FRONT); glStencilOp(GL_KEEP, GL_INCR, GL_KEEP); for(i=0;i<numPolysInScene; i++) renderShadowVolumeForPolygon(i); glCullFaces(GL_BACK); glStencilOp(GL_KEEP, GL_DECR, GL_KEEP); for(i=0;i<numPolysInScene; i++) renderShadowVolumeForPolygon(i);

23. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow volumes Problems of shadow volumes – Huge number of shadow geometries have to be generated, stored and rendered Shadow volumes are in general too costly to be applied for complex scene Shadow volumes are in general too costly to be applied for complex scene Volumes have to be re-generated whenever the light source moves relatively to object Volumes have to be re-generated whenever the light source moves relatively to object Using simplified geometries can help Using simplified geometries can help

24. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes: Zfail Render scene to initialize depth buffer Render scene to initialize depth buffer – Depth values indicate the closest visible fragments Use a stencil enter/leave counting approach Use a stencil enter/leave counting approach – Draw shadow volume twice using face culling 1st pass: render back faces and increment when depth test fails 1st pass: render back faces and increment when depth test fails 2nd pass: render front faces and decrement when depth test fails 2nd pass: render front faces and decrement when depth test fails – Don’t update depth or color Pixel’s stencil is non-zero if pixel in shadow, and zero if illuminated Pixel’s stencil is non-zero if pixel in shadow, and zero if illuminated Problem now is far clipping plane Problem now is far clipping plane

25. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes: Zfail Conclusion: shadow volume approach requires clipping against near or far plane to be avoided Conclusion: shadow volume approach requires clipping against near or far plane to be avoided Avoiding near plane clipping Avoiding near plane clipping – Not really possible – Objects can always be behind you Avoiding far plane clipping Avoiding far plane clipping – Perspective make it possible to render at infinity

26. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes: Zfail Recall: glFrustum projection matrix Recall: glFrustum projection matrix

27. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes: Zfail Projection matrix with infinite Far

28. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes: Zfail What happens to vertex at infinity? – Most distant vertex is (0,0,-D,0) where D>0 OpenGL is looking along negative z OpenGL is looking along negative z – Transform (0,0,-D,0) to window space – Vertex is still in valid depth range; it is not clipped

29. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes Shadow volume must be closed Three sets of polygons close the shadow volume – Possible silhouette edges extruded to infinity away from the light position L – All of the occluder’s back-facing (w.r.t. the light) triangles projected away from the light to infinity – All of the occluder’s front-facing (w.r.t. the light) triangles are “slightly” projected away from the light Use homogeneous vector differences to project vertex V to infinity along the light source direction – V-L represent direction heading away from the light – Has w coordinate equal to zero

30. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes Example Example

31. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes Algorithm using programmable vertex shader – Specify for each edge a shadow edge by duplicating edge vertices Create so-called zero-area quads Create so-called zero-area quads – Test for silhouette edge, front/back facing triangles – Extrude respective vertices to infinity in the vertex shader

32. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Hardware shadow volumes Algorithm using DirectX10Graphics – Includes a geometry shader Sits behind the vertex shader Sits behind the vertex shader Provides functionality to amplify/deamplify primitives Provides functionality to amplify/deamplify primitives – For shadow volumes: duplicate silhouette edges and extrude to infinity

33. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow maps Shadow maps – a two-pass approach – Render the scene from the light source position using a virtual image plane – Grab the depth buffer and store it as a shadow map – Before rendering a fragment from the camera Project it towards the light source onto the shadow map Project it towards the light source onto the shadow map Discard fragment if distance to light source is less than entry in shadow map Discard fragment if distance to light source is less than entry in shadow map Scene Shadow map

34. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow maps Implementation – Generate homogeneous texture coordinates (s,t,r,q) as light space coordinates (x,y,z,w) Texture matrix to be issued

35. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Steps to Shadow Mapping Scene from EyeScene from LightDepth from Light Projected Depth Depth ComparisonFinal Scene

36. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow maps Algorithm Algorithm – Render scene from light source – Read depth buffer and store as 2D texture map Use glCopyTexImage2D with glReadBuffer(GL_DEPTH_BUFFER) Use glCopyTexImage2D with glReadBuffer(GL_DEPTH_BUFFER) – Render lit/textured scene Use shadow map as projective texture Use shadow map as projective texture Compare r/q texture coordinate with shadow map entry Compare r/q texture coordinate with shadow map entry Alternatively employ hardware support Alternatively employ hardware support – Does depth comparison as texture filtering operation – Shadow map fragment color is 0 or 1 Modulate fragment color with shadow map color Modulate fragment color with shadow map color

37. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow maps Pros and cons of shadow maps – General method for computing all shadows in a scene – Representation of shadows is independent of scene complexity – But, artifacts due to discrete sampling of the shadow map and quantization of depth values 4096x40961024x1024

38. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow textures – Render shadow on the receiver and load it as shadow texture Light source and shadow object dependent Light source and shadow object dependent Unshadowed Scene Shadow Texture Texture mapped floor

39. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Shadow techniques Soft shadows – Similar to motion blur – Assume area light source Jitter light position within area Jitter light position within area Render from jittered positions Render from jittered positions Combine results in the accumulation buffer Combine results in the accumulation buffer Use decreasing scale with increasing distance to center of area

40. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Transparency How to handle non-opaque (semi-transparent) objects – Multiple fragments contribute to a pixel – Visibility order of fragments is important – Depth buffer can be used for hidden surface removal but not for semi-transparent objects

41. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Transparency See-through objects Blending non commutative Object order affects final color Blending non commutative  1 C 1 + (1- 1 ) 0 C 0   0 C 0 + (1- 0 ) 1 C 1 Depth test discards objects

42. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Transparency General solution – Visibility sort all semi-transparent objects in the scene – Render opaque objects first and update the depth values – Render semi-transparent objects in visibility order Apply depth test but do not alter depth values Apply depth test but do not alter depth values – Use alpha-blending in order to obtain linear combination of src and dst color

43. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Transparency Correct transparency by sorting and alpha-blending

44. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling Recall: standard depth test gives nearest fragment at each pixel without imposing any order But: there is also a second, third … fragment, depth test does not provide a means to render the nth nearest surface. Depth peeling solves this problem: – With n passes over a scene, n layers deeper into the scene are generated – E.g. with 2 passes, the nearest and second nearest surfaces are extracted – Both the depth and RGBA information is obtained for each layer

45. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling Example

46. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling The method peels different depth layers in front- to-back order The method peels different depth layers in front- to-back order Drawback: as many rendering passes as objects depth complexity (maximum number of surface points mapping to one fragment) Drawback: as many rendering passes as objects depth complexity (maximum number of surface points mapping to one fragment)

47. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling How to determine the depth complexity – Render the scene without depth test and increment the stencil bit (or entries in a texture render target using additive blend) whenever a fragment is generated – „Reduce“ the buffer to obtain the maximum In each pass, maximum of 4 texels is computed and stored into 1 texel of output texture In each pass, maximum of 4 texels is computed and stored into 1 texel of output texture Reduce m x m region in fragment shader 1 pass: render quad that covers 1/4 pixels 2 pass: render quad that covers 1/16 pixels …

48. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling Algorithm: – First pass: render as normal - depth test gives us nearest surface – Second pass: Use depth buffer computed in the first pass to “peel away” depths less than or equal to nearest depths from the first pass All other depths (for fragments passing the first test) are resolved by the standard depth test – The second pass generates depths of the second nearest surface, which are used to peel away the first and second nearest surfaces in the third pass

49. computer graphics & visualization Image Synthesis – WS 07/08 Dr. Jens Krüger – Computer Graphics and Visualization Group Depth peeling Depth peeling issues – Depth peeling needs two depth tests In the nth pass a test with the (n-1)th nearest depths The standard depth test – The first test can be done in a fragment shader Therefore, in each pass the depth values are rendered into a texture render target – The depth test remains active even if rendering is to such a target In the upcoming pass, a shader reads the depth value from this target and discard fragments with depth values less than this value All surviving fragments are passed through the standard test – Use 32 Bit float texture render target for highest accuracy