1. Advanced Computer Graphics Bas Zalmstra Marries van de Hoef Tim de Jager

2. Ask questions

3. Who the.. are these guys?! Graphics teaching assistants http://marries.nl http://baszalmstra.com

4. Who the.. are these guys?! Marries van de Hoef Experimental Game Developer @marriesvdh www.marries.nl

5. Seven Dimensions PLAY NAO -> www.marries.nl

6. Who the.. are these guys?! Bas Zalmstra Lead Developer @ Abbey Games @baszalmstra www.baszalmstra.nl

7. Who the.. are these guys?! Tim de Jager Game Developer @ Abbey Games @tldejager

8. Reus www.reusgame.com

9. What are we talking about? Deferred shading variants Culling Variance Shadow mapping Subsurface scattering GPU Particle Physics

10. DEFERRED SHADING VARIANTS

11. Dynamic lighting – Calculating lighting real-time – Moving objects – Moving light sources

12. A lot of variants Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

13. FORWARD SHADING Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

14. Forward shading Multiple light sources – Graphics practical 3 assignment E3 – Additive blending of light sources

15. Forward shading Lighting is calculated while rendering geometry – Calculated for the entire surface Two methods: Add multiple light sources in single shader – Variable number of light sources in a shader Render the object multiple times (additively) – Unnecessary vertex shader calculations

16. DEFERRED SHADING Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

17. Deferred shading Render lighting per light source – Render the bounding volume

18. Deferred shading What do we need for (Lambertian) shading? – Object: Normals Position Surface color (albedo) – Light source: Position Lighting color Object data is not available when rendering light volumes

19. Deferred shading Store object properties in buffers (Texture2D’s)! – Everything has to be rendered once – Lighting uses data from buffers Geometry-Buffer (G-buffer): NormalsDepth (Position)Albedo

20. Deferred shading For each light source – Use data from the G-buffer – Light source properties as shader parameters NormalsDepth (Position)Albedo

21. DEFERRED LIGHTING Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

22. Deferred lighting Deferred shading has limited object variability. – Defined by the properties in the G-Buffer. What if we want a different shader per object?

23. Deferred lighting (Also called Light pre-pass rendering) Only calculate lighting – Specular lighting, diffuse lighting – DON’T combine with diffuse/specular color yet

24. Deferred lighting Render all objects again For each object – Use lighting buffer as lighting information – We can use a different shader for each object!

25. Deferred lighting Recap: – Render each object to G-Buffer: – Use G-Buffer to calculate lighting: – Render each object again: Use lighting buffer for shading

26. Deferred lighting We also want different lighting per object Still not possible…

27. INFERRED LIGHTING Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

28. Inferred lighting What about transparency – A buffer can store 1 value per pixel – Use stippling for the lighting Search for surface when fetching the lighting

29. Inferred lighting Lighting does not vary much over a surface – Lighting could be calculated at a lower resolution Also search for the correct surface when sampling from the lighting buffer

30. Inferred lighting At lower resolution – Render each object to G-Buffer: – Use G-Buffer to calculate lighting: Render each object again: – Use lighting buffer for shading

31. TILE BASED DEFERRED SHADING Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

32. Tile based deferred shading Rendering light volumes is slow – A lot of overdraw Render lighting per tile – List of active light sources per tile Frustum culling (on GPU?)

33. FORWARD+ Forward Shading Deferred Shading Deferred Lighting Inferred Lighting Tile based deferred shading Forward+ Light indexed deferred shading

34. Forward+ We want different lighting per object Don’t calculate lighting, but store which light sources are active Store light indices per tile – Similar to tile based deferred rendering

35. Forward+ Light indexed deferred rendering – Store light indices for each pixel in a texture Limited number of light indices Forward+ – Published in 2012 – Store light indices in DX11 linked list structure

36. Usage in games Forward shading: Half-life 2, Portal, etc. Deferred shading: Killzone 2, Starcraft 2, etc. Deferred lighting: Most games Inferred lighting: Red faction / Saints Row Tile based def. sh.: Battlefield 3, Uncharted, …? Forward+ : Dirt Showdown (so far)

37. Questions?

38. CULLING

39. What is culling? You No culling

40. What is culling? You No culling

41. What is culling? You Frustum culling Cannot see that!

42. What is culling? You Frustum + Occlusion culling Cannot see that!

43. CULLING Frustum culling Occlusion culling

44. Frustum culling: Traditional approaches Culling a few thousand objects is slow – Need optimizations Create spatial hierarchy – Useful for quickly culling a lot of objects – Bounding Volume Hierarchy (BVH) Battlefield 1 and 2 use a Bounding Sphere Hierarchy

45. Bounding Sphere Hierarchy A B C A BC B+C A+B+C

46. Problems Hierarchical culling doesn’t scale well – Most computers have 4 or more cores Hard to handle dynamic objects Streaming in new parts of a level

47. Solution Just do it parallel brute force – Scales over all available cores – No overhead of tree – CPU can do a lot more low level optimizations Statistics (DICE): – 3 times faster – 1/5 of the original code – Seamlessly handles dynamic objects – Easy to optimize further

48. Grid

49. CULLING Frustum culling Occlusion culling

50. Oclusion culling: Traditional approaches Generate a static occlusion culling data structure Used in Quake1-3 Also used in newer engines like Unity3 Umbra (used in Unreal Engine 3)

51. Occlusion grid

52. Visible from within the yellow cell Not visible from within the yellow cell

53. Problems Doesn’t handle dynamic objects as occluders CANT BLOW UP THE EVERYTHING!!

54. Solution 1 Simply render the object and see if pixels made it on the screen – Low detail version Supported by todays hardware – Hardware Occlusion Queries Downside – The GPU is already busy enough with cool effects – CPU has to wait for the GPU to answer Stalls the CPU

55. Solution 2 Do the same thing but on the CPU – Moves the burden from the GPU to the CPU – Used in the Frostbite engine and the Cryengine Artists make occlusion geometry for occluders – Very large objects CPU rasterizes these occluders – 256x114 depth buffer – Parallel Test all objects in the frustum in parallel against the depth buffer

56. Software rasterizer

57. Recap Frustum culling – Selecting the objects in the viewing volume Occlusion culling – GPU: Hardware Occlusion Queries – CPU: Software Rasterizer References – Battlefield 3: Culling the Battlefield - Daniel Collin (DICE) – Practical Occlusion Culling on the PS3 - Will Vale

58. Questions?

59. VARIANCE SHADOW MAPPING

60. Regular shadow mapping Stage 1:

61. Regular shadow mapping Stage 2:

62. Shadow mapping problems Problem is: – Biasing Hard edges Depth

63. Shadow mapping problems Biasing: – Shadow acne

64. Improving shadow mapping We can’t filter the images

65. Improving shadow maps Percentage Close Filtering PCF Result Regular

66. Filtering We need to think of something So that we can filter...

67. Variance shadow maps Represent not as a single depth but as a distribution Use first and second moments to recover distribution: – M1 & M2

68. Algorithm 1. Store both depth and squared depth 2. Recover mean and variance using M1 & M2 3. Use Chebychev’s inequality to compute an upper bound 4. Use the upper bound to compute the lighting percentage

69. Expected value

70. First and Second Moment First and second moment can be recovered by: The mean and variance can be recovered by:

71. Chebychev!

72. Single occluder and surface

73. Computing the mean and variance: Where p represent the fraction that is onoccluded

74. Single occluder and surface Chebychev’s inequality returns the fraction that is unfiltered:

75. Single occluder and surface 2 2 4 4

76. Results

77. Problems Light bleeding: Solutions offered by: – http://www.punkuser.net/lvsm/lvsm_web.pdf http://www.punkuser.net/lvsm/lvsm_web.pdf – http://http.developer.nvidia.com/GPUGems3/gpugem s3_ch08.html http://http.developer.nvidia.com/GPUGems3/gpugem s3_ch08.html

78. Questions?

79. SUBSURFACE SCATTERING

80. Subsurface Scattering Simulate partial translucency

81. Subsurface Scattering Affected by volume and internal structure Makes some objects look more lively

82. Subsurface Scattering Light scatters diffusely through the material Very complex

83. Subsurface Scattering To use this in games: approximate (cheat) As long as it looks good

84. Subsurface Scattering By Colin Barré-Brisebois and Marc Bouchard from EA – Used in Battlefield 3

85. Subsurface Scattering Translucency depends on – The position of the light source – The position of the camera V · -L

86. Subsurface Scattering Translucency depends on – The position of the light source – The position of the camera – The thickness of the object Camera thickness Light

87. Subsurface Scattering Precompute local thickness

88. Subsurface Scattering Local thickness map – Add color to simulate internal color – Add detail to simulate internal structure

89. Subsurface Scattering

90. Can be incorrect with concave objects

91. Subsurface Scattering More information? Check http://zigguratvertigo.comhttp://zigguratvertigo.com

92. Questions?

93. GPU PARTICLE PHYSICS

94. GPU Particle Physics 25000 Particles colliding with the environment CPU processing not an option – Slow CPU-GPU communication – A lot of physics

95. GPU Particle Physics By Chris Tchou from Bungie – Used in Halo Reach Do everything on the GPU

96. GPU Particle Physics Particles are stored in buffers (Texture2D’s) Texture2D for: – Position – Velocity – Particle data (lifetime, appearance, lighting,...) One pixel in the Texture2D contains data for one particle

97. GPU Particle Physics Spawning new particles – CPU draws on the next pixels in the Texture2D – Set position, velocity, etc... 12345678910111213141516 17181920212223242526272829303132 333435363738 Old particles Add new particles here

98. GPU Particle Physics

99. GPU Particle Physics Collision detection Normal collision detection is not an option – For every particle, world geometry would have to be retrieved and checked... – Very inefficient

100. GPU Particle Physics Use deferred shading buffers instead! NormalsDepth (Position)

101. GPU Particle Physics Detect collision – Compare depth of particle with depth in buffer Depth (Position) Camera Particle Depthbuffer Depth to Particle Depth from buffer

102. GPU Particle Physics Detect collision – Compare depth of particle with depth in buffer Depth (Position) Camera Particle Depthbuffer Depth to Particle Depth from buffer

103. GPU Particle Physics Handle collision – Retrieve surface normal Camera Particle Normals Normal Velocity

104. GPU Particle Physics Handle collision – Retrieve surface normal Reflect velocity Apply damping Camera Particle Normals Normal Velocity

105. GPU Particle Physics

106. GPU Particle Physics Disadvantages: – Particles can only bounce against objects on screen – Collision errors can occur with high speed particles

107. Questions?

108. Thanks! Thanks for listening

109. Shout outs Wolfgang Hürst DGDARC for the juizz DGG for being awesome Vleeschtorpedo