2. Part 4: Many-Light Rendering on Mobile Hardware 40 min Markus Billeter VMML University of Zürich

3. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Many-Light on Mobiles Outline

4. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Many-Light on Mobiles Outline –What’s different?

5. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Many-Light on Mobiles Outline –What’s different? –Review with a twist towards Mobile HW

6. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Many-Light on Mobiles Outline –What’s different? –Review with a twist towards Mobile HW –Case study: Clustered Implementation

7. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 M OBILE H ARDWARE Part 1

8. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile vs. Desktop So far: considered modern high-end systems How is mobile hardware different?

9. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile vs. Desktop (II) Considerations: –Lower specs, similar amounts of pixels

10. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile vs. Desktop (II) Considerations: –Lower specs, similar amounts of pixels –Energy consumption (more) important –Fewer features for now…

11. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 A few numbers… (I) Android OpenGL Version: OpenGL ES VersionPercentage 2.063% 3.035% 3.12% Data from June 2015 Source: http://developer.android.com/about/dashboards/index.html

12. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 A few numbers… (II) Unity Mobile Stats (various platforms): OpenGL ES Version(*)Percentage 2.058% 3.040% 3.1~2% Data from April 2015; (*) = Shader Generations Source: http://hwstats.uinty3d.com/mobile/gpu.html

13. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 A few numbers… (III) Respectable amount of ES 3.0 devices Very few ES 3.1 devices for now :-( Big chunk of ES 2.0

14. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 A few numbers… (III) Respectable amount of ES 3.0 devices Very few ES 3.1 devices for now :-( Big chunk of ES 2.0 … will include ES 2.0 some considerations

15. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 OpenGL ES ES 2.0 : shader-based and FBO support –But no MRT in core

16. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 OpenGL ES ES 2.0 : shader-based and FBO support –But no MRT in core ES 3.0 “fixes” that

17. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 GPU Compute What about compute shaders?

18. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 GPU Compute What about compute shaders? –Tricky

19. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 GPU Compute (II) OpenCL –Typically not officially supported –(some devices have it anyway)

20. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 GPU Compute (II) OpenCL –Typically not officially supported –(some devices have it anyway) GL Compute >= ES 3.1

21. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile Architecture

22. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile Architecture Mobile GPUs are commonly tile based –“TBR” (Tile Based Renderer) Contrast: desktop GPUs are IMR –“IMR” = Immediate Mode Renderer

23. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Mobile Architecture Mobile GPUs are commonly tile based –“TBR” (Tile Based Renderer) Contrast: desktop GPUs are IMR –“IMR” = Immediate Mode Renderer –IMR mobile GPUs exist too, though.

24. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Immediate Mode Renderer (IMR) “Normal” rendering Geometry transformed & rasterized immediately

25. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Immediate Mode Renderer (IMR) “Normal” rendering Geometry transformed & rasterized immediately Framebuffer resides mainly in VRAM

26. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Immediate Mode Renderer (IMR) “Normal” rendering Geometry transformed & rasterized immediately Framebuffer resides mainly in VRAM Multiple writes to VRAM

27. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR)

28. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR) Framebuffer divided into tiles Geometry binned into tiles

29. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR) Framebuffer divided into tiles Geometry binned into tiles Tiles processed later independently

30. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR) Framebuffer divided into tiles Geometry binned into tiles Tiles processed later independently Tile’s portion of the framebuffer stays in fast on-chip memory!

31. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR) (II) Tile’s FB-contents stored to RAM when needed

32. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tile Based Renderer (TBR) (II) Tile’s FB-contents stored to RAM when needed Best case: once –When all rendering to that tile has finished

33. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Some Examples By the presented classification –TBR: Mali, Adreno, PowerVR … –IMR: Tegra (and most desktop GPUs)

34. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Intro - Summary Majority of mobile GPUs are TBR –Pick a method that works well with this

35. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Intro – Summary (II) TBR: keep tile’s portion of the FB on-chip

36. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Intro – Summary (II) TBR: keep tile’s portion of the FB on-chip –Make sure it stays there –Loads/stores from/to RAM use precious BW

37. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Intro – Summary (II) TBR: keep tile’s portion of the FB on-chip –Make sure it stays there –Loads/stores from/to RAM use precious BW Goal: keep data on-chip when possible

38. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Intro – Summary (II) TBR: keep tile’s portion of the FB on-chip –Make sure it stays there –Loads/stores from/to RAM use precious BW Goal: keep data on-chip when possible –Preferably without affecting IMR negatively

39. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Important! Tile-based renderer (TBR) != Tiled Shading

40. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Important! TBR: hardware property –It’s out of your hands Tiled Shading: software algorithm –Your choice Src: Wikipedia/Exynos, by User:Köf3, CC-SA3

41. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Important! It’s perfectly valid to use Tiled Shading on a TBR platform –As we’ll see shortly.

42. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 M ANY -L IGHT M ETHODS R EVISITED Part 2

43. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Many-Light Methods Revisited Ola listed a number of methods in his introduction earlier in the course –Quickly revisit these Plus two new methods for mobile arch.

44. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Forward Traditional Deferred Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack

45. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled & Clustered I’ll lump tiled and clustered shading together for now

46. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled & Clustered I’ll lump tiled and clustered shading together for now Basic idea is similar enough –Pick the one that suits your use case better –E.g., tiled for 2D-ish settings, clustered for 3D.. Just Cause 2 (Avalanche Studios) Star Craft 2 (Blizzard) vs

47. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Forward Traditional Deferred Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack

48. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Traditional Deferred Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack

49. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 “Plain” Forward Assign lights per geometry batch Loop over lights in fragment shader

50. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 “Plain” Forward Assign lights per geometry batch Loop over lights in fragment shader Default method with no extra frills Possible “anywhere”

51. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 “Plain Forward” (II) Scales badly with number of lights –See Ola’s introduction

52. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Traditional Deferred Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack

53. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Traditional Deferred Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓

54. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓

55. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Traditional Deferred Render scene to G-Buffers Color Normal Position(Z)

56. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Traditional Deferred Render scene to G-Buffers Render lights using proxy geometry Color Position(Z) Normal

57. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Traditional Deferred Render scene to G-Buffers Render lights using proxy geometry In shader: Read G-Buffers, compute lighting, blend results. Color Position(Z) Normal

58. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Traditional Deferred (II) Many reads from G-Buffer + many writes to Framebuffer

59. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Traditional Deferred (II) Many reads from G-Buffer + many writes to Framebuffer Store G-Buffer to RAM (and potentially restore depth to on-chip)

60. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓

61. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Clustered Deferred Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No

62. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No

63. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred Designed to avoid some of the issues of traditional deferred

64. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (II) Render scene to G-Buffers Compute light assignment –Use e.g. depth from G-Buffers 0063064 … 0 1 1 3 4 3 7 1 66 1 67 2 69 2 … … L0L0 L1L1 L2L2 L3L3 L4L4 L5L5 … L6L6 4 … …

65. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (II) Render scene to G-Buffers Compute light assignment –Use e.g. depth from G-Buffers 0063064 … 0 1 1 3 4 3 7 1 66 1 67 2 69 2 … … L0L0 L1L1 L2L2 L3L3 L4L4 L5L5 … L6L6 4 … …

66. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (II) Render scene to G-Buffers Compute light assignment –Use e.g. depth from G-Buffers Full screen pass: compute lighting

67. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (III) Lighting pass: –Read G-Buffer once –Find pixel’s tile/cluster –Loop over assigned lights => compute lighting –Write result to FB once

68. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (IV) Still requires off-chip G-Buffers –But avoids reading multiple times from them –Also avoids loading back depth to on-chip

69. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (V) Original clustered method relies on compute shaders to –Identify active clusters –Perform light assignment

70. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (V) Original clustered method relies on compute shaders to –Identify active clusters –Perform light assignment Avoid with Emil’s Practical Clustered

71. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Deferred (VI) For tiled shading –Perform light assignment on CPU –Compute per-tile bounds in fragment shader –No compute shaders, but potential read-back

72. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No

73. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clustered Forward Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No

74. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No

75. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward Use depth-pass to determine per-tile Z-bounds or clusters –For light assignment 0063064 … 0 1 1 3 4 3 7 1 66 1 67 2 69 2 … … L0L0 L1L1 L2L2 L3L3 L4L4 L5L5 … L6L6 4 … …

76. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward (II) Render scene normally For each sample –Determine sample’s tile/cluster –Read associated light list

77. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward (II) Render scene normally For each sample –Determine sample’s tile/cluster –Read associated light list Tiled Forward a.k.a. Forward+

78. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward (III) No G-Buffers / MRT!

79. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward (III) No G-Buffers / MRT! Depth buffer still transferred to RAM –And potentially back

80. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Tiled/Clustered Forward (IV) Tiled Forward possible in ES 2.0 Handy extensions: –Render to depth texture –Dynamic looping in fragment shader

81. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No

82. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Practical Forward Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓*

83. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓*

84. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Practical Clustered Forward Clustered variant presented by Emil

85. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Practical Clustered Forward Clustered variant presented by Emil Applicable to both deferred and forward –Or a mix of them Mostly interested in forward-only…

86. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Practical Clustered Forward (II) Perform light-assignment up-front –To dense cluster structure –Potentially on CPU Then render scene in “normally”

87. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓*

88. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Deferred Tile Store Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓

89. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓

90. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage Very similar to traditional deferred.

91. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage Very similar to traditional deferred. Stores G-Buffer temporarily on-chip –G-Buffer only available while a tile is being processed!

92. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage (II) Relies on OpenGL Extensions –EXT_shader_pixel_local_storage –ARM_shader_framebuffer_fetch –ARM_shader_framebuffer_fetch_depth_stencil

93. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage (III) Key: EXT_shader_pixel_local_storage

94. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage (III) Key: EXT_shader_pixel_local_storage –Gives small amount of on-chip per-pixel storage –Preserved across fragment shader instances –But not backed by external RAM

95. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Deferred with Tile Storage (IV) A bit finicky –Writes to ordinary color output destroys storage contents –Incompatible with MSAA See Sam Martin’s original presentation from SIGGRAPH 2013 + Read Spec.

96. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Vulkan Note Probably achievable in Vulkan? –E.g., with the transient FB-attachments –Becomes Traditional Deferred on IMR.

97. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓

98. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Forward Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No

99. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Def. Tile Store Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No

100. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack Uses on-chip storage for lights instead

101. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack Uses on-chip storage for lights instead Do depth-only pre-pass Splat lights to build per-pixel light lists –Store using EXT_shader_pixel_local_storage

102. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack (I) Forward render geometry

103. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack (II) Limited light list size

104. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack (II) Limited light list size Incompatible with MSAA

105. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack (II) Limited light list size Incompatible with MSAA Blending possible –“By hand” –Requires some bytes per per-pixel storage

106. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Forward with Light Stack (III) Limitations may be relaxed in the future? –MSAA –More storage

107. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Def. Tile Store Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No

108. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd Trad. Def. Clust. Def. Clust. Fwd. Practical Fwd. Def. Tile Store Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No ✓ Color2PreZ ✓

109. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd. Trad. Def. Clust. Def. Clust. Fwd Practical Fwd Def. Tile Store Fwd. Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No ✓ Color2PreZ ✓

110. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Methods Plain Fwd. Trad. Def. Clust. Def. Clust. Fwd Practical Fwd Def. Tile Store Fwd. Light Stack Many Light Off-Chip Buffers? geometry passes over- shading MSAABlend.GL|ES Color1Yes ✓✓ ✓ G-Buffers1No ✓ G-Buffers1No ✓ Depth2PreZ ✓*✓* ✓*✓* ✓ Color1Yes ✓✓ ✓ Color1No ✓ Color2PreZ ✓

111. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Practical Clustered Forward Supported on many devices –TBR: stays on-chip –IMR: nothing special MSAA & Blending

112. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Practical Clustered Forward Supported on many devices –TBR: stays on-chip –IMR: nothing special MSAA & Blending I’m maybe a bit biased here… ;-)

113. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 C USTOM P RACTICAL C LUSTERED F ORWARD. Part 3

114. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Clustering, Redux Shown a few different ways of clustering –“original” sparse exponential way –Emil’s up-front dense clustering –2D-Tiling

115. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Clustering, Redux Shown a few different ways of clustering –“original” sparse exponential way –Emil’s up-front dense clustering –2D-Tiling One more variation

116. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem Up-front clustering assigns into a dense structure

117. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem Up-front clustering assigns into a dense structure Possibly a ton of clusters

118. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (II) Reduce #clusters… How?

119. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (II) Reduce #clusters… How? –Lower resolution?

120. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (II) Reduce #clusters… How? –Lower resolution? Observation: Especially problematic close to the camera

121. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (II) Reduce #clusters… How? –Lower resolution? Observation: Especially problematic close to the camera

122. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (III) Lot’s of unnecessary work during light assignment. –Almost guaranteed to occur…

123. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 The Problem (IV) Discussed previously –E.g., move back first Z-subdivision –Still get a lot of slices in XY

124. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Cascaded Clustering Different solution: Cascaded Clustering

125. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Cascaded Clustering Different solution: Cascaded Clustering Subdivide frustum into cascades and select individual resolution for each

126. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Cascaded Clustering Different solution: Cascaded Clustering Subdivide frustum into cascades and select individual resolution for each

127. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Cascaded Clustering (II) Select resolutions to give –Approx. cubical clusters with NxN footprint –But enforcing a minimum size of each cluster

128. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Cascaded Clustering (II) Select resolutions to give –Approx. cubical clusters with NxN footprint –But enforcing a minimum size of each cluster Currently use 12 cascades –Tweakable…

129. Quick Results

130. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Quick Results 192 lights Clustering: 0.35ms –Galaxy Alpha –3500 non-empty clusters –with 3.5 lights avg. (worst = 20 lights)

131. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Quick Results 30ms per frame average –Worst: ~35ms –Small lights + limited overlap With PreZ

132. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 Conclusion

133. Real-Time Many-Light Management and Shadows with Clustered Shading – 2015 References Tiled Deferred Blending, Ladlani, 2014 The Revolution in Mobile Game Graphics, Martin and Bjørge 2014 Challenges with High Quality Mobile Graphics, Martin et al. 2013 An Evolution of Mobile Graphics, Shebanow 2013 Performance Tuning for Tile-Based Architectures, Merry 2012 Courses from earlier this week: –Moving Mobile Graphics –An Overview of Next-Generation Graphics APIs Forward+: Bringing deferred lighting to the next level, Harada et al. 2012 Light Indexed Deferred Lighting, Treblico 2007 Our work on Tiled / Clustered Shading: –Clustered Deferred and Forward Shading, Olsson et al. 2012 –Tiled and Clustered Forward Shading, Olsson et al. 2012 –Tiled Forward Shading, Billeter et al. 2013 (This slide show brought to you at approx. 0.057 SPS [slides per second])