1. Advances in Real-Time Rendering in Games

2. Dynamic lighting in GOW3

3. Advances in Real-Time Rendering in Games

4. MotivationMotivation   RSX is a bottleneck   How do we use SPUs to improve lighting quality and keep it cheap in terms of RSX processing?   On PS3 the artists asked for more lights per pixel

5. Advances in Real-Time Rendering in Games Types of lights in GOW3   Ambient   Omni (point)   Directional

6. Advances in Real-Time Rendering in Games Types of lights in GOW3   Ambient   Omni (point)   Directional Combined into single RGB interpolator

7. Advances in Real-Time Rendering in Games Types of lights in GOW3   Ambient   Omni (point)   Directional Represented as hybrid vertex lights

8. Advances in Real-Time Rendering in Games Hybrid vertex lights

9. Advances in Real-Time Rendering in Games Hybrid vertex lights   For 1 light identical to pixel lights

10. Advances in Real-Time Rendering in Games Hybrid vertex lights   For 1 light identical to pixel lights   For multiple lights   Calculate distance falloff per vertex.   Combine into a single Aggregate Light per-vertex   Interpolate Aggregate Light Position per pixel   Perform N dot L, N dot H etc. in the fragment program as if we had a single pixel light

11. Advances in Real-Time Rendering in Games Interpolating Direction (wrong) AB Light

12. Advances in Real-Time Rendering in Games AB Light L=(½L A + ½ L B ) Interpolating Direction (wrong)

13. Advances in Real-Time Rendering in Games Interpolating Direction (wrong) AL=normalize(½L A + ½ L B )B Light

14. Advances in Real-Time Rendering in Games Interpolating Direction (wrong) AL=normalize(½L A + ½ L B )B Light

15. Advances in Real-Time Rendering in Games Interpolating Direction (wrong) AL=normalize(½L A + ½ L B )B Light WRONG

16. Advances in Real-Time Rendering in Games AB Light Interpolating Position

17. Advances in Real-Time Rendering in Games A L = Light_Pos−(½A+½B) B Light Interpolating Position

18. Advances in Real-Time Rendering in Games A Normalize(Light_Pos−(½A+½B)) B Light Interpolating Position

19. Advances in Real-Time Rendering in Games A Normalize(Light_Pos−(½A+½B)) B Light BETTER: Interpolating Position

20. Advances in Real-Time Rendering in Games Falloff function 1/3   Smooth   Cheap   Hopefully, first derivative approaches 0 as the function itself approaches 0.   Why?

21. Advances in Real-Time Rendering in Games   Identical directional lights shining straight down.   Left: Linear falloff. Right: our falloff   Falloff functions were set to reach zero at the same distance Falloff function 2/3

22. Advances in Real-Time Rendering in Games Falloff function(3/3)

23. Advances in Real-Time Rendering in Games   We are storing light position per vertex   We have chosen our falloff function

24. Advances in Real-Time Rendering in Games   How do we come up with that light position we are going to store per vertex?

25. Advances in Real-Time Rendering in Games   How do we come up with that light position we are going to store per vertex?   Is it enough to just average light positions with weights based on falloff?

26. Advances in Real-Time Rendering in Games This is still not so good Light 0 Vertex V Light 1

27. Advances in Real-Time Rendering in Games This is still not so good Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal. Light 0 Vertex V Light 1

28. Advances in Real-Time Rendering in Games This is still not so good  Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal.  Light weights will be also identical for simplicity set them to 1 Light 0 Vertex V Light 1

29. Advances in Real-Time Rendering in Games This is still not so good Light 0 Vertex V Aggregate light Light 1  Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal.  Light weights will be also identical for simplicity set them to 1

30. Advances in Real-Time Rendering in Games This is still not so good Light 0 Vertex V Light 1 Aggregatelight  Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal.  Light weights will be also identical for simplicity set them to 1

31. Advances in Real-Time Rendering in Games This is still not so good Light 0 Vertex V Aggregate light Light 1  Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal.  Light weights will be also identical for simplicity set them to 1  Resulting lighting will over-emphasize Light 1

32. Advances in Real-Time Rendering in Games This is still not so good Ideal aggregate light Light 0 Vertex V Aggregate light Light 1  Assume: with falloff, intensities generated at Vertex V by Light 0 and Light 1 are equal.  Light weights will be also identical for simplicity set them to 1  Resulting lighting will over-emphasize Light 1

33. Advances in Real-Time Rendering in Games   Interpolating light directions per-pixel does not work   Averaging light positions per-vertex does not work   What do we do?

34. Advances in Real-Time Rendering in Games   We will average light directions per- vertex   And re-create an aggregate light position before sending it to be interpolated per- pixel.

35. Advances in Real-Time Rendering in Games “Better” approach Light 0 Vertex V Light 1

36. Advances in Real-Time Rendering in Games “Better” approach   Subtract the world vertex position from each world light position to create relative vectors. Light 0 Vertex V Light 1

37. Advances in Real-Time Rendering in Games “Better” approach Light 0 Vertex V L0L0 L1L1 Light 1   Subtract the world vertex position from each world light position to create relative vectors.   Calculate lengths

38. Advances in Real-Time Rendering in Games “Better” approach Light 0 Vertex V L0L0 L1L1 Light 1 W 0 = 1/L 0 W 1 = 1/L 1   Subtract the world vertex position from each world light position to create relative vectors.   Calculate lengths and weights (remember light intensities are 1 for both lights)

39. Advances in Real-Time Rendering in Games “Better” approach   Subtract the world vertex position from each world light position to create relative vectors.   Calculate lengths and weights (remember light intensities are 1 for both lights)   Now multiply relative vectors by weights to go to direction domain Light 0 Vertex V Light 1 W 0 = 1/L 0 W 1 = 1/L 1

40. Advances in Real-Time Rendering in Games “Better” approach   Add up light directions Light 0 Vertex V Light 1 W 0 = 1/L 0 W 1 = 1/L 1

41. Advances in Real-Time Rendering in Games “Better” approach   Add up light directions Light 0 Vertex V Light 1 W 0 = 1/L 0 W 1 = 1/L 1

42. Advances in Real-Time Rendering in Games “Better” approach   Add up light directions   And accumulate weights Light 0 Vertex V Light 1 W 0 = 1/L 0 W 1 = 1/L 1 + W total

43. Advances in Real-Time Rendering in Games “Better” approach   Add up light directions   And accumulate weights   Multiply aggregate direction by accumulated weight to go back to positions domain Light 0 Vertex V Light 1 W 0 = 1/L 0 W 1 = 1/L 1 W total

44. Advances in Real-Time Rendering in Games “Better” approach   We ended up with relative light vector for aggregate light Light 0 Vertex V Light 1

45. Advances in Real-Time Rendering in Games “Better” approach Light 0 Vertex V Light 1   We ended up with relative light vector for aggregate light   Now add vertex world position to it to get world position of the aggregate light

46. Advances in Real-Time Rendering in Games “Better” approach math

47. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position

48. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i

49. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V

50. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context

51. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights

52. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

53. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

54. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

55. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

56. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

57. Advances in Real-Time Rendering in Games “Better” approach L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

58. Advances in Real-Time Rendering in Games “Better” approach L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

59. Advances in Real-Time Rendering in Games “Better” approach math L agg is aggregate light position L i is light position for Light i W i is light weight for Light i based on intensity of Light i at V n is number of lights in the current light context V is position of the vertex for which we combine lights Light 0 Vertex V

60. Advances in Real-Time Rendering in Games   We came up with a way to calculate aggregate light position   We have chosen our falloff function

61. Advances in Real-Time Rendering in Games Back facing lights   Since lights are combined without regard for shadowing, it is important to eliminate the contributions of lights which are back facing to the vertex.

62. Advances in Real-Time Rendering in Games Back facing lights

63. Advances in Real-Time Rendering in Games Back facing lights W i is the weight of Light i

64. Advances in Real-Time Rendering in Games Back facing lights W i is the weight of Light i N v is vertex normal at the vertex V

65. Advances in Real-Time Rendering in Games Back facing lights W i is the weight of Light i N v is vertex normal at the vertex V L i is light direction of Light i at vertex V

66. Advances in Real-Time Rendering in Games   We came up with a way to calculate aggregate light position   We have chosen our falloff function   We know how to eliminate back facing lights   Is this enough?

67. Advances in Real-Time Rendering in Games This is still broken!

68. Advances in Real-Time Rendering in Games This is still broken! B A

69. Advances in Real-Time Rendering in Games This is still broken! AB

70. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNa NbNb

71. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNa NbNb L blue

72. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNa NbNb L blue P

73. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNaNaNa NbNbNbNb L blue P

74. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNa NbNb L blue P LpLp NpNp

75. Advances in Real-Time Rendering in Games This is still broken! A L red B NaNaNaNa NbNbNbNb L blue P

76. Advances in Real-Time Rendering in Games

77. Fix it   In the fragment program we get light position interpolated from vertices   We then calculate the light vector and normalize it before computing N dot L   Stop normalizing if interpolated light vector if it is shorter than a threshold   Takes care of the problem nicely

78. Advances in Real-Time Rendering in Games

79.   We came up with a way to calculate aggregate light position   We know how to eliminate back facing lights   We have chosen our falloff function   Now we need a way to calculate aggregate light color.

80. Advances in Real-Time Rendering in Games Aggregate Light Color   Data to calculate a “physically correct” value is lost   We need to settle for something that will give reasonable result when interpolated in the fragment program

81. Advances in Real-Time Rendering in Games Our method

82. Advances in Real-Time Rendering in Games Our method  Calculate aggregate light position

83. Advances in Real-Time Rendering in Games Our method  Calculate aggregate light position  Calculate normalized light directions

84. Advances in Real-Time Rendering in Games Our method  Calculate aggregate light position  Calculate normalized light directions  Calculate dot products

85. Advances in Real-Time Rendering in Games Our method Calculate aggregate light position Calculate normalized light directions Calculate dot products x+ x =

86. Advances in Real-Time Rendering in Games Aggregate color formula

87. Advances in Real-Time Rendering in Games Aggregate color formula C agg is aggregate light color at the Vertex V

88. Advances in Real-Time Rendering in Games Aggregate color formula C agg is aggregate light color at the Vertex V C i is color of Light i

89. Advances in Real-Time Rendering in Games Aggregate color formula C agg is aggregate light color at the Vertex V C i is color of Light i L i is direction to Light i at Vertex V

90. Advances in Real-Time Rendering in Games Aggregate color formula C agg is aggregate light color at the Vertex V C i is color of Light i L i is direction to Light i at Vertex V L agg is direction to aggregate light

91. Advances in Real-Time Rendering in Games Aggregate color formula C agg is aggregate light color at the Vertex V C i is color of Light i L i is direction to Light i at Vertex V L agg is direction to aggregate light x+x=

92. Advances in Real-Time Rendering in Games How well does this approximate actual lighting by several lights?

93. Advances in Real-Time Rendering in Games Why this works This is what we are trying to approximate:

94. Advances in Real-Time Rendering in Games Why this works This is what we are trying to approximate:

95. Advances in Real-Time Rendering in Games Why this works This is what we are trying to approximate:

96. Advances in Real-Time Rendering in Games Why this works This is what we are trying to approximate: Assume lights have a single scalar luminance value and no color.

97. Advances in Real-Time Rendering in Games Why this works Aggregate light approximation (with luminances) Exact luminance calculation

98. Advances in Real-Time Rendering in Games Why this works Aggregate light approximation (with luminances) Exact luminance calculation Our approximation

99. Why this works Aggregate light approximation (with luminances)

100. Why this works Aggregate light approximation (with luminances)

101. Why this works Aggregate light approximation (with luminances)

102. Why this works Aggregate light approximation (with luminances)

103. Advances in Real-Time Rendering in Games Why this works Aggregate light approximation (with luminances)

104. Remember this? Light 0 Vertex V

105. Remember this? Light 0 Vertex V

106. Remember this? Light 0 Vertex V

107. Remember this? Light 0 Vertex V L agg

108. Remember this? Light 0 Vertex V L agg

109. Remember this? Light 0 Vertex V L agg

110. Remember this? Light 0 Vertex V L agg

111. Remember this? Light 0 Vertex V L agg

112. Remember this? Light 0 Vertex V L agg

113. Advances in Real-Time Rendering in Games Why this works

114. Advances in Real-Time Rendering in Games Why this works

115. Advances in Real-Time Rendering in Games A note on vector arithmetic

116. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors V0V0V0V0

117. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors V0V0V0V0 V1V1V1V1

118. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors V0V0V0V0 V1V1V1V1 V2V2V2V2

119. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors V0V0V0V0 V1V1V1V1 V2V2V2V2

120. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors Length of the result Length of the result V0V0V0V0 V1V1V1V1 V2V2V2V2

121. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors Length of the result is equal to the sum of projections Length of the result is equal to the sum of projections V0V0V0V0 V1V1V1V1 V2V2V2V2

122. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors Length of the result is equal to the sum of projections Length of the result is equal to the sum of projections V0V0V0V0 V1V1V1V1 V2V2V2V2

123. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors Length of the result is equal to the sum of projections Length of the result is equal to the sum of projections V0V0V0V0 V1V1V1V1 V2V2V2V2

124. Advances in Real-Time Rendering in Games A note on vector arithmetic Adding multiple vectors Adding multiple vectors Length of the result is equal to the sum of projections Length of the result is equal to the sum of projections V0V0V0V0 V1V1V1V1 V2V2V2V2

125. Last step

131. L Agg

132. Advances in Real-Time Rendering in Games Last Step

133. Advances in Real-Time Rendering in Games Last Step For luminance our approximation is exact

134. Advances in Real-Time Rendering in Games Extending to color Which is what we use Extending to RGB:

135. Advances in Real-Time Rendering in Games ImplementationImplementation   We run our per-vertex lighting calculations as custom code in the EDGE job   Heavily optimized   Still keep PPU version running for reference and debugging

136. Advances in Real-Time Rendering in Games ConclusionsConclusions   Reasonably cheap and good looking dynamic lights   Our artists like them. They can apply a lot of them per draw call.  Additional notes  We are using very similar light aggregation in our baked vertex lighting  This allows to treat baked lighting as yet another vertex light at render time.

137. Advances in Real-Time Rendering in Games Questions ?

138. Advances in Real-Time Rendering in GamesHistoryHistory Vector X is essentially the same as the light vector from The Light Field (Gershun, 1936) and the vector irradiance from The Irradiance Jacobian for Partially Occluded Polyhedral Sources (Arvo, 1994).