1. GLSL Applications: 2 of 2 Patrick Cozzi University of Pennsylvania CIS 565 - Spring 2011

2. Administrivia Assignment 2 due today  11:59pm on Blackboard Assignment 3  handed out today  Due Wednesday, 02/09 at 11:59pm

3. Agenda Finish last Monday’s slides  VBO Layout  OpenGL Multithreading Today’s slides  OpenGL Textures and Multitexturing  OpenGL Framebuffers and Deferred Shading  Ambient Occlusion

4. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id);

5. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Pixels for an image in system memory.

6. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Standard business.

7. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); I hate global state. You should too. What is the alternative design?

8. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Sampler state. More info to follow.

9. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Transfer from system memory to driver-controlled (likely, GPU) memory. Does it need to block?

10. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Pixel data format and datatype

11. Textures unsigned char *pixels = //... GLuint id; glGenTextures(1, &id); glBindTexture(GL_TEXTURE_2D, id); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixels); //... glDeleteTextures(1, &id); Internal (GPU) texture format

12. Texture Wrap Parameters Images from: http://http.download.nvidia.com/developer/NVTextureSuite/Atlas_Tools/Texture_Atlas_Whitepaper.pdf GL_MIRRORED_REPEAT GL_REPEAT GL_CLAMP Set with:  glTexParameteri ()

13. Multitexturing Using multiple textures in the same rendering pass Each is bound to a different texture unit and accessed with a different sampler uniform in GLSL

14. Multitexturing: Light Map Recall our Light Map example: x = Precomputed lightSurface color “lit” surface

15. Multitexturing: Light Map uniform sampler2D lightMap; uniform sampler2D surfaceMap; in vec2 fs_TxCoord; in vec3 out_Color; void main(void) { float intensity = texture(lightMap, fs_TxCoord).r; vec3 color = texture(surfaceMap, fs_TxCoord).rgb; out_Color = intensity * color; } Each texture is accessed with a different sampler

16. Multitexturing: Light Map uniform sampler2D lightMap; uniform sampler2D surfaceMap; in vec2 fs_TxCoord; in vec3 out_Color; void main(void) { float intensity = texture(lightMap, fs_TxCoord).r; vec3 color = texture(surfaceMap, fs_TxCoord).rgb; out_Color = intensity * color; } Pass the sampler to texture() to read from a particular texture

17. Multitexturing: Terrain How was this rendered? Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

18. Multitexturing: Terrain First hint: two textures Images courtesy of A K Peters, Ltd. www.virtualglobebook.com GrassStone

19. Multitexturing: Terrain Second hint: terrain slope Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

20. Multitexturing: Terrain Second hint: terrain slope Image courtesy of A K Peters, Ltd. www.virtualglobebook.com 0 is flat 1 is steep

21. Multitexturing: Terrain Third and final hint: a blend ramp Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

22. Multitexturing: Terrain uniform sampler2D grass; uniform sampler2D stone; uniform sampler2D blendRamp; in vec3 out_Color; void main(void) { //... out_Color = intensity * mix( texture(grass, repeatTextureCoordinate).rgb, texture(stone, repeatTextureCoordinate).rgb, texture(u_blendRamp, vec2(0.5, slope)).r); }

23. Multitexturing: Terrain uniform sampler2D grass; uniform sampler2D stone; uniform sampler2D blendRamp; in vec3 out_Color; void main(void) { //... out_Color = intensity * mix( texture(grass, repeatTextureCoordinate).rgb, texture(stone, repeatTextureCoordinate).rgb, texture(u_blendRamp, vec2(0.5, slope)).r); } Three samplers blendRamp could be 1D; it is just 1xn

24. Multitexturing: Terrain uniform sampler2D grass; uniform sampler2D stone; uniform sampler2D blendRamp; in vec3 out_Color; void main(void) { //... out_Color = intensity * mix( texture(grass, repeatTextureCoordinate).rgb, texture(stone, repeatTextureCoordinate).rgb, texture(u_blendRamp, vec2(0.5, slope)).r); } Use terrain slope to look up a blend value in the range [0, 1]

25. Multitexturing: Terrain uniform sampler2D grass; uniform sampler2D stone; uniform sampler2D blendRamp; in vec3 out_Color; void main(void) { //... out_Color = intensity * mix( texture(grass, repeatTextureCoordinate).rgb, texture(stone, repeatTextureCoordinate).rgb, texture(u_blendRamp, vec2(0.5, slope)).r); } Linearly blend between grass and stone

26. Multitexturing: Globe How will you render this? Imagery from http://planetpixelemporium.com/

27. Multitexturing: Globe How will you render this? Imagery from http://planetpixelemporium.com/ Day texture Night texture

28. Multitexturing: Globe Imagery from http://planetpixelemporium.com/ Day TextureDay Night

29. Multitexturing: Globe Videos  Night and Day Night and Day  Clouds Clouds  Specularity Specularity

30. Framebuffer Objects (FBOs)  Allow fragment shader to write to one or more off-screen buffers  Can then use the off-screen buffer as a texture in a later rendering pass  Allows render to texture  Don’t worry about the OpenGL API; we’ve already coded it for you

31. Framebuffer Objects (FBOs) FBOs are lightweight containers of textures FBO Depth Texture Color Texture 0 Color Texture 1 …

32. Framebuffer Objects (FBOs) FBO use case: post processing effects  Render scene to FBO with depth and color attachment  Render a viewport-aligned quad with texture that was the color attachment and apply effect  How would you design a post processing framework?

33. Deferred Shading FBO use case: deferred shading  Render scene in two passes 1 st pass: Visibility tests 2 nd pass: Shading

34. Deferred Shading 1 st Pass: Render geometry into G-Buffers Fragment ColorsNormals DepthEdge Weight Images from http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html

35. Deferred Shading 2 nd pass: shading == post processing effects Render viewport-aligned quads that read from G-Buffers Objects are no longer needed

36. Deferred Shading Light accumulation result Image from http://http.developer.nvidia.com/GPUGems3/gpugems3_ch19.html

37. Deferred Shading What are the benefits:  Shading and depth complexity?  Memory requirements?  Memory bandwidth?  Material and light decoupling?

38. Ambient Occlusion Ambient Occlusion (AO)  "shadowing of ambient light“  "darkening of the ambient shading contribution“ Image from Bavoil and Sainz. http://developer.download.nvidia.com/SDK/10.5/direct3d/Source/ScreenSpaceAO/doc/ScreenSpaceAO.pdf

39. Ambient Occlusion  "the crevices of the model are realistically darkened, and the exposed parts of the model realistically receive more light and are thus brighter“  "the soft shadow generated by a sphere light of uniform intensity surrounding the scene"

40. Ambient Occlusion Image from Iñigo Quílez. http://iquilezles.org/www/articles/ssao/ssao.htm

41. Ambient Occlusion Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/ Evenly lit from all directions Ambient Occlusion Global Illumination

42. Ambient Occlusion Image from Bavoil and Sainz. http://developer.download.nvidia.com/SDK/10.5/direct3d/Source/ScreenSpaceAO/doc/ScreenSpaceAO.pdf "the integral of the occlusion contributed from inside a hemisphere of a given radius R, centered at the current surface point P and oriented towards the normal n at P"

43. Object Space Ambient Occlusion AO does not depend on light direction Precompute AO for static objects using ray casting  How many rays?  How far do they go?  Local objects? Or all objects?

44. Object Space Ambient Occlusion Image courtesy of A K Peters, Ltd. http://www.realtimerendering.com/ Cosine weight rays or use importance sampling: cosine distribute number of rays

45. Object Space Ambient Occlusion Depends on scene complexity Stored in textures or vertices How can we  Support dynamic scenes  Be independent of scene complexity

46. Screen Space Ambient Occlusion Apply AO as a post processing effect using a combination of depth, normal, and position buffers Not physically correct but plausible Visual quality depends on  Screen resolution  Number of buffers  Number of samples

47. Depth Buffer

48. Normal Buffer

49. View Space Eye Position Buffer

50. Screen Space Ambient Occlusion Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/

51. Screen Space Ambient Occlusion Image from Martin Mittring. http://developer.amd.com/documentation/presentations/legacy/Chapter8-Mittring-Finding_NextGen_CryEngine2.pdf

52. Screen Space Ambient Occlusion Image from Martin Mittring. http://developer.amd.com/documentation/presentations/legacy/Chapter8-Mittring-Finding_NextGen_CryEngine2.pdf

53. Screen Space Ambient Occlusion Image from Mike Pan. http://mikepan.com Blur depth buffer Subtract it from original depth buffer Scale and clamp image, then subtract from original Superficially resembles AO but fast