1. Wilf LaLonde @2012 Comp 4501 95.4501 OpenGL Odds and Ends

2. Wilf LaLonde @2012 Comp 4501 Odds and Ends A few odds and ends to help with assignment #3 particularly issues that might trip you. A bit about blending. A bit about OpenGL stacks. A bit about OpenGL shaders. Generally, things you SHOULD be able to figure out by yourself but might not have.

3. Wilf LaLonde @2012 Comp 4501 95.4002 Transformation Extensions

4. Wilf LaLonde @2012 Comp 4501 There is a lookAtForObject which can build an entire transformation for you… If you need the inverse (for something playing the role of a camera), it might be nice to have a version call lookAtForCamera. The next slide provides it… IF YOU NEED IT. A Useful Addition To Transformation

5. Wilf LaLonde @2012 Comp 4501 //In.h file static Transformation lookAtForCamera (const Point &from, const Vector &direction, const Vector &up, const Vector &alternateUp) ; //In.cpp file Transformation Transformation::lookAtForCamera (const Point &from, const Vector &direction, const Vector &up, const Vector &alternateUp) { //This version returns the same thing as gluLookAt but computes it without using the stack... //Note: up is an approximate yAxis. If it is erroneously directed into the z-direction, use alternateUp instead. Vector zAxis = -direction.normalized (); Vector xAxis = (up.cross (zAxis)).normalized (); if (xAxis.isZero ()) xAxis = (alternateUp.cross (zAxis)).normalized (); Vector yAxis = zAxis.cross (xAxis); Transformation result; //Compute the inverse of rotateToAxes (xAxis, yAxis, zAxis, from) which interpreted as RT means computing T-1R-1. result.set ( xAxis.x, yAxis.x, zAxis.x, 0.0, xAxis.y, yAxis.y, zAxis.y, 0.0, xAxis.z, yAxis.z, zAxis.z, 0.0, -from.x*xAxis.x - from.y*xAxis.y - from.z*xAxis.z, -from.x*yAxis.x - from.y*yAxis.y - from.z*yAxis.z, -from.x*zAxis.x - from.y*zAxis.y - from.z*zAxis.z, 1.0); return result; } A Useful Addition To Transformation

6. Wilf LaLonde @2012 Comp 4501 95.4002 Blending

7. Wilf LaLonde @2012 Comp 4501 Many Effect Require Blending Techniques Blending permits the colors of a texture to be merged with the colors already drawn on the back buffer. To use it, must enable it; disable when done. glEnable (GL_BLEND); glDisable (GL_BLEND);

8. Wilf LaLonde @2012 Comp 4501 Must Specify Which Blending Factors To Use result = SourceFactor*S + DestinationFactor*D S is the input texture (the source) D is the screen pixel (the destination or what’s already there) 15 factors available: can get by with 3 or 4. glBlendFunc (sourceFactor, destinationFactor); coming up1. Example Factors, 2. Example Blend functions

9. Wilf LaLonde @2012 Comp 4501 What’s The Complete Set of Factors? GL_ZERO0 GL_ONE1 GL_SRC_COLORS GF_DST_COLORD GL_SRC_ALPHA  from S GL_DST_ALPHA  from D GL_ONE_MINUS_SRC_ALPHA1-  from S GL_ONE_MINUS_DST_ALPHA1-  from D

10. Wilf LaLonde @2012 Comp 4501 Complete Set... GL_ONE_MINUS_SRC_COLOR1-S GL_ONE_MINUS_DST_COLOR1-D There are a few additional rules (available through extensions).. Examples coming up

11. Wilf LaLonde @2012 Comp 4501 What the Blending Factors Apply To result = SourceFactor*S + DestinationFactor*D S is the input texture (the source) D is the screen pixel (the destination or what’s already there) glBlendFunc (sourceFactor, destinationFactor); Note that result has no alpha in it once it’s combined. Combination applies to R,G,B components independently (each ranges from 0 to 1)…

12. Wilf LaLonde @2012 Comp 4501 Useful Combination 1: Brightening Take a little from each AND ignore transparency. glBlendFunc (GL_ONE, GL_ONE); Consequences: Result = 1*S + 1*D(clamped to 1 if > 1) glBlendFunc (sourceFactor, destinationFactor) Creates a brightening (whitening) effect… 1 * 0.5 (gray) + 1 * 0.5 (gray) => 1 (white) Creates a brightening (whitening) effect… 1 * 0.5 (gray) + 1 * 0.5 (gray) => 1 (white)

13. Wilf LaLonde @2012 Comp 4501 Useful Combination 2: Raw Transparency Use transparency (called alpha blending) glBlendFunc ( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA ); Result =  *S + (1-  )*D(clamped to 1 if > 1) where  is the alpha in the textured color glBlendFunc (sourceFactor, destinationFactor) If  = 0.7 (mostly opaque), get.7S+.3D

14. Wilf LaLonde @2012 Comp 4501 What Combination Is Like Having Blending Off? Same as using glBlendFunc (GL_ONE, GL_ZER0); Result = 1*S + 0*D = S So pixel behind is overwritten glBlendFunc (sourceFactor, destinationFactor) Better to say, glDisable (GL_BLEND) instead.

15. Wilf LaLonde @2012 Comp 4501 One that multiplies rather than adds so that white (1) * what is there => stays the same gray (.5) * what is there => goes darker black (0) * what is there => black glEnable (GL_BLEND); glBlendFunc (GL_ZERO, GL_SRC_COLOR); Consequences: Result = 0*S + S*D(clamped to 1 if > 1) Useful Combination 3: Darkening This simulates MULTIPLY 0.8*0.8=0.64 (darkens)

16. Wilf LaLonde @2012 Comp 4501 95.4002 The OpenGL Stack

17. Wilf LaLonde @2012 Comp 4501 Summary: Thinking With the Left To Right Convention How to use the stack? NEXT v * ABC MATH works from left to right CBA growth direction From most local to most global The OpenGL Stack works from right to left So push C; mult B; mult A; pop

18. Wilf LaLonde @2012 Comp 4501 How Do I Put Something On The Stack Want to translate by T = [t x, t y, t z ], and scale by S = [s x, s y, s z ]. glMatrixMode (GL_MODELVIEW_MATRIX); glPushMatrix (); glTranslated (t x, t y, t z ); glScaled (s x, s y, s z ); glPopMatrix (); This puts S*T*whateverIsAlreadyThere on the stack (since it works from right to left) If W transforms a vehicle to a place in the world with camera C, the stack should contain W * C -1

19. Wilf LaLonde @2012 Comp 4501 95.4002 Shaders

20. Wilf LaLonde @2012 Comp 4501 If a shader uses a texture variable called “birdTexture” via uniform sampler2D birdTexture How does the game engine know what texture to use; e.g., texture called “turkey” Questions YOU Should Have Had In Your Mind Generally, stuff explained in Shader::example ()

21. Wilf LaLonde @2012 Comp 4501 Explain how the ambient occlusion demo does all this. Relate how this is different in the engine… How The Explanation Will Go

22. Wilf LaLonde @2012 Comp 4501 A host of routines are used with names that are synonyms… glCreateProgram versus glCreateProgramObjectARB glUseProgram versus glUseProgramObjectARB glUniform1i versus glUniform1iARB Aside

23. Wilf LaLonde @2012 Comp 4501 95.4002 How The Ambient Occlusion Demo Connects shaders with their textures

24. Wilf LaLonde @2012 Comp 4501 //Getting glGenTextures (1, &textureHandle); shaderProgramHandle = glCreateProgram (); //Deleting glDeleteTextures (1, &textureHandle); glDeleteProgram (shaderProgramHandle); //Using glBindTexture ( GL_TEXTURE_2D, textureHandle); or glBindTexture ( GL_TEXTURE_RECTANGLE, textureHandle); glUseProgram (shaderProgramHandle); Everything done via handles For coordinates in range 0..799 0 means “don’t use a shader”

25. Wilf LaLonde @2012 Comp 4501 //Uploading textures (MESSY) glBindTexture (GL_TEXTURE_2D, textureHandle); Supply a host of sampling information; e.g. GL_CLAMP versus GL_REPEAT Upload the bits to the card… via glTexImage2D or gluBuild2DMipmaps Everything done via handles AO Demo has no bits to upload, only game engine which also provides a Texture object to do it all.

26. Wilf LaLonde @2012 Comp 4501 //Uploading shaders (MESSY) Compile vertex shader, compile pixel shader. shaderProgramHandle = glCreateProgram (); Attach the shaders to the program. Link the program Everything done via handles AO Demo provides their code for doing this; game engine provides a Shader object for doing this.

27. Wilf LaLonde @2012 Comp 4501 //Connecting shader samplers to textures In shader “lesterLightingDiffuse” uniform sampler2D texture; After creating the shader (do this once) glUseProgram (shaderProgramHandle); glUniform1i ("texture", 0); //a texture unit When you want to draw with a shader glUseProgram (shaderProgramHandle); glActiveTexture ( GL_TEXTURE0 ); glBindTexture ( GL_TEXTURE_2D, textureHandle); OR glBindTexture ( GL_TEXTURE_RECTANGLE, textureHandle); What You Wanted To Know 0, 1, 2, 3, … For coordinates in range 0..799

28. Wilf LaLonde @2012 Comp 4501 Create via handles glGenFramebuffers (howMany, frameBufs); Plus code to create z-buffer and textures to draw into and code to attach to frame buffer Delete via handles glDeleteFramebuffers (howMany, frameBufs); Activate via handles glBindFramebuffer (GL_FRAMEBUFFER, frameBufs[i]); OR glBindFramebuffer (GL_FRAMEBUFFER, 0); FrameBuffers Are The SAME This means now draw into backbuffer Only AO used frame buffers

29. Wilf LaLonde @2012 Comp 4501 95.4002 How The Engine Differs From The AO Demo

30. Wilf LaLonde @2012 Comp 4501 Has texture and shader objects do some work. Texture provides test = Texture::read (“turkey”) //Creates handle + path, reads it texture->load (mipmapping, forceClamp) //Defaults to true, false (to upload onto card) activate () //To bind to the correct texture handle Shader provides activate () //To bind to the correct program handle The Engine

31. Wilf LaLonde @2012 Comp 4501 Shaders are a recent introduction that were meant to be specified in level file “castle4.wrl”. The start of the file contain the list of shaders used Shaders: 0; //currently Shader 1 name Shader 2 name … A shader manager keeps track of the shaders which it associates with an object and the terrain via a shader index… Each object including the terrain has a property "shader" => "-1“ //-1 means no shader Some Things That Can Cause Problems

32. Wilf LaLonde @2012 Comp 4501 It creates a default shader via defaultShaderIndex = addShader ("lesterlightingDiffuse"); When a object is imported, it executes if (shaderIndex == -1) shaderIndex = defaultShaderIndex; When the terrain is imported, it executes //if (shaderIndex == -1) shaderIndex = defaultShaderIndex; When the Shader Manager is Setup Please UNCOMMENT THIS LINE

33. Wilf LaLonde @2012 Comp 4501 When drawing an object (the castle) or the terrain, it activates the shader to use activates the texture to use But it fails to execute glActiveTexture ( GL_TEXTURE0 ); BECAUSE IT DOES THIS ONCE WHEN OPENGL IS SET UP AND ASSUMES IT’S STILL TRUE… The AO Demo code switches all over the place… How Else Does It Work

34. Wilf LaLonde @2012 Comp 4501 What strategy did I use? Create one file “ssao.cpp” containing all the ambient occlusion code and comment out stuff not needed. Add it to the “physicsForStudent” project including the AO shaders. Find a spot in “game.cpp” to call “WilfDisplay”. Find a spot in “ssao.cpp” to invoke “world->draw ()” Find a spot in “ssao.cpp” to blend AO with what game engine already drew… Making Ambient Occlusion Work With The Engine BASIC IDEA: AVOID CHANGING ANYTHING BEFORE IT WORKS.

35. Wilf LaLonde @2012 Comp 4501 Create one file “ssao.cpp” containing the contents of the ambient occlusion file “main.cpp”, “wilfMain.cpp”, and the routines that compile the shader. Add to project. Copied over the ambient occlusion shaders into a subdirectory of shaders and called it “AOshaders”. Changes lines from old locations to new “./src/shaders/ortho.vert”//OLD “../shaders/aoshaders/ortho.vert”//NEW Edited each shader so that the first line was #version 120 More Detail Steps

36. Wilf LaLonde @2012 Comp 4501 Fixed the "glew" problem by uncommenting "#include glew.h", commenting out 2 redeclaration error messages, and moving "glewInit" to spot after call to "setupOpenGL ()". Much later, I found out that I would crash in routine glBindFragDataLocation (how ssao routine binds a texture to a texture unit) but that it would work if I added EXT to the routine… INSTEAD, added the following after "#include glew.h", #undef glBindFragDataLocation #define glBindFragDataLocation glBindFragDataLocationEXT Making Ambient Occlusion Work With The Engine

37. Wilf LaLonde @2012 Comp 4501 There were 3 places in ssao.cpp file that draws quads. Not all of them were drawing counterclockwise (it matters because face culling now has to be on)… Making Ambient Occlusion Work With The Engine glBegin(GL_QUADS); //glVertex2d(0, 0); //glVertex2d(0, size); //glVertex2d(size, size); //glVertex2d(size, 0); glVertex2d (0, 0); //WILF: COUNTER-CLOCKWISE SO ALWAYS WORKS. glVertex2d (size, 0); glVertex2d (size, size); glVertex2d (0, size); glEnd();

38. Wilf LaLonde @2012 Comp 4501 Added 2 routines “setupAmbientOcclusion ()” and “wrapupAmbientOcclusion ()” to contain the setup/wrapup code in the ssao.cpp file and called them at the end of “Game::setup ()” and “Game::wrapup ()” Also added the following at the end of setupAmbientOcclusion and WilfDisplay… Add the following after call to Render2GBuffer. Making Ambient Occlusion Work With The Engine glBindFramebuffer(GL_FRAMEBUFFER, 0); |glActiveTexture(GL_TEXTURE0); glColor4d (1.0, 1.0, 1.0, 1.0); glUseProgram (0); glDisable (GL_TEXTURE_RECTANGLE); glEnable (GL_TEXTURE_RECTANGLE); Used to be permanently enabled (wrecking normal draws)

39. Wilf LaLonde @2012 Comp 4501 Made resolution, near, far, FOV same for both… Making Ambient Occlusion Work With The Engine //glutInitWindowSize (800, 600); glutInitWindowSize (800, 800); //WILF$$ MAKE THE SAME AS AO SHADER In game engine (main.cpp) const float zNear = 1.0; //0.1f; // WILF$$ MAKE NEAR AND FAR THE SAME AS ssao.cpp const float zFar = 2000; //1000.0f; // WILF$$ MAKE NEAR AND FAR THE SAME AS ssao.cpp const float fov = 40; //65.238f; //WILF$$ MAKE THE SAME AS ENGINE... In ssao.cpp

40. Wilf LaLonde @2012 Comp 4501 Making Ambient Occlusion Work With The Engine //glGetFloatv(GL_MODELVIEW_MATRIX, mVMat); CopyMemory (mVMat, &camera->inverseCameraMatrix.m11, sizeof (Transformation)); In ssao.cpp (replaced code that was grabbing the model view matrix mVMat from the stack by code that gets it from the existing camera objects). //M3DInvertMatrix44f(mVMat, iMVMat); CopyMemory (iMVMat, &camera->cameraMatrix.m11, sizeof (Transformation)); Similarly, code that compute the inverse is obtained from the camera THIS IS AN INVERSE

41. Wilf LaLonde @2012 Comp 4501 Assuming the engine draws the world and the ambient occlusion code draws the world, what’s the difference? The engine wants to draws with the "lesterlightingDiffuse“ shader… The ssao.cpp code draws with a bunch of different ambient occlusion shaders… The Most Important Thing To Understand HOW DO WE MAKE THAT WORK?

42. Wilf LaLonde @2012 Comp 4501 bool disableShaders = false; void Shader::activate () { extern bool disableShaders; if (disableShaders) return; if (isBroken) {deactivate (); return;} glUseProgramObjectARB (privateProgramHandle); } void Shader::deactivate () { extern bool disableShaders; if (disableShaders) return; glUseProgramObjectARB (0); } Controlling The Game Added a global just before void Game::draw () {…} Changed the routines that activate/deactivate shaders…

43. Wilf LaLonde @2012 Comp 4501 bool disableShaders = false; #define DISABLE disableShaders = true; #define ENABLE disableShaders = false; bool activateAmbientOcclusionBlending = false; void Game::draw () { //If there is no world, draw a teapot; otherwise, draw the world... //Neither the input manager nor the camera draws itself... enum Choice {AOOnly, NormalOnly, AOThenNormal, NormalThenAO}; Choice choice = NormalThenAO; camera->beginCamera (); if (world == NULL) { drawTeapots (); } else { //REVISED PART } player->draw (); camera->endCamera (); drawFrameRate (); drawHelp (); } REVISED GAME DRAWING

44. Wilf LaLonde @2012 Comp 4501 activateAmbientOcclusionBlending = choice == NormalThenAO; switch (choice) { case AOOnly: void WilfDisplay (); DISABLE; WilfDisplay (); break; case NormalOnly: ENABLE; world->draw (); break; case AOThenNormal: DISABLE; WilfDisplay (); ENABLE; world->draw (); break; case NormalThenAO: ENABLE; world->draw (); drawTeapots (); DISABLE; WilfDisplay (); break; } REVISED GAME DRAWING #define DISABLE disableShaders = true; #define ENABLE disableShaders = false;

45. Wilf LaLonde @2012 Comp 4501 void DrawModel () { //Used to draw sponza or sibenik models here… //Game engine needs to draw with texture unit 0… glActiveTexture (GL_TEXTURE0); game->world->draw (); } void Render2GBuffer () { …. DrawModel(); …. } REVISED ssao.cpp DRAWING

46. Wilf LaLonde @2012 Comp 4501 Recall: ssao.cpp makes and uses frame buffers to draw with… Frame buffers draw into textures. So you never see what they draw on the screen… Ultimately, though, it must draw into a back buffer… What controls that… BLENDING AO WITH GAME

47. Wilf LaLonde @2012 Comp 4501 void RenderAO (int size, int index) { … extern bool activateAmbientOcclusionBlending; if (size < RESOLUTION) { glBindFramebuffer(GL_FRAMEBUFFER, frameBufs[index]); glDrawBuffer(GL_COLOR_ATTACHMENT2); } else { glBindFramebuffer (GL_FRAMEBUFFER, 0); if (activateAmbientOcclusionBlending) { glEnable (GL_BLEND); glBlendFunc (GL_DST_COLOR, GL_ZERO); } CODE (NOT SHOWN) TO DRAW ONE QUAD… if (activateAmbientOcclusionBlending) {glDisable (GL_BLEND); } if (size < RESOLUTION && USE_EXTRA_UPSAMPLING_BLUR) Blur (size, index); } What Was The Last Thing Drawn By SSAO.cpp RUNS IN A LOOP WITH size going from small up to RESOLUTION LAST ITERATION, size is RESOLUTION

48. Wilf LaLonde @2012 Comp 4501 I did observe a glitch which I assumed was due to the temporal coherence feature. When I rotated continuously, I would sometimes see a dark triangle appear. Hacked the code in the shader not to do it... A Glitch float temporallyBlendedOcclusion (float occlusion) { return occlusion; //Disabling hack... return useTemporalSmoothing ? occlusion //temporalBlend (...) : occlusion; }

49. Wilf LaLonde @2012 Comp 4501 Since shader “geometry.frag” is computing the normal in camera space and face culling is on, only faces that face the camera will draw... So the normal MUST be pointing in the +z direction... If it’s not, make it so... Found a Fix for the ATI DFDY Instruction Bug Shader “geometry.frag #version 120#extension GL_ARB_texture_rectangle : require in vec4 pos; out vec4 Pos; out vec4 Norm; void main() { Pos = pos / pos.w; vec3 n = cross (dFdx (pos.xyz), dFdy (pos.xyz)); if (n.z < 0.0) n = -n; //If differentials have a bug, fix it.. Norm = vec4 (normalize(n), sign (abs (pos.z))); }

50. Wilf LaLonde @2012 Comp 4501 This should convince you that it’s doable… Did not try the shadow algorithm… So don’t know if anything is tricky… Conclusion