Advanced Graphics Algorithms Ying Zhu Georgia State University Lecture 12 Fog, Transparency, and other per-fragment operations

Rasterization

Sub-stages of rasterization What we have covered What we will cover in this lecture

Fog If enabled, fog blends a fog color with a rasterized fragment’s post-texturing color using a blending factor f. Fog effect is simulated by having f depends on distance to the camera Blend source color Cs and fog color Cf by Cs’=f Cs + (1-f) Cf f is the fog factor, computed according one of the three equations Exponential Gaussian Linear (depth cueing)

Fog Functions

Setting fog parameters in OpenGL GLfloat fcolor[4] = {……}: glEnable(GL_FOG); // enable fog glFogf(GL_FOG_MODE, GL_EXP);// specify fog mode glFogf(GL_FOG_DENSITY, 0.5);// specify fog density glFOgv(GL_FOG, fcolor); // specify fog color Or glEnable(GL_FOG); glFogf(GL_FOG_MODE, GL_LINEAR); glFogf(GL_FOG_START, 5); glFogf(GL_FOG_END, 100); glFOgv(GL_FOG, fcolor);

OpenGL Fog Functions The equation for GL_LINEAR fog is Specify fog mode by glFogf(GL_FOG_MODE, param) GL_LINEAR GL_EXP GL_EXP2 The equation for GL_LINEAR fog is f = (end - z) / (end - start) The equation for GL_EXP fog is f = e^(-(density * z)) The equation for GL_EXP2 fog is f = e^(-(density * z)² )

OpenGL Fog Example Code segment taken from http://www.opengl.org/resources/code/samples/redbook/fog.c /* Initialize depth buffer, fog, light source, material property, and lighting model. */ static void init(void) { GLfloat position[] = { 0.5, 0.5, 3.0, 0.0 }; glEnable(GL_DEPTH_TEST); glLightfv(GL_LIGHT0, GL_POSITION, position); glEnable(GL_LIGHTING); glEnable(GL_LIGHT0); GLfloat mat[3] = {0.1745, 0.01175, 0.01175}; glMaterialfv (GL_FRONT, GL_AMBIENT, mat); mat[0] = 0.61424; mat[1] = 0.04136; mat[2] = 0.04136; glMaterialfv (GL_FRONT, GL_DIFFUSE, mat); mat[0] = 0.727811; mat[1] = 0.626959; mat[2] = 0.626959; glMaterialfv (GL_FRONT, GL_SPECULAR, mat); glMaterialf (GL_FRONT, GL_SHININESS, 0.6*128.0); }

OpenGL Fog Example glEnable(GL_FOG); { GLfloat fogColor[4] = {0.5, 0.5, 0.5, 1.0}; fogMode = GL_EXP; glFogi (GL_FOG_MODE, fogMode); glFogfv (GL_FOG_COLOR, fogColor); glFogf (GL_FOG_DENSITY, 0.35); glHint (GL_FOG_HINT, GL_DONT_CARE); } glClearColor(0.5, 0.5, 0.5, 1.0); /* fog color */ // color_after_fog = color * e^(-0.35 * z) + (1 - e^(-0.35 * z))*0.5

OpenGL Fog Example static void renderTeapot (GLfloat x, GLfloat y, GLfloat z) { glPushMatrix(); glTranslatef (x, y, z); glutSolidTeapot(1.0); glPopMatrix(); } /* display() draws 5 spheres at different z positions. */ void display(void) glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); renderTeapot (-2., -0.5, -1.0); renderTeapot (-1., -0.5, -2.0); renderTeapot (0., -0.5, -3.0); renderTeapot (1., -0.5, -4.0); renderTeapot (2., -0.5, -5.0); glFlush();

Pixel (fragment) shader What we have discussed so far is called fixed-function rasterization Line/polygon rasterization, texture mapping, fog, line anti-aliasing, etc. You can write a pixel (fragment) shader to handle texture mapping and fog If a fragment shader is active, OpenGL texture mapping and fog is turned off Pixel shaders allow developers to implement more complex texture mapping and fog algorithms Bump mapping, environment mapping, projective texturing, shadow map, height based fog, etc. Line anti-aliasing in pixel shader is still tricky Pixel shaders have no access to line/polygon rasterization

Fragment shader

Frame buffer The frame buffer consists of a set of pixels arranged as a 2D array Each pixel in the framebuffer is simply a set of some number of bits Each pixel has different bits for different purposes Color buffer bits Depth buffer bits Stencil buffer bits Accumulation buffer bits

OpenGL frame fuffer

Frame buffer

Color buffer Contains RGB color data and optionally alpha values per pixel May consist of several logical buffers: Front left, front right, back left, back right, and some auxiliary buffers Most graphics cards have at least a front and a back buffers (double buffering) The contents of the front buffers are displayed on a color monitor while OpenGL writing into the back buffer Reducing flashing effect Switch buffers by glutSwapBuffers()

Depth buffer Stores a depth value for each pixel Depth is usually measured in terms of distance to the eye Pixels with larger depth-buffer values are overwritten by pixels with smaller values A hidden surface removal technique Sometimes called the z buffer

Stencil buffer To restrict drawing to certain portions of the screen Similar to using a cardboard stencil for spay paint to make precise painted images For example, you want to draw an image as it would appear through an odd-shaped windshield Store an image of the windshield’s shape in the stencil buffer and draw the entire scene The stencil buffer prevents anything that wouldn’t be visible through windshield from being draw

Accumulation buffer Holds RGBA color data per pixel Typically used for accumulating a series of images into a final, composite image Can be used to achieve FSAA, motion blur, depth of field, shadow, etc. Data in accumulation buffer is usually copied from a color buffer

Clearing buffers Set the clearing values for each buffer glClearColor() glClearDepth() glClearStencil() glClearAccum() After selecting the clearing value, clear the buffer by calling glClear() E.g. glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT | GL_ACCUM_BUFFER_BIT)

Read from or write into color buffer glReadPixels(x,y,width,height,format,type,myimage) type of pixels start pixel in frame buffer size type of image pointer to processor memory GLubyte myimage[512][512][3]; glReadPixels(0,0, 512, 512, GL_RGB, GL_UNSIGNED_BYTE, myimage); glDrawPixels(width,height,format,type,myimage) starts at raster position

Per-fragment operations When you draw geometry (or text, images) on the screen, OpenGL will rotate, translate, scale objects determine lighting project the objects into perspective figure out which pixels in the window are affected determine the colors of those pixels After that, each pixel has to go through several more processing stages to determine how and whether the pixel is draw into the frame buffer.

Per-fragment operations

Per-fragment operations

Stencil test Turn on stencil test by glEnable(GL_STENCIL_TEST) The stencil test compares a reference value with the value stored at a pixel in the stencil buffer Depending on the result of the test, the fragment is accepted or rejected The value in the stencil buffer is modified based the test result In OpenGL, you can set Comparison function (glStencilFunc()) Reference value (glStencilFunc()) How to modification stencil buffer (glStencilOp())

Stencil test The most typical use of the stencil test is to mask out an irregularly shaped region of the screen to prevent drawing from occurring within it. Steps (two pass rendering): Fill the stencil buffer with 0’s Draw the desired shape in the stencil buffer with 1’s. (Draw the desired shape while modifying stencil buffer) Set stencil reference value to 1 Set the comparison function such that the fragment passes if the reference value equal to the corresponding stencil buffer value Draw the scene without modifying the contents of stencil buffer

Stencil test example /* code segment taken from http://www.sgi.com/products/software/opengl/examples/redbook/source/stencil.c */ void init (void) { … glEnable(GL_LIGHT0); glEnable(GL_LIGHTING); glEnable(GL_DEPTH_TEST); glClearStencil(0x0); glEnable(GL_STENCIL_TEST); }

Stencil test example /* Draw a sphere in a diamond-shaped section in the middle of a window with 2 torii. */ void display(void) { glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); // draw blue sphere where the stencil is 1 (inside quad area) glStencilFunc (GL_EQUAL, 0x1, 0x1); glStencilOp (GL_KEEP, GL_KEEP, GL_KEEP); // Don’t modify stencil buffer glCallList (BLUEMAT); // set sphere surface material glutSolidSphere (0.5, 15, 15); // draw the tori where the stencil is not 1 (outside the quad area) glStencilFunc (GL_NOTEQUAL, 0x1, 0x1); glPushMatrix(); glRotatef (45.0, 0.0, 0.0, 1.0); glRotatef (45.0, 0.0, 1.0, 0.0); glCallList (YELLOWMAT); glutSolidTorus (0.275, 0.85, 15, 15); glRotatef (90.0, 1.0, 0.0, 0.0); glPopMatrix(); }

Stencil test example /* Whenever the window is reshaped, redefine the coordinate system and redraw the stencil area. */ void reshape(int w, int h) { glViewport(0, 0, (GLsizei) w, (GLsizei) h); glMatrixMode(GL_PROJECTION); glLoadIdentity(); … // create projection matrix glMatrixMode(GL_MODELVIEW);

Stencil test example /* create a diamond shaped stencil area */ glClear(GL_STENCIL_BUFFER_BIT); // Pixels always pass stencil test (no restriction on pixels) glStencilFunc (GL_ALWAYS, 0x1, 0x1); // When you draw object, also draw to stencil buffer glStencilOp (GL_REPLACE, GL_REPLACE, GL_REPLACE); // The quad is drawn to both color buffer and stencil buffer, but we only need the one in the stencil buffer glBegin(GL_QUADS); glVertex2f (-1.0, 0.0); glVertex2f (0.0, 1.0); glVertex2f (1.0, 0.0); glVertex2f (0.0, -1.0); glEnd(); // set projection // set eye position, etc. }

Stencil test example /* Main Loop * Be certain to request stencil bits. */ int main(int argc, char** argv) { glutInit(&argc, argv); glutInitDisplayMode (GLUT_SINGLE | GLUT_RGB | GLUT_DEPTH | GLUT_STENCIL); glutInitWindowSize (400, 400); glutInitWindowPosition (100, 100); glutCreateWindow (argv[0]); init (); glutReshapeFunc(reshape); glutDisplayFunc(display); glutKeyboardFunc(keyboard); glutMainLoop(); return 0; }

Depth Test For each pixel on the screen, the depth buffer keeps track of the distance between the viewpoint and the object occupying that pixel. If the specified depth test passes, the incoming depth value replaces the value already in the depth buffer. The depth buffer is generally used for hidden-surface removal. Multiple pixels may have the same (Xw,Yw) and depth value A pixel is draw to color buffer only if that pixel’s depth value is smaller than the current depth value Think depth test as sorting pixels with same depth value and only keeps the one with the smallest depth and draw it to color buffer

Depth test With depth test turned on, after the entire scene is drawn, only objects that aren’t obscured by other items remain. Steps to set up depth test Call glEnable(GL_DEPTH_TEST) Include GLUT_DEPTH in glutInitDisplayMode () Include GL_DEPTH_BUFFER_BIT when you call glClear() in dislay() function before you draw each frame See http://www.sgi.com/products/software/opengl/examples/redbook/source/stencil.c for example

Depth test for hidden surface removal The depth buffer algorithm is as follows: for each polygon P for each pixel (x, y) in P compute z_depth at x, y if z_depth < z_buffer (x, y) then set_pixel (x, y, color) z_buffer (x, y)  z_depth

Depth test By default, the depth test pass if a pixel’s depth value is less than the corresponding depth value in the depth buffer (previous smallest value) You can change the comparison function by calling glDepthFunc() By default, the new depth value is written into the depth buffer is pixel passes depth test You can make depth buffer read-only by calling glDepthMask(GL_FALSE)

Depth buffer algorithm Depth buffer is only one of the many hidden-surface removal algorithms Advantages of z-buffer algorithm: It always works and is simple to implement. Disadvantages: May paint the same pixel several times and computing the color of a pixel may be expensive. Large memory requirements: not a big problem for modern graphics cards.

Compositing and Blending In RGBA mode, pixels can be drawn using a function that blends the incoming (source) RGBA values with the RGBA values already in the frame buffer (destination values). Can Alpha component in RGBA color as a blending factor for Translucent surfaces Compositing images Anti-aliasing

Opacity and Transparency Opaque surfaces permit no light to pass through Transparent surfaces permit all light to pass Translucent surfaces pass some light translucency = 1 – opacity (a) opaque surface a =1

OpenGL Blending and Compositing Must enable blending and pick source and destination factors glEnable(GL_BLEND) glBlendFunc(source_factor, destination_factor) Only certain factors supported GL_ZERO, GL_ONE GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA GL_DST_ALPHA, GL_ONE_MINUS_DST_ALPHA See glBlendFunc() man page for complete list

OpenGL Blending and Compositing When blending is enabled, the final color in the frame buffer is final_color = source_factor * source_color + destination_factor * destination_color Example: Current pixel color (0.2, 0.4, 0.3, 0.7), corresponding pixel color in frame buffer (0.9, 0.2, 0.1, 1.0) glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) Final color: R = 0.2 * 0.7 + 0.9 * 0.3; G = 0.4 * 0.7 + 0.2 * 0.3; B = 0.3 * 0.7 + 0.1 * 0.3;

Typical uses of glBlendFunc() For transparency and line anti-aliasing use glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) Where alpha value represents material opacity for the incoming pixel For polygon anti-aliasing, use glBlendFunc(GL_SRC_ALPHA_SATURATE, GL_ONE) with polygons sorted from nearest to farthest

Order Dependency Is this image correct? Probably not Polygons are usually rendered in the order they specified in OpenGL program Blending functions are order dependent With the same blending function, the result may be different when the polygons are drawn in different order The value of source alpha and destination alpha may be different depending on which pixel gets drawn first

OpenGL Example static void init(void) { glEnable (GL_BLEND); glBlendFunc (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA); glShadeModel (GL_FLAT); glClearColor (0.0, 0.0, 0.0, 0.0); } static void drawLeftTriangle(void) /* draw yellow triangle on LHS of screen */ glBegin (GL_TRIANGLES); glColor4f(1.0, 1.0, 0.0, 0.75); glVertex3f(0.1, 0.9, 0.0); glVertex3f(0.1, 0.1, 0.0); glVertex3f(0.7, 0.5, 0.0); glEnd();

OpenGL Example static void drawRightTriangle(void) { /* draw cyan triangle on RHS of screen */ glBegin (GL_TRIANGLES); glColor4f(0.0, 1.0, 1.0, 0.75); glVertex3f(0.9, 0.9, 0.0); glVertex3f(0.3, 0.5, 0.0); glVertex3f(0.9, 0.1, 0.0); glEnd(); } void display(void) glClear(GL_COLOR_BUFFER_BIT); if (leftFirst) { drawLeftTriangle(); drawRightTriangle(); else { drawRightTriangle(); drawLeftTriangle(); glFlush();

Accumulation Buffer Compositing and blending are limited by resolution of the frame buffer Typically 8 bits per color component The accumulation buffer is a higher precision frame buffer (16 or more bits per component) used to accumulate intermediate rendering results. Viewing and projection matrices can be altered to provide multiple samples. These multiple samples result in multiple images, and these images are typically added into the accumulation buffer. The resulting accumulated images are then usually scaled to produce an average that represents a filtered reconstruction of the individual sample images.

Manipulating Accumulation Buffer in OpenGL glAccum(GLenum op, GLfloat value) Op GL_ACCUM : obtains R, G, B, and A values from the current color buffer and multiply them by value and added to the corresponding pixel component in the accumulation buffer, thereby updating the accumulation buffer. GL_LOAD: similar to GL_ACCUM, except that the current value in the accumulation buffer is not used in the calculation of the new value. That is, the R, G, B, and A values from the currently selected buffer, multiplied by value, and then stored in the corresponding accumulation buffer cell, overwriting the current value. GL_ADD: Adds value to each R, G, B, and A in the accumulation buffer. GL_MULT: Multiplies each R, G, B, and A in the accumulation buffer by value and returns the scaled component to its corresponding accumulation buffer location. GL_RETURN: Transfers accumulation buffer values to the color buffer or buffers currently selected for writing. Each R, G, B, and A component is multiplied by value. To clear the accumulation buffer, call glClearAccum() with R, G, B, and A values to set it to, then call glClear with the accumulation buffer enabled.

Accumulation buffer applications: FSAA Full Scene Anti-Aliasing (FSAA) Draw the scene several times Each time slightly shift the scene Draw scenes into accumulation buffer and blend them http://www.opengl.org/resources/code/samples/glut_examples/advanced/accumaa.c

Accumulation buffer applications: Motion Blur Remove the jerkiness from a computer-generated animation Create the illusion of high speed motion. Strategy Re-render a scene multiple times, incrementing the position and/or orientation of an object in the scene. Render the scene without the moving object, using glAccum(GL_LOAD,.5f) Accumulate the scene 10 more times, with the moving object, using glAccum(GL_ACCUM,.05f) http://www.opengl.org/resources/code/samples/glut_examples/advanced/motionblur.c

Accumulation buffer applications: Depth of Field Computer graphics uses the pinhole camera model This results in perfectly sharp images Real cameras use lenses with variable aperture sizes This causes depth-of-field: out-of-focus objects appear blurry

Depth of field

Depth of Field An important part of the storytelling process Direct viewer’s eyes to a certain area of the scene Can use accumulation buffer to blur the background (or a certain area) http://www.opengl.org/resources/code/samples/glut_examples/advanced/field.c

Summary Rasterization Frame buffer Per-fragment operations Scan conversion, texture mapping, fog, line-antialiasing Frame buffer Color, depth, stencil, accumulation buffers Per-fragment operations Scissor, alpha, stencil, depth tests Blendling, dithering, and logical operations Whole frame buffer operations Use accumulation buffers for FSAA, DOF, motion blur

3D pipeline revisited We have taken a trip down the geometry and rasterizer stage of the traditional 3D graphics pipeline These two stages are now almost entirely implemented on graphics hardware Most of the OpenGL function calls are executed on graphics card Along the way, we have covered some basic OpenGL function calls Transformation, viewing, lighting, projection, texture mapping, fog, per-fragment operation, etc.

3D pipeline revisited

Readings OpenGL Programming Guide: chapter 10 Code reading: http://www.sgi.com/products/software/opengl/examples/redbook/ Fog.c, aargb.c, alpha.c, stencil.c, accpersp.c, accanti.c, dof.c,

Next lecture GPU and shader