Multi-pass Rendering

© 2002 James K. Hahn, N.H. Baek2 Multi-pass Rendering Repeat: image  Rendering pass  result imageRepeat: image  Rendering pass  result image Final image = combination of the result imagesFinal image = combination of the result images Example: final image = texture map + light map + fog mapExample: final image = texture map + light map + fog mapReason Speed : faster to separate passes, supported by hardwareSpeed : faster to separate passes, supported by hardware Lower algorithm complexityLower algorithm complexity Many effects possible by different combinations of passesMany effects possible by different combinations of passes Repeat: image  Rendering pass  result imageRepeat: image  Rendering pass  result image Final image = combination of the result imagesFinal image = combination of the result images Example: final image = texture map + light map + fog mapExample: final image = texture map + light map + fog mapReason Speed : faster to separate passes, supported by hardwareSpeed : faster to separate passes, supported by hardware Lower algorithm complexityLower algorithm complexity Many effects possible by different combinations of passesMany effects possible by different combinations of passes

© 2002 James K. Hahn, N.H. Baek3 Combining rendered images Porter and Duff, “Composing digital images”, Computer Graphics, 18(3):253~259, (1984).Porter and Duff, “Composing digital images”, Computer Graphics, 18(3):253~259, (1984). Duff, “Compositing 3D rendered images”, Computer Graphics, 19(3):41~44, (1985).Duff, “Compositing 3D rendered images”, Computer Graphics, 19(3):41~44, (1985). Possible to combine images with Z-valuesPossible to combine images with Z-values Pixel represented by up to 5 channels: R G B  ZPixel represented by up to 5 channels: R G B  Z  (alpha channel) : usually blending ratio  (alpha channel) : usually blending ratio Z : depth informationZ : depth information Porter and Duff, “Composing digital images”, Computer Graphics, 18(3):253~259, (1984).Porter and Duff, “Composing digital images”, Computer Graphics, 18(3):253~259, (1984). Duff, “Compositing 3D rendered images”, Computer Graphics, 19(3):41~44, (1985).Duff, “Compositing 3D rendered images”, Computer Graphics, 19(3):41~44, (1985). Possible to combine images with Z-valuesPossible to combine images with Z-values Pixel represented by up to 5 channels: R G B  ZPixel represented by up to 5 channels: R G B  Z  (alpha channel) : usually blending ratio  (alpha channel) : usually blending ratio Z : depth informationZ : depth information

© 2002 James K. Hahn, N.H. Baek4 Combining rendered images Basic binary operator:c = f op bBasic binary operator:c = f op b Z min operator : combine images using Z channelZ min operator : combine images using Z channel –RGB c = (if Z f < Z b then RGB f else RGB b ) –Z c = min(Z f, Z b ) over operator : combine images using alpha channelover operator : combine images using alpha channel –RGB c = RGB f + (1 –  f ) RGB b –  c =  f + (1 –  f )  b Basic binary operator:c = f op bBasic binary operator:c = f op b Z min operator : combine images using Z channelZ min operator : combine images using Z channel –RGB c = (if Z f < Z b then RGB f else RGB b ) –Z c = min(Z f, Z b ) over operator : combine images using alpha channelover operator : combine images using alpha channel –RGB c = RGB f + (1 –  f ) RGB b –  c =  f + (1 –  f )  b

© 2002 James K. Hahn, N.H. Baek5 composition operator: combine both of above operatorscomposition operator: combine both of above operators –RGB c =  (f over b) + (1 –  ) (b over f) –  area of pixel where f is in front of b composition operator: combine both of above operatorscomposition operator: combine both of above operators –RGB c =  (f over b) + (1 –  ) (b over f) –  area of pixel where f is in front of b

© 2002 James K. Hahn, N.H. Baek6 Blending (OpenGL) Alpha blending (OpenGL: dst is in frame buffer, src is incoming)Alpha blending (OpenGL: dst is in frame buffer, src is incoming) –(RGB  ) new = (RGB  ) src * blend src + (RGB  ) dst * blend dst –Depending on blend src and blend dst different effect Standard renderStandard render –(RGB  ) new =(RGB  ) src *  src +(RGB  ) dst *(1-  ) src Light map (src is light map)Light map (src is light map) –(RGB  ) new = (RGB  ) src * 0 + (RGB  ) dst * (RGB  ) src Alpha blending (OpenGL: dst is in frame buffer, src is incoming)Alpha blending (OpenGL: dst is in frame buffer, src is incoming) –(RGB  ) new = (RGB  ) src * blend src + (RGB  ) dst * blend dst –Depending on blend src and blend dst different effect Standard renderStandard render –(RGB  ) new =(RGB  ) src *  src +(RGB  ) dst *(1-  ) src Light map (src is light map)Light map (src is light map) –(RGB  ) new = (RGB  ) src * 0 + (RGB  ) dst * (RGB  ) src

© 2002 James K. Hahn, N.H. Baek7 Z-buffer (OpenGL) Original Z-buffer : update when Z value is lessOriginal Z-buffer : update when Z value is less Generalized in OpenGLGeneralized in OpenGL –Z comparison:, >=, ==, !=, always, never –Depth testing and z-value updating separated Original Z-buffer : update when Z value is lessOriginal Z-buffer : update when Z value is less Generalized in OpenGLGeneralized in OpenGL –Z comparison:, >=, ==, !=, always, never –Depth testing and z-value updating separated Z stored Z incoming

© 2002 James K. Hahn, N.H. Baek8 Accumulation buffer (OpenGL) accumaccum –(RGB  ) fb = (RGB  ) fb + value * (RGB  ) new loadload –(RGB  ) fb = value * (RGB  ) new addadd –(RGB  ) fb = value + (RGB  ) fb multmult –(RGB  ) fb = value * (RGB  ) fb accumaccum –(RGB  ) fb = (RGB  ) fb + value * (RGB  ) new loadload –(RGB  ) fb = value * (RGB  ) new addadd –(RGB  ) fb = value + (RGB  ) fb multmult –(RGB  ) fb = value * (RGB  ) fb

© 2002 James K. Hahn, N.H. Baek9 Stencil Buffer (OpenGL) Frame buffer over-written only if both stencil test and depth test passFrame buffer over-written only if both stencil test and depth test pass Stencil test:Stencil test: –if (ref & mask) op (stencil & mask) op:, >=, ==, !=, always, never ref: reference value for test mask: bit mask –Stencil buffer updated as: increase, decrease, invert, zero, keep, etc. Frame buffer over-written only if both stencil test and depth test passFrame buffer over-written only if both stencil test and depth test pass Stencil test:Stencil test: –if (ref & mask) op (stencil & mask) op:, >=, ==, !=, always, never ref: reference value for test mask: bit mask –Stencil buffer updated as: increase, decrease, invert, zero, keep, etc.

© 2002 James K. Hahn, N.H. Baek10 Examples of multi-pass algorithms

© 2002 James K. Hahn, N.H. Baek11 Back-to-front transparency Usual method 1. Draw opaque objects1. Draw opaque objects 2. Sort transparent objects back-to-front2. Sort transparent objects back-to-front 3. Render back-to-front using opacity/transparency factor t3. Render back-to-front using opacity/transparency factor t C = t C trans + (1 – t) C back Disadvantage : slow because of sortingDisadvantage : slow because of sorting Usual method 1. Draw opaque objects1. Draw opaque objects 2. Sort transparent objects back-to-front2. Sort transparent objects back-to-front 3. Render back-to-front using opacity/transparency factor t3. Render back-to-front using opacity/transparency factor t C = t C trans + (1 – t) C back Disadvantage : slow because of sortingDisadvantage : slow because of sorting C back C trans

© 2002 James K. Hahn, N.H. Baek12 Back-to-front transparency without sorting Deifenbach, “Pipeline rendering: Interaction and realism through hardware based multi-pass rendering”, Ph.D Thesis, Univ. Penn, (1996).Deifenbach, “Pipeline rendering: Interaction and realism through hardware based multi-pass rendering”, Ph.D Thesis, Univ. Penn, (1996). 2 OpenGL Z-buffers2 OpenGL Z-buffers Deifenbach, “Pipeline rendering: Interaction and realism through hardware based multi-pass rendering”, Ph.D Thesis, Univ. Penn, (1996).Deifenbach, “Pipeline rendering: Interaction and realism through hardware based multi-pass rendering”, Ph.D Thesis, Univ. Penn, (1996). 2 OpenGL Z-buffers2 OpenGL Z-buffers Zcurrent : depth of current pixel furthest from viewer but closer than Zprev Zprev : depth of previous processed transparent pixel

© 2002 James K. Hahn, N.H. Baek13 Back-to-front transparency initialize : draw opaque objectsinitialize : draw opaque objects looploop –Find transparent object furthest –Blend on top of opaque, update initialize : draw opaque objectsinitialize : draw opaque objects looploop –Find transparent object furthest –Blend on top of opaque, update Advance drawing opaque objects Z min Advance drawing transparent obj Don’t draw transparent, advance Z

© 2002 James K. Hahn, N.H. Baek14 Planar reflections Two approaches for modeling objects inside mirrorTwo approaches for modeling objects inside mirror –Reflect object –Reflect view point Two approaches for modeling objects inside mirrorTwo approaches for modeling objects inside mirror –Reflect object –Reflect view point mirror Reflect object Reflect view point

© 2002 James K. Hahn, N.H. Baek15 Planar reflection Rendering order (reverse also works) Render everything except mirror objectsRender everything except mirror objects Mirror area stencil buffer turned ONMirror area stencil buffer turned ON Reflect object to new locationReflect object to new location Render againRender again –Because stencil buffer only mirror area rendered Rendering order (reverse also works) Render everything except mirror objectsRender everything except mirror objects Mirror area stencil buffer turned ONMirror area stencil buffer turned ON Reflect object to new locationReflect object to new location Render againRender again –Because stencil buffer only mirror area rendered

© 2002 James K. Hahn, N.H. Baek16 Example : final result

© 2002 James K. Hahn, N.H. Baek17 Example : stencil buffer & mirror

© 2002 James K. Hahn, N.H. Baek18 Multi-pass shadow volumes McReynolds and Blythe, “Advanced graphics programming techniques using OpenGL”, SIGGRAPH’98 course notes, (1998).McReynolds and Blythe, “Advanced graphics programming techniques using OpenGL”, SIGGRAPH’98 course notes, (1998). Render scene as if it was entirely in shadow using Z-buffer (ambient light only)Render scene as if it was entirely in shadow using Z-buffer (ambient light only) Make stencil buffer from shadow volumeMake stencil buffer from shadow volume –“Render” front then back facing shadow volume to determine pixels that are between the two Render scene using lightingRender scene using lighting –Render outside stencil buffer –Outside shadow becomes brighter McReynolds and Blythe, “Advanced graphics programming techniques using OpenGL”, SIGGRAPH’98 course notes, (1998).McReynolds and Blythe, “Advanced graphics programming techniques using OpenGL”, SIGGRAPH’98 course notes, (1998). Render scene as if it was entirely in shadow using Z-buffer (ambient light only)Render scene as if it was entirely in shadow using Z-buffer (ambient light only) Make stencil buffer from shadow volumeMake stencil buffer from shadow volume –“Render” front then back facing shadow volume to determine pixels that are between the two Render scene using lightingRender scene using lighting –Render outside stencil buffer –Outside shadow becomes brighter

© 2002 James K. Hahn, N.H. Baek19 Spherical Light Objects Using Z-buffer create stencil bufferUsing Z-buffer create stencil buffer –Render front face with inverted Z-test (part below surface) –Render back face with inverted Z-test Render inside stencil buffer with lightRender inside stencil buffer with light Using Z-buffer create stencil bufferUsing Z-buffer create stencil buffer –Render front face with inverted Z-test (part below surface) –Render back face with inverted Z-test Render inside stencil buffer with lightRender inside stencil buffer with light camera stencil buffer ON Front-facing

© 2002 James K. Hahn, N.H. Baek20 Anti-aliasing Original use of accumulation bufferOriginal use of accumulation buffer Determine n view windowsDetermine n view windows Average the n imagesAverage the n images Result: super samplingResult: super sampling Can also work for other “Monte Carlo” integration methodsCan also work for other “Monte Carlo” integration methods –Temporal anti-aliasing –Depth of field Original use of accumulation bufferOriginal use of accumulation buffer Determine n view windowsDetermine n view windows Average the n imagesAverage the n images Result: super samplingResult: super sampling Can also work for other “Monte Carlo” integration methodsCan also work for other “Monte Carlo” integration methods –Temporal anti-aliasing –Depth of field

© 2002 James K. Hahn, N.H. Baek21 Simulated Depth of Field Render scene from different view point but same view plane and center of interest (shear projection)Render scene from different view point but same view plane and center of interest (shear projection) When averaged objects on focus plane in focusWhen averaged objects on focus plane in focus Render scene from different view point but same view plane and center of interest (shear projection)Render scene from different view point but same view plane and center of interest (shear projection) When averaged objects on focus plane in focusWhen averaged objects on focus plane in focus Object in focus Object out of focus viewpoints

© 2002 James K. Hahn, N.H. Baek22 Multi-texture Using hardware to combine a number of texture maps simultaneously with dynamically changing mapsUsing hardware to combine a number of texture maps simultaneously with dynamically changing maps –Texture map static, light map changed by light motion, fog map changed by camera motion Implemented in hardware for OpenGL graphics cardsImplemented in hardware for OpenGL graphics cards Using hardware to combine a number of texture maps simultaneously with dynamically changing mapsUsing hardware to combine a number of texture maps simultaneously with dynamically changing maps –Texture map static, light map changed by light motion, fog map changed by camera motion Implemented in hardware for OpenGL graphics cardsImplemented in hardware for OpenGL graphics cards texture map light map fog map mapping H/W mapping H/W mapping H/W no-texture image frame buffer

© 2002 James K. Hahn, N.H. Baek23 Multi-texture example: Detail modulation original image grain image

© 2002 James K. Hahn, N.H. Baek24 Detail modulated image

© 2002 James K. Hahn, N.H. Baek25 Example of Multi-texture: view-dependent light maps Light maps are usually for view independent (diffuse) light – slowly changing: low frequencyLight maps are usually for view independent (diffuse) light – slowly changing: low frequency Calculate number of high-frequency light maps corresponding to different view directionsCalculate number of high-frequency light maps corresponding to different view directions At runtime, blend between the different high- frequency light maps based on current view-direction using multi-textureAt runtime, blend between the different high- frequency light maps based on current view-direction using multi-texture Similar effect as bump mappingSimilar effect as bump mapping Light maps are usually for view independent (diffuse) light – slowly changing: low frequencyLight maps are usually for view independent (diffuse) light – slowly changing: low frequency Calculate number of high-frequency light maps corresponding to different view directionsCalculate number of high-frequency light maps corresponding to different view directions At runtime, blend between the different high- frequency light maps based on current view-direction using multi-textureAt runtime, blend between the different high- frequency light maps based on current view-direction using multi-texture Similar effect as bump mappingSimilar effect as bump mapping

© 2002 James K. Hahn, N.H. Baek26 Complete Light Model Routine rendering implemented entirely as set of map blending operationsRoutine rendering implemented entirely as set of map blending operations General light equationGeneral light equation –C = fog * (texture map * light map + specular map + environment map) Routine rendering implemented entirely as set of map blending operationsRoutine rendering implemented entirely as set of map blending operations General light equationGeneral light equation –C = fog * (texture map * light map + specular map + environment map)