1. Stenciling Effects Glenn G. Chappell CHAPPELLG@member.ams.org
U. of Alaska Fairbanks
CS 481/681 Lecture Notes
Monday, February 9, 2004

2. Review: Details of Jittering [1/2]
Some of the more interesting accumulation-based effects can be accomplished via “jittering”.
Jittering = making small, random-ish movements in a scene, rendering for each movement, and blending the results.
Jittering is useful for making things that look “fuzzy”.
We generally jitter in two dimensions.
Using a table of x & y values is convenient.
The file jitter.h contains a number of such tables. See simplejitter.cpp for an example program that uses them.
For each jitter, we use x and y to generate a transformation.
What sort of transformation?
Where in the pipeline does it go?
Answering these can be the trickiest part of jittering.
9 Feb 2004
CS 481/681

3. Review: Details of Jittering [2/2]
We looked at three effects that can be created using jittering:
Anti-Aliasing
Jitter the entire rendered image.
Transformations are simple translations.
They go after the usual projection (before it, in the code).
Depth-of-Field Effect
Jitter the 3-D scene, leaving a plane fixed.
Transformations are shears, and thus cannot be created using only glTranslate*, glRotate*, glScale*.
They go before the usual projection (after it, in the code).
Soft Shadows
Use some shadowing method. Jitter the light source.
Transformations are translations, perpendicular to the predominant light direction.
They go before (after, in the code) the rest of the model/view transformation(s), for the light-source only.
9 Feb 2004
CS 481/681

4. Review: Introduction to Stenciling [1/3]
Stenciling involves the stencil buffer and the stencil test.
Remember: allocate the buffer, enable the test.
Clear the buffer the same way you clear any buffer.
The two major functions used in stenciling are glStencilFunc and glStencilOp.
glStencilFunc determines what the stencil test does.
glStencilOp determines what happens to the stencil buffer if the stencil test passes or fails.
If the stencil test passes, then you can also have different outcomes based on the depth test.
9 Feb 2004
CS 481/681

5. Review: Introduction to Stenciling [2/3]
glStencilFunc takes three parameters:
A GLenum: what comparison the stencil test will do.
A GLint used as a “reference value” in the stencil test.
It is also used by the GL_REPLACE option of glStencilOp.
A GLuint used as a mask (an “and” mask).
Think: REF COMPARE (buffer pixel & mask)
Examples
Stencil test passes if bit in SB is on: glStencilFunc(GL_EQUAL, 1, 1);
Stencil test passes if bit in SB is off: glStencilFunc(GL_NOTEQUAL, 1, 1);
Test passes if 20 < low 8 bits in SB: glStencilFunc(GL_LESS, 20, 0xff);
9 Feb 2004
CS 481/681

6. Review: Introduction to Stenciling [3/3]
glStencilOp takes three parameters, all GLenum’s:
Operation to perform if stencil test fails.
Op. to perform if stencil test passes and depth test fails.
Op. to perform if stencil test passes and depth test passes.
Examples
Replace the SB value with the reference value (from glStencilFunc): glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
Do not modify the SB: glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
Replace SB value with zero, the reference value, or the bitwise inversion of the current SB value, respectively: glStencilFunc(GL_ZERO, GL_REPLACE, GL_INVERT);
Increment or decrement the SB value, as appropriate: glStencilFunc(GL_DECR, GL_INCR, GL_INCR);
9 Feb 2004
CS 481/681

7. Stenciling Effects: “Ordinary” Stenciling
First we draw a shape in the stencil buffer.
If the shape is fixed, then we only need to redraw it when the viewport changes size. So this code can go in the reshape function.
glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_NEVER, 1, 1); // No "real" drawing.
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
Draw a shape here.
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
Then we use the above shape, in the display function.
glStencilFunc(GL_EQUAL, 1, 1);
Draw something; it will appear only inside the above shape.
glStencilFunc(GL_NOTEQUAL, 1, 1);
Draw something; it will appear only outside the above shape.
9 Feb 2004
CS 481/681

8. Stenciling Effects: Reverse Painter’s Algorithm
Recall that the R.P.A. can be used, with a method to order polygons front-to-back, to do HSR.
To order polygons, you might use a BSP tree.
To use the R.P.A., we draw front-to-back, and we make sure that no pixel color is set more than once.
We can do the latter using stenciling.
glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT |
GL_COLOR_BUFFER_BIT); // No depth buffer!
glStencilFunc(GL_NOTEQUAL, 1, 1);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
Draw scene front-to-back.
9 Feb 2004
CS 481/681

9. Stenciling Effects: Capping
Here is an implementation of “capping” (see Red p. 446).
Say we are drawing a number of closed objects. We wish to make sure that the inside of these is never visible, even if the near clipping plane slices one of them.
glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT |
GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glStencilFunc(GL_ALWAYS, 1, 1);
glStencilOp(GL_INVERT, GL_INVERT, GL_INVERT);
Draw scene.
glStencilFunc(GL_EQUAL, 1, 1);
Draw rectangle covering entire viewport, in “capping” color.
9 Feb 2004
CS 481/681

10. Stenciling Effects: Odd Things to Do
Draw each pixel at most 5 times:
glClearStencil(0);
glClear(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_GREATER, 5, 0xff);
glStencilOp(GL_KEEP, GL_INCR, GL_INCR);
Draw each pixel successfully only on every other attempt:
glStencilFunc(GL_EQUAL, 0, 1);
glStencilOp(GL_INVERT, GL_INVERT, GL_INVERT);
9 Feb 2004
CS 481/681

11. Shadow Volumes: Introduction
In CS 381 we covered two shadowing techniques:
Shadowing via Projection
The shadow is the shadow-caster drawn using the proper projection in the model/view matrix (and probably in a different color, too).
Shadowing via Textures
Draw the shadow-caster from the point of view of the light source. Grab the resulting image for use as a texture.
Using the texture transformation, draw the shadow on the shadow-receiver.
Now we look at a third technique: shadow volumes.
This technique uses stenciling.
9 Feb 2004
CS 481/681

12. Shadow Volumes: The Idea [1/2]
Consider a number of polygons that cast non-overlapping shadows.
The “non-overlapping” part is important, unfortunately.
These polygons may be part of a single surface.
For each polygon, the region in its shadow is called its shadow volume.
If we can draw the portions of polygons that lie inside shadow volumes into the stencil buffer, then we can use stenciling to draw the shadowed scene.
Right?
So, how do we do this?
9 Feb 2004
CS 481/681

13. Shadow Volumes: The Idea [2/2]
Think about the boundaries of shadow volumes.
A point is in shadow precisely when we must look through an odd number of these boundaries to see it.
Assuming the viewer is not in shadow. If the viewer is in shadow, change “odd” to “even”.
Zero is even (and not odd)!
Do you see why it is important that shadow volumes do not overlap?
9 Feb 2004
CS 481/681

14. Shadow Volumes: The Technique
First, draw the scene in the depth buffer:
Clear the depth buffer.
Set up the stencil test so that it always passes.
Draw the scene in the depth buffer, with the usual depth test.
Next the magic. Draw shadow-volume boundaries, inverting the SB value if the depth test passes.
Clear the SB.
Set up stenciling:
The depth buffer is never changed.
The depth test is as usual.
The stencil test always passes (this is already set).
If the stencil and depth tests pass, then invert the SB value. Otherwise, leave it alone.
Draw the shadow-volume boundaries.
Now the non-zero portions of the SB are those in which we see something in shadow, and we can draw the final scene:
Fix the stencil buffer so it cannot be changed, and the depth buffer so it can.
Clear the color and depth buffers.
Draw scene with only ambient lighting, restricted to non-zero in SB.
Draw scene with full lighting, restricted to zero in SB.
9 Feb 2004
CS 481/681

15. Note
Due to the Discovery Lab becoming available, we will stop the buffers/tests/blending unit here.
Next we will look at dealing with transformations in ways that are helpful in VR.
9 Feb 2004
CS 481/681