1. Ref: OpenGL Programming Guide (The Red Book)
Stencil Buffer
Ref: OpenGL Programming Guide (The Red Book)
Spring 2016 revised

2. Types of Buffers color (front/back, left/right, aux)
Front/back: animation
Left/right: stereo vision
Aux: not available on PC (use framebuffer object FBO instead)
depth buffer: depth test
VC: 24 bits available
Value : [0.0,1.0]
stencil buffer: restrict drawing to certain portions of the screen
VC: 8 bits available
Value: [0, 2^8-1]
accumulation buffer: accumulating composite image

3. Using Framebuffers clearing buffers
clearing individual buffer is expensive
Use glClear with bitwise-ORed masks to clear multiple buffers
selecting color buffers for writing/clearing
glDrawBuffer: useful in FBO (framebuffer object)

4. Masking Buffers
Before OpenGL writes data into the enabled color, depth, or stencil buffers, a masking operation is applied to the data, as specified with one of the following commands.
A bitwise logical AND is performed with each mask and the corresponding data to be written

5. Masking Buffers (cont)
void glColorMask(GLboolean red, GLboolean green, GLboolean blue, GLboolean alpha);
void glDepthMask(GLboolean flag);
void glStencilMask(GLuint mask);
If a 1 appears in mask, the corresponding bit in the stencil buffer is written; where a 0 appears, the bit is not written.
The default values of all the GLboolean masks are GL_TRUE, and the default values for the two GLuint masks are all 1's

6. Only Green Mask TRUE

7. Per-fragment Operation Sequence
Fragments: the pieces after rasterization, sent to framebuffers if they survive these tests
glScissor
restrict drawing to part of the window
a simpler (and faster) version of stencil test (that does not need stencil buffer)
application: gui panel
Dithering and logical op not covered here

8. Scissor Test See also stencil_close
While the scissor test is enabled, only pixels that lie within the scissor box can be modified by drawing commands.
Including glClear (shown right)
Note: validity of current raster position is determined by frustum (world coordinate); while scissor test affects the window coordinate
Single-pixel window: glScissor (x,y,1,1);
See also stencil_close

9. Stenciling

10. Basic Applications
Restrict rendering to limited portions of the screen
“Stenciling”
Decal
Shadow and lightmap

11. Mimicking Stencil Compose stencil template
Control template then render
Multi-pass rendering
silhouette

12. Decal Using Stencil Buffers
Stencil to resolve z-fighting

13. Shadow and Lightmap
Shadow and light blended with the textured polygon underneath

14. Technical Details glStencilFunc (fun, ref, mask)
func Specifies the test function. Eight tokens are valid: GL_NEVER, GL_LESS, GL_LEQUAL, GL_GREATER, GL_GEQUAL, GL_EQUAL, GL_NOTEQUAL, and GL_ALWAYS.
If it's GL_LESS, for example, then the fragment passes if reference value is less than the value in stencil buffer
ref Specifies the reference value for the stencil test. ref is clamped to the range [0, 2n - 1], where n is the number of bitplanes in the stencil buffer.
VC: n = 8
mask Specifies a mask that is ANDed with both the reference value and the stored stencil value when the test is done

15. Technical Details (cont)
glStencilOp (fail, zfail, zpass)
fail Specifies the action to take when the stencil test fails.
zfail Specifies stencil action when the stencil test passes, but the depth test fails.
zpass Specifies stencil action when both the stencil test and the depth test pass, or when the stencil test passes and either there is no depth buffer or depth testing is not enabled.
Six symbolic constants are accepted: GL_KEEP, GL_ZERO, GL_REPLACE, GL_INCR, GL_DECR, and GL_INVERT.
Set value to ref
Bitwise invert

16. Stenciling Steps to draw 2 coplanar rectangles:
Make the stencil for yellow one first (by drawing the green polygon)
Draw the yellow one with the stencil
Draw the green one

17. Stenciling (cont)
Stencil buffer
Color buffer

18. Decaling Draw everything else (set up the depth buffer)
[0] Base: (-5,0)(5,6)
[1] R: (-3,1)(2,4)
[2] G: (1,2)(3,5)
[3] B: (0,0)(4,3)
Foreground
blocker
Draw everything else (set up the depth buffer)
Draw base; set the blocker id to be 8; rest 0
Arrange the decal id in increasing order
The decals can be drawn in any order
Test the program…

19. Color buffer Stencil buffer Foreground blocker
StencilOp(fail, zfail, zpass)
Foreground
blocker 8 2 1 3
Color buffer
Stencil buffer

20. Drawn in different order!
StencilOp(fail, zfail, zpass)
Foreground
blocker 8 2 1 3
Color buffer
Stencil buffer
Drawn in different order!

21. Generating Silhouette
Stencil Buffer
Debug with GLIntercept

22. See also Different line style Hidden line removal Haloed lines
Cheap silhouette

23. Stencil Planar Shadow
Shadow matrix to project model onto a flat shadow receiver
Blending + stencil buffer techniques involved

24. Notice right image’s reflection falls off the floor!
Dinosaur is reflected by the planar floor.
Easy hack, draw dino twice, second time has glScalef(1,-1,1) to reflect through the floor
Notice right image’s reflection falls off the floor!

25. Stencil Reflection Reflection and shadow
Clear stencil to zero. Draw floor polygon with stencil set to one. Only draw reflection where stencil is one.
Reflection and shadow