Stenciling Effects

Glenn G. ChappellCHAPPELLG@member.ams.org

U. of Alaska Fairbanks

CS 481/681 Lecture NotesMonday, February 9, 2004

9 Feb 2004 CS 481/681 2

Review:Details of Jittering [1/2] Some of the more interesting accumulation-based effects

can be accomplished via “jittering”. Jittering = making small, random-ish movements in a scene,

rendering for each movement, and blending the results. Jittering is useful for making things that look “fuzzy”.

We generally jitter in two dimensions. Using a table of x & y values is convenient. The file jitter.h contains a number of such tables. See

simplejitter.cpp for an example program that uses them. For each jitter, we use x and y to generate a

transformation. What sort of transformation? Where in the pipeline does it go? Answering these can be the trickiest part of jittering.

9 Feb 2004 CS 481/681 3

Review:Details of Jittering [2/2] We looked at three effects that can be created using

jittering: Anti-Aliasing

• Jitter the entire rendered image.• Transformations are simple translations.• They go after the usual projection (before it, in the code).

Depth-of-Field Effect• Jitter the 3-D scene, leaving a plane fixed.• Transformations are shears, and thus cannot be created using

only glTranslate*, glRotate*, glScale*.• They go before the usual projection (after it, in the code).

Soft Shadows• Use some shadowing method. Jitter the light source.• Transformations are translations, perpendicular to the

predominant light direction.• They go before (after, in the code) the rest of the model/view

transformation(s), for the light-source only.

9 Feb 2004 CS 481/681 4

Review:Introduction to Stenciling [1/3]

Stenciling involves the stencil buffer and the stencil test. Remember: allocate the buffer, enable the test. Clear the buffer the same way you clear any buffer.

The two major functions used in stenciling are glStencilFunc and glStencilOp. glStencilFunc determines what the stencil test does. glStencilOp determines what happens to the stencil

buffer if the stencil test passes or fails.• If the stencil test passes, then you can also have different

outcomes based on the depth test.

9 Feb 2004 CS 481/681 5

Review:Introduction to Stenciling [2/3] glStencilFunc takes three parameters:

A GLenum: what comparison the stencil test will do. A GLint used as a “reference value” in the stencil test.

• It is also used by the GL_REPLACE option of glStencilOp. A GLuint used as a mask (an “and” mask).

Think: REF COMPARE (buffer pixel & mask) Examples

Stencil test passes if bit in SB is on:glStencilFunc(GL_EQUAL, 1, 1);

Stencil test passes if bit in SB is off:glStencilFunc(GL_NOTEQUAL, 1, 1);

Test passes if 20 < low 8 bits in SB:glStencilFunc(GL_LESS, 20, 0xff);

9 Feb 2004 CS 481/681 6

Review:Introduction to Stenciling [3/3] glStencilOp takes three parameters, all GLenum’s:

Operation to perform if stencil test fails. Op. to perform if stencil test passes and depth test fails. Op. to perform if stencil test passes and depth test passes.

Examples Replace the SB value with the reference value (from

glStencilFunc):glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);

Do not modify the SB:glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

Replace SB value with zero, the reference value, or the bitwise inversion of the current SB value, respectively:glStencilFunc(GL_ZERO, GL_REPLACE, GL_INVERT);

Increment or decrement the SB value, as appropriate:glStencilFunc(GL_DECR, GL_INCR, GL_INCR);

9 Feb 2004 CS 481/681 7

Stenciling Effects:“Ordinary” Stenciling First we draw a shape in the stencil buffer.

If the shape is fixed, then we only need to redraw it when the viewport changes size. So this code can go in the reshape function.

glClearStencil(0);glClear(GL_STENCIL_BUFFER_BIT);glStencilFunc(GL_NEVER, 1, 1); // No "real" drawing.glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);Draw a shape here.glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

Then we use the above shape, in the display function.

glStencilFunc(GL_EQUAL, 1, 1);Draw something; it will appear only inside the above shape.glStencilFunc(GL_NOTEQUAL, 1, 1);Draw something; it will appear only outside the above shape.

9 Feb 2004 CS 481/681 8

Stenciling Effects:Reverse Painter’s Algorithm Recall that the R.P.A. can be used, with a method to order

polygons front-to-back, to do HSR. To order polygons, you might use a BSP tree.

To use the R.P.A., we draw front-to-back, and we make sure that no pixel color is set more than once. We can do the latter using stenciling.

glClearStencil(0);

glClear(GL_STENCIL_BUFFER_BIT |

GL_COLOR_BUFFER_BIT); // No depth buffer!

glStencilFunc(GL_NOTEQUAL, 1, 1);

glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);

Draw scene front-to-back.

9 Feb 2004 CS 481/681 9

Stenciling Effects:Capping Here is an implementation of “capping” (see Red p. 446).

Say we are drawing a number of closed objects. We wish to make sure that the inside of these is never visible, even if the near clipping plane slices one of them.

glClearStencil(0);

glClear(GL_STENCIL_BUFFER_BIT |

GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glStencilFunc(GL_ALWAYS, 1, 1);

glStencilOp(GL_INVERT, GL_INVERT, GL_INVERT);

Draw scene.glStencilFunc(GL_EQUAL, 1, 1);

Draw rectangle covering entire viewport, in “capping” color.

9 Feb 2004 CS 481/681 10

Stenciling Effects:Odd Things to Do Draw each pixel at most 5 times:

glClearStencil(0);glClear(GL_STENCIL_BUFFER_BIT);glStencilFunc(GL_GREATER, 5, 0xff);glStencilOp(GL_KEEP, GL_INCR, GL_INCR);

Draw each pixel successfully only on every other attempt:

glClearStencil(0);glClear(GL_STENCIL_BUFFER_BIT);glStencilFunc(GL_EQUAL, 0, 1);glStencilOp(GL_INVERT, GL_INVERT, GL_INVERT);

9 Feb 2004 CS 481/681 11

Shadow Volumes:Introduction In CS 381 we covered two shadowing

techniques: Shadowing via Projection

• The shadow is the shadow-caster drawn using the proper projection in the model/view matrix (and probably in a different color, too).

Shadowing via Textures• Draw the shadow-caster from the point of view of the light

source. Grab the resulting image for use as a texture.• Using the texture transformation, draw the shadow on the

shadow-receiver. Now we look at a third technique: shadow

volumes. This technique uses stenciling.

9 Feb 2004 CS 481/681 12

Shadow Volumes:The Idea [1/2]

Consider a number of polygons that cast non-overlapping shadows. The “non-overlapping” part is important, unfortunately. These polygons may be part of a single surface.

For each polygon, the region in its shadow is called its shadow volume.

If we can draw the portions of polygons that lie inside shadow volumes into the stencil buffer, then we can use stenciling to draw the shadowed scene. Right? So, how do we do this?

9 Feb 2004 CS 481/681 13

Shadow Volumes:The Idea [2/2]

Think about the boundaries of shadow volumes. A point is in shadow precisely when we must

look through an odd number of these boundaries to see it. Assuming the viewer is not in shadow. If the viewer is

in shadow, change “odd” to “even”. Zero is even (and not odd)!

Do you see why it isimportant that shadowvolumes do not overlap?

9 Feb 2004 CS 481/681 14

Shadow Volumes:The Technique First, draw the scene in the depth buffer:

Clear the depth buffer. Set up the stencil test so that it always passes. Draw the scene in the depth buffer, with the usual depth test.

Next the magic. Draw shadow-volume boundaries, inverting the SB value if the depth test passes.

Clear the SB. Set up stenciling:

• The depth buffer is never changed.• The depth test is as usual.• The stencil test always passes (this is already set).• If the stencil and depth tests pass, then invert the SB value. Otherwise, leave it alone.

Draw the shadow-volume boundaries. Now the non-zero portions of the SB are those in which we see something

in shadow, and we can draw the final scene: Fix the stencil buffer so it cannot be changed, and the depth buffer so it can. Clear the color and depth buffers. Draw scene with only ambient lighting, restricted to non-zero in SB. Draw scene with full lighting, restricted to zero in SB.

9 Feb 2004 CS 481/681 15

Note

Due to the Discovery Lab becoming available, we will stop the buffers/tests/blending unit here.

Next we will look at dealing with transformations in ways that are helpful in VR.