03/30/05© 2005 University of Wisconsin NPR Today “Stylized Rendering Techniques for Scalable Real-Time 3D Animation”, Adam Lake, Carl Marshall, Mark Harris and Marc Blackstein, NPAR 2001 "Computer-Generated Pen-and-Ink Illustration of Trees", Oliver Deussen and Thomas Strothotte, SIGGRAPH 2000 "Sketchy Drawings", Marc Nienhaus and Jurgen Dollner, AfriGraph 2004

03/30/03© 2005 University of Wisconsin Stylized Rendering Techniques for Scalable Real- Time 3D Animation Paper Overview Leo Chao

03/30/03© 2005 University of Wisconsin Paper Information Published –NPAR 2001 (Non-Photorealistic Animation and Rendering) –Authors Adam Lake Carl Marshall Mark Harris Marc Blackstein Graphics Algorithms and 3D Technologies Group –Intel Architecture Labs UNC – Chapel Hill

03/30/03© 2005 University of Wisconsin Paper Information Prior Work –No novel techniques, adapts from prior works to hardware Motion Lines –Maic Masuch. “Speedlines: Depicting Motion in Motionless Pictures.” ACM SIGGRAPH 99 Technical sketch. –Siu Chi Hsu and Irene H. H. Lee. "Drawing and Animation using Skeletal Strokes". In Proceedings of ACM SIGGRAPH 94, pages Silhouette Edges –Lee Markosian, Michael Kowalski, Samuel Trychi, Lubomir Bourdev, Daniel Goldstein, and John Hughes. Real-Time Nonphotorealistic Rendering. In Proceedings of ACM SIGGRAPH 97, pages Cartoon Shading –Philippe Decaudin. Rendu de scenes 3D imitant le style dessin anime. Rapport de Recherche 2919, Institut National de Recherche en Informatique et en Automatique

03/30/03© 2005 University of Wisconsin Contributions Hardware Implementation of NPR Techniques –Stylistic Shading Cartoon Shading Pencil Sketch Shading –Stylistic Inking Silhouette Edge Detection Stylized Silhouette Rendering –Motion Lines

03/30/03© 2005 University of Wisconsin Stylistic Shading Fundamentals –Preprocess colour textures –Render as normal –Except: Modify lighting/shading calculations May require texture coordinate processing “Painter” – Modified Shading Step –Cartoon Shading –Pencil Sketch Shading

03/30/03© 2005 University of Wisconsin Stylistic Shading Cartoon Shading –Reducing detail aids comprehension –No Smooth Shading Color Reduction Hard Transitions Variants Simple

03/30/03© 2005 University of Wisconsin Stylistic Shading

03/30/03© 2005 University of Wisconsin ALGORITHM Cartoon Shade Preprocess: 1.Calculate the illuminated diffuse color for each material: 2. Calculate the shadowed diffuse color: 3. For each material, create a texture with two colors - u=1 is coloured C i - u=0 is coloured C s Stylistic Shading C i Vertex Color Cs Shadow Diffuse Color a l Light Ambient d l Light Diffuse a m Material Ambient d m Material Diffuse a g Coefficient of Global Ambient Lighting

03/30/03© 2005 University of Wisconsin ALGORITHM Cartoon Shade Runtime: 1.Calculate the one-dimensional texture coordinate at each vertex 2. Render the model using a standard graphics API - Lighting disabled - Enable texture mapping Stylistic Shading L normalized light vector n unit surface normal

03/30/03© 2005 University of Wisconsin Stylistic Shading Pencil Sketch Shading –Extends cartoon shading to 2D textures –Extends cartoon shading to multiple textures –Multiple textures to represent light density –Associate texture coordinates Project texture from the viewport onto the model

03/30/03© 2005 University of Wisconsin Stylistic Shading

03/30/03© 2005 University of Wisconsin Stylistic Inking Fundamentals –Identify “important” edges Silhouette Border Crease –Modify drawing of these edges “Inker” – Modified Line Drawing –Visually important edge detection –Stylized edge drawing

03/30/03© 2005 University of Wisconsin Stylized Inking Visually important edge detection –Silhouettes Front-Face and Back-Face share an edge –Borders Edge shared by two materials Unshared polygon edge –Creases Large dihedral angle between polygons

03/30/03© 2005 University of Wisconsin Stylized Inking

03/30/03© 2005 University of Wisconsin Stylized Inking ALGORITHM SED Preprocess: 1.Create edge-list structure with Edge Type flags a.Create a unique edge list using a hash table. b.Edge Hash Function: sum the two vertex indices c.Border Flag -Edges with only one neighboring face -Two neighboring faces with different materials d.Set crease flags -Non-deformable geometry -Dihedral angle of two faces > threshold

03/30/03© 2005 University of Wisconsin Stylized Inking ALGORITHM SED Runtime: 1.For deformable meshes: a.Calculate face normals b.Detect crease edges from face normals c.Set Crease Edge flags 2. Detect silhouette edges a.

03/30/03© 2005 University of Wisconsin Stylized Inking Stylized “Important” Lines –Emphasis on these lines –Option 1: Straight Line Fast/Easy –Option 2: Texture Maps “Artistic” Style Curvature driven “strokes” Involves a graph driven search This is a HACK

03/30/03© 2005 University of Wisconsin Stylized Inking

03/30/03© 2005 University of Wisconsin Motion Lines Gives the appearance of motion For fast moving objects Simple algorithm Not much overhead –One circular buffer to maintain position data

03/30/03© 2005 University of Wisconsin Performance Claim: Maintain Real-Time Speeds –30 fps, in 2001 Tests: –Integrated with a level of detail scheme –Altered model complexity to maintain FPS

03/30/03© 2005 University of Wisconsin Performance

03/30/03© 2005 University of Wisconsin Future Work Inker –Better “artistic” lines Painter –Pencil sketch “doesn’t look right” –Follow surface gradient perhaps Incorporate other NPR techniques –Those used by artists –Doesn’t say which would amenable

03/30/03© 2005 University of Wisconsin Questions? ques·tion (kw s ch n) n. –An expression of inquiry that invites or calls for a reply. heck·le (h k l) tr.v. –To try to embarrass and annoy (someone speaking or performing in public) by questions, gibes, or objections

03/30/05© 2005 University of Wisconsin

03/30/03© 2005 University of Wisconsin “Computer-Generated Pen-and-Ink Illustration of Trees” (SIGGRAPH 2000) Oliver Deussen Thomas Strothotte

03/30/03© 2005 University of Wisconsin Motivation The paper presents a method for creating pen-and-ink-like illustrations from detailed geometric models. Motivated by the need for illustrative representations of plant life in architectural and landscape renderings. We want to be able to produce images similar to this:

03/30/03© 2005 University of Wisconsin Related Work “Cartoon Tree” – Alvay Ray Smith –Use small disks to represent bunches of leaves Reeves and Blau –Use structured particle system to represent realistic trees. Kowalski et al. –Two step process 1.Render shaded primitives that approximate foliage 2.Overlay graftals on rendered image using Salisbury’s line drawing method

03/30/03© 2005 University of Wisconsin Related Work Examples Kowalski’s Graftal Trees: Reeves “particle forest”:Smith’s “cartoon tree”:

03/30/03© 2005 University of Wisconsin Traditional Illustration of Trees The Skeleton is drawn up to the second level of branching. Foliage is broken up into three parts: 1.The directly lit canopy – represented by outlines and scarce inner detail 2.The half shadow – greater detail provides an overall darker grey level 3.The shaded area – achieved by thicker lines or drawing leaves in black Artists will general use a combination of these to draw the leaves. Rarely do they use all three.

03/30/03© 2005 University of Wisconsin Drawing the Skeleton The tree skeleton is represented by generalized cylinders. Analytical Silhouette Algorithms –Markosian et al. –Rakar and Cohen –Depth Differences The skeleton is shaded with crosshatch marks. Marks could be placed using the algorithm outlined by Salisbury et al. –The authors actually use a variant of Floyd-Steinberg diffusion that places strokes when the accumulated grey value is above a threshold.

03/30/03© 2005 University of Wisconsin Drawing the Foliage The xfrog modeling system can generate models with 200,000 leaves on a tree. We want to pair this down to 10,000 or so. Leaves are represented abstractly by particles: a location and an orientation vector. The goal is to illustrate groups of with a set of strokes. Idea: Render particles as primitives and use depth buffer information to determine which leaves are spatially close and outline them.

03/30/03© 2005 University of Wisconsin Depth Differences Initially, each particle is rendered with a view-facing primitive, such as a disk. Basic Algorithm: –For each pixel compute the maximum positive difference between it and its neighbors If the distance is above the given threshold, mark as an edge.

03/30/03© 2005 University of Wisconsin Depth Differences Continued… The threshold and primitive size can be varied to achieve different effects: Primitive size: 0.15Primitive size: 0.70 Threshold: 1000Threshold: 2000 The threshold can be varied using other buffers, achieving greater detail in areas that results in a shadowing effect:

03/30/03© 2005 University of Wisconsin Abstract Drawing Primitives More realistic leaves can be achieved by using the particles’ orientation vectors to interpolate between a set of polygons that represent leaves from different views

03/30/03© 2005 University of Wisconsin Level-of-Abstraction Two Approaches 1.Convert raw z-buffer values from non-linear, projected values to linear values using formula below and vary threshold according to depth: d: depth value between 0 and 1 d 0 : minimal depth value (e.g. 0) d 1 : maximal depth value (e.g ) z 0 : depth value of near clipping plane z 1 : depth value of far clipping plane 2.Take advantage of the non-linearity to automatically generate more abstract representations. –Geometry farther away will be pushed into fewer values in the z- buffer. –A tree in the background will represented by fewer strokes than the same tree in the foreground.

03/30/03© 2005 University of Wisconsin Level of Abstraction Continued… The level-of-abstraction can be furthered by scaling the primitives so that the amount of screen space they occupy is independent of their depth. r: scaling factor d: desired screen space s: actual screen space No Scaling Scaled

03/30/03© 2005 University of Wisconsin Software Implementation 1.Calculate depth differences and mark pixels above threshold. In the interactive mode this bitmap is directly used and blended with other geometry. 2.Generate shadows in software. Large number of isolated surfaces precludes hardware implementation. 3.Vectorize bitmap using least squares fitting or primitive identification method. Draw lines using spline interpolation Apply line styles

03/30/03© 2005 University of Wisconsin Results

03/30/03© 2005 University of Wisconsin Future Work Apply cross-hatch shading to the leaves Create a continuous level-of-detail algorithm that stays performance while getting rid of artifacts that come with discrete representations. Apply new styles and color for cartoons. Questions?

03/30/05© 2005 University of Wisconsin

03/30/05© 2005 University of Wisconsin Sketchy Drawings Marc Niehaus Jürgen Döllner University of Potsdam

03/30/05© 2005 University of Wisconsin Sketchy Drawings Sketchiness shows unfinished state Useful for architectural and product design

03/30/05© 2005 University of Wisconsin Previous Work G-Buffers (Saito & Takahashi 1990) Edge enhancement (Nienhaus and Döllner 2003) Loose and Sketchy filter (Curtis 1998) Artistic Strokes (Northrup & Markosian 2000)

03/30/05© 2005 University of Wisconsin Algorithm Overview Obtain edge and shading data Perturb this data to get sketchy look Problem: It needs to be done in real time

03/30/05© 2005 University of Wisconsin G-Buffers Need to access in real time Depth/Color/Alpha/Stencil Buffer on graphics card Textures Current hardware allows multiple textures Manipulate with fragment and pixel shaders

03/30/05© 2005 University of Wisconsin Edge Map Need to find edges for most NPR algorithms Store three types of edges –Silhouette –Crease –Border See Edge Enhancement (Nienhaus & Döllner 2003 ) for more details

03/30/05© 2005 University of Wisconsin Shade Map Render unlit 3D geometry to capture flat colors Store results in screen aligned texture instead of color buffer

03/30/05© 2005 University of Wisconsin Adding Uncertainty Apply uncertainty function to edge and shade maps Store function in texture Use Perlin function to get frame to frame consistency Multiply uncertainty values with 2x2 matrix Use different matrix for edge and shade maps to get a more sketchy feel

03/30/05© 2005 University of Wisconsin End Result Use shaders to combine results into texture Render texture on screen-aligned quad

03/30/05© 2005 University of Wisconsin Style Variations Render edges multiple times Roughen edge and surface colors

03/30/05© 2005 University of Wisconsin Issues Depth Information Shower-Door Effect

03/30/05© 2005 University of Wisconsin Updating Depth Info Depth info is lost when rendering 3D geometry to 2D textures Solution: Render geometry into a depth map Render perturbed depth sprite –use shaders to update depth buffer Need to account for both edge and shade map –take min value and use this for depth

03/30/05© 2005 University of Wisconsin Shower-Door Effect When model moves, edges tend to swim around Need to account for geometrical data Also want to keep uncertainty past geometry edges Solution: Render geometry info into another G-Buffer Use geometry as a parameter to the noise function

03/30/05© 2005 University of Wisconsin Process Summary

03/30/05© 2005 University of Wisconsin Questions?

03/30/05© 2005 University of Wisconsin