Non-Photorealistic Rendering CS 446: Real-Time Rendering & Game Technology David Luebke University of Virginia

2 David Luebke Real-Time Rendering Demo Today: Sean Arietta, Super Mario Sunshine Thursday: Greg Tylka

3 David Luebke Real-Time Rendering Assignments Assn 4 description due tonight at midnight –Remember to post these on forum, not send by –Assn 4 due Mar 31, integrated demo Mar 23

4 David Luebke Real-Time Rendering Non-Photorealistic Rendering “Using a term like ‘nonlinear science’ is like referring to the bulk of zoology as ‘the study of nonelephant animals’” Photorealism: age-old goal of graphics Non-photorealistic rendering –Broadly, techniques for rendering that don’t strive for realism, but style, expressiveness, abstraction, uncertainty, etc –A terrible term that we may be stuck with Better terms: stylized rendering, artistic rendering, abstract rendering, comprehensible rendering, etc.

5 David Luebke Real-Time Rendering Stylized Rendering NPR usually used to refer to graphics techniques that emulate a particular artistic style or medium –Impressionistic painting –Pen-and-ink (cross hatching, outlining, etc) –Colored-pencil, copperplate engraving, you name it – Cartoon drawing

6 David Luebke Real-Time Rendering Toon shading Toon shading is the term that we use to refer to cartoon-like rendering effects Tendencies in cartoon/comic style: –Simple, flat shading (cel shading) Two-tone (light/shadow) or three-tone (light/shadow/highlight) –Edge highlighting Boundary (border edge) Crease (hard edge) Material edge Silhouette edge

7 David Luebke Real-Time Rendering Toon Shading Cel shading –Threshold the lighting equation Ex: the Cg program we looked at Diffuse below 0.5 ? dark color : light color Specular above 0.5 ? hilight color : diffuse color If no pixel shaders, can use N ● L to look up a 1-D texture containing the shaded colors –In general, can think of these procedures as remapping the traditional lighting equation elements ( tone mapping )

8 David Luebke Real-Time Rendering Silhouette Rendering Boundary (border edge) –Not shared by two polygons –E.g. the edge of a sheet of paper –Solid models usually have no boundaries Crease (hard edge) –Shared by two polygons at a dihedral angle greater than some threshold (often 60°) –Or a vertex with two normals/two colocated vertices – Ridge or valley edges

9 David Luebke Real-Time Rendering Silhouette Rendering Material edge –Triangles sharing edge have different materials/texture maps/etc –Or just an edge that the artist wants to emphasize Silhouette edge –Triangles sharing edge face different directions (towards/away from viewer) –Lots of techniques to find at runtime!

10 David Luebke Real-Time Rendering Finding/Rendering Silhouette Edges Surface angle silhouetting –Again, like the Cg program we looked at –Calculate N●V, if below threshold  draw black –Problem: depends on surface curvature (see Fig 7.5)

11 David Luebke Real-Time Rendering Procedural Geometry Silhouetting Idea: render the geometry in such a way that the silhouettes “fall out” Ex: –Draw frontfacing polygons –Draw backfacing polygons But draw them in thick (2-pixel) wireframe Or draw them in wireframe, z-biased forward a bit Or “fatten” them Or displace them along their normals (“halo” effect)

12 David Luebke Real-Time Rendering Results WireframeTranslationFattening 40 fps 50 fps 11.5 fps Original Venus model : 5672 triangles, 66 fps

13 David Luebke Real-Time Rendering Image Processing Silhouetting Idea: analyze the image after it’s rendered, and extract silhouettes (like edge detection) Can help by rendering e.g. depth image, object-ID image, normal image Perfect for fragment program!

14 David Luebke Real-Time Rendering Silhouette Edge Detection Idea: find silhouette edges geometrically and render them explicitly –Hierarchy of normal cones –Lots of other interesting geometric approaches Most work, but gives the most flexibility in how silhouettes are drawn

15 David Luebke Real-Time Rendering Recap: Edge Highlighting Toon shading (and other NPR techniques based on drawing) requires drawing/highlighting some edges: –Silhouette edges –Mesh boundaries (always on silhouette) –Creases (ridge and valley) –Material boundaries Find first at run-time, precalculate the others (unless object is deformable)

16 David Luebke Real-Time Rendering Recap: Silhouette Edges Surface angle silhouetting –Calc N●V, if below threshold  draw black Best as a per-pixel routine The Cg program we looked at Options: spheremap, or use a mip-map with top-level textures dark –Pros: Uses the texture hardware  fast Can antialias the resulting lines –Cons: Line width depends on curvature Doesn’t work for some models (e.g., a cube)

Recap: Silhouette Edges Procedural Geometry Silhouetting –Idea: render geometry such that the silhouettes “fall out”, e.g.: Draw frontfacing polygons Draw backfacing polygons –But draw them in (possibly thick) wireframe –Or draw them z-biased forward a bit –Or “fatten” them –Or displace them along their normals (“halo” effect) »Flip normals »Amount of displacement varies w/ distance ( why? ) »Perfect task for vertex shader! –Pros: relatively robust, doesn’t need connectivity info –Cons: Wastes some fill & some polys, needs antialiasing

Recap: Silhouette Edges Silhouette Edge Detection –Idea: find silhouette edges geometrically on the CPU and render them explicitly Brute force: test every edge to see if its adjoining polygons face opposite directions in eye space –Can speed this up with hierarchy or randomized coherent search Most work, but gives most flexibility in how silhouettes drawn –GPU variant: Draw degenerate quadrilateral at each edge “Fatten” quad into a “fin” when edge is on silhouette –Fin thickness based on distance to eyepoint –Vertex shader & replicated vertices today  geometry shader!

19 David Luebke Real-Time Rendering Highlighting Ridge Edges Clever related technique by Raskar: –Add “fins” to every edge at dihedral angle –Size fins according to distance to viewer –Again, perfect for vertex shader, geometry shader Similar but more complicated technique for highlighting valley edges

20 David Luebke Real-Time Rendering Drawing Lines: Outlining Polygons Surprisingly hard to draw polys as filled outlines –Problem: depth buffer values of edge & polys same –2-pass technique: draw polys, then draw edges Z-bias edges forward or polygons back ( glPolygonOffset ) Works okay, but has occasional problems –3-pass technique: Render filled polygon –Disable depth buffer writes (leave depth test on) –Enable color buffer writes Render polygon edges polygon –Normal depth & color buffering Render filled polygon again –Enable depth buffer writes –Disable color buffer writes

21 David Luebke Real-Time Rendering Drawing Lines: Hidden-Line Rendering Hidden-line vs. obscured line vs halos –Hidden-line Draw polygons to depth buffer (not color buffer) Draw edges using previous technique –Obscured (light, dotted, dashed) line Draw all edges in obscured style Draw polygons to depth buffer (not color buffer) Draw edges using previous technique –Haloed line Draw all edges as thick background-color lines Draw edges using biasing, foreground-color

22 David Luebke Real-Time Rendering Other Styles Impressionistic or “painterly” rendering: –Sprinkle particles on object surface –Draw particles as brushstrokes –Can render images to encode normals, surface curvature, depth, color/tone info

23 David Luebke Real-Time Rendering Painterly Rendering More info if time permits…

24 David Luebke Real-Time Rendering Other Styles Hatching: –Store different cross-hatch patterns representing different tones as textures –Clever ways to use texture hardware to blend between tones at run-time More info if time permits…

Other Styles “Graftals” are a general term used for strokes, decals, little bits of geometry Dynamic placement of graftals to achieve certain effects/styles:

26 David Luebke Real-Time Rendering Other Styles Technical Illustration

Generalizing NPR Recent work by researchers in NPR attempts to generalize many of these concepts: Coherent Stylized Silhouettes Kalnins, Davidson, Markosian, Finkelstein

Generalizing NPR Recent work by researchers in NPR attempts to generalize many of these concepts: Suggestive Contours DeCarlo, Finkelstein, Reinkiewicz, Santella

Gallery of NPR images Technical Illustration Coherent Stylized Silhouettes Suggestive Contours SIGGRAPH papers… NPAR