Rendering Complexity in Computer-Generated Pen- and-Ink Illustrations Brett Wilson & Kwan-Liu Ma The University of California, Davis

How would an artist treat this scene? Ambiguous boundaries Ambiguous depth

Method 1: Abstraction Merge similar regions Strokes don’t follow geometry exactly Good color / texture

Method 2: Separation Separate similar regions Geometry is clear Color is not as true

How can a computer make these decisions? Introduction to NPR pipelines Hybrid 2D / 3D pipeline Abstraction: when and how? –Silhouette rendering –Hatching

Image-based NPR Good abstraction Low detail — always lose information Image

Geometry-based NPR Poor abstraction High detail — gain information GeometryImage

Geometry rendering techniques Hierarchical textures [Winkenbach & Salesin 1994] Arbitrary meshes [Girshick et al. 2000] Smoothed direction fields [Hertzmann & Zorin 2000]

Neither of these techniques works well for complex scenes. 2D approach gives too little detail, no relative importance 3D approach gives too much detail, hard to pick out important things Challenge: Intelligent use of abstraction

2 + D NPR processing Hamel & Strothotte: Capturing and Re-Using Rendering Styles for NPR [EG ’99] Generate multiple renderings Match image attributes to example input Discard geometry

Tree rendering. Deussen & Strothotte: Computer-Generated Pen- and-Ink Illustrations of Trees [SIGGRAPH 2000] Generate 2D depth renderings to extract important feature lines of the foliage. Requires complex areas (leaves) to be tagged.

A generalized hybrid pipeline. Add rendering and segmentation to the middle of the pipeline.

Silhouette rendering Generate a complexity map Indicates regions of high geometric complexity Simplify areas likely to be confusing

A complexity map generated from an edge rendering. Silhouette renderingComplexity map Many other ways to measure complexity

Silhouette image should match a grayscale rendering. EdgesTarget Too light Too dark

Resulting edge rendering Use Deussen’s technique to keep edges in order of importance Add occluded edges for darkening

Grayscale rendering with hatching. Artists don’t draw every object with separate strokes Small, similar objects grouped and use the same strokes Apply based on complexity

Segmentation for hatching Use segmentation to identify groups of strokes. –Depth –Angle –Color –Texture –…etc.

Notes on segmentation. Much easier than general image segmentation No image understanding necessary Simple segmentation is acceptable –Region growing

Segmentation-based hatching with important silhouette lines.

Primate Chest Isosurface 3.5 M triangles High detail Ambiguous area

Silhouette edges

Fully hatched rendering

The rendering is separated into complex and non-complex regions. SimpleComplex

Hybrid pen/paint rendering Hatching for non-complex areas Solid black shading for complex areas Preserves feel while simplifying rendering

Close-up comparison of hybrid rendering

Sharp boundaries Blur operation affects boundaries “Knock out” large objects Future work: Clustering in Z?

Conclusion Abstraction –When –How How will the viewer perceive the scene? –Incorporate segmentation in 3D pipeline Clearer, more artistically believable pictures

Future work Better models Higher-quality hatching More rendering styles in general More possibilities with segmentation

Thank You Funded by the U.S. National Science Foundation under –ACI (PECASE award) –ACI (ITR) –ACI