Stylization and Abstraction of Photographs Doug Decarlo and Anthony Santella

Outline Introduction Goals / Motivation Image Structure and Analysis Hierarchical Image Representation Results Video Summary Conclusion

Introduction The human visual system is very powerful It masks the complex perceptual and cognitive processing that is needed to understand images Good information design makes it easer for us to understand images –Keep detail light in unimportant regions –Have fine detail only in important areas

Example Invert these heuristics and…

Goals Paper goal: Present a method for stylizing and abstracting photographs to clarify the meaningful information in them with no artistic ability required

Motivation and Contributions Artists have known about abstraction for years Henri de Toulouse-LautrecMoulin Rouge-La Goulue

Motivation and Contributions (Cont) Many different heuristics have been created for emphasizing certain parts of images But it is very hard to automatically detect meaningful elements in a photograph Paper’s solution: Use eye movements to help detect the important areas of a photograph

Algorithm Summary To transform an image: –Instruct a user to look at the image for a short time, obtaining a record of eye movements. –Disassemble the image into its constituents of visual form using visual analysis (image segmentation and edge detection) –Render the image, preserving the form predicted to be meaningful by applying a model of human visual perception to the eye-movement data

Image Structure and Analysis Edge Detection: The process of extracting out location of high contrast in an image that are likely to form the boundary of objects

Image Structure and Analysis Image Segmentation: Partitioning an image into contiguous regions of pixels that have similar appearance.

Image Structure and Analysis For image segmentation, colors were represented in L*u*v space, to produce region boundaries that were more meaningful for human observers

Image Structure and Analysis Scale-space Theory: Provides a description of images in terms of how content across different resolutions is related This theory serves as the basis for their hierarchical representation of the image It uses segmentation algorithms applied at a variety of scales, and finds containment relationships between their results

Visual Perception Eye movements give a strong indication of the important elements in an image People can examine only a small area of an image at a time, and therefore scan them in a series of “fixations”

Eye tracker Use an eye tracking device to figure out the important areas of a photograph

Hierarchical Image Representation Create an image pyramid of the input image –The bottom image is the original image –Each layer up is a down- sampled by a constant factor –A segmentation algorithm is computed at each later Edges are detected using the original image, a process called Edge Tracking is used to smooth them

Building the Hierarchy Construct a tree, a leaf is created for each segmentation in the tree’s bottom image

Rendering with a Perceptual Model To create the line drawings, the hierarchal tree that was created is “pruned” of leaf nodes in areas where it is determined the user didn’t see Pruning is done based on the fixations that were recorded and perceptual cues such contrast sensitivity

Region Smoothing The detail level of boundaries is uniformly high, since all boundaries derive from the lowest segmentation

Drawing Lines After the segments have been drawn, we draw the lines When drawing the lines, we ignore lines that were not important (using factors like how close they were to a fixation, how long they are, etc)

Results

Video Video time…

Future Work Have the segmenter use a model of shading Find new ways to represent textures

Conclusion This paper presented a new visual style using bold edges and constant color and a method of interaction using eye tracking that helps find important areas of a photograph for image rendering

Questions? ???