Processing Image and Video for An Impressionist Effect Peter Litwinowicz Apple Computer, Inc. Siggraph1997

Introduction To exploit temporal coherence in video clips to design an automatic filter with a hand-drawn animation quality. produce an Impressionist Effect

Introduction Impressionist In 1800’s,Claude Monet created paintings that attempted to “catch the fleeting impression of sunlight on objects. And it was the out-of-doors world he wanted to capture in paint”.

Introduction Goal 1.To produce images that are impressions of an input image sequence. 2.To produce painterly animation from video clips automatically.

Introduction Advantage To use optical flow fields to push brush strokes from frame to frame in the direction of pixel movements. Randomness is used to perturb the brush stroke’s length, color and orientation to enhance the hand-touched look.

The Process Rendering Strokes Brush Stroke Orientation Frame-to-Frame Coherence

Rendering Strokes Stroke Generation Assume that each brush stroke is rendered with an antialiased line centered at (cx,cy),with a given length, a given brush thickness radius,and a given orientation,theta. The brush stroke positioned every two pixels in both the X and Y directions for the image.

Rendering Strokes Stroke Generation The order that stroke are drawn is randomized to help create a hand-touched look. The color for a particular stroke is assigned the bilinearly interpolated color of the original image at (cx,cy)

Rendering Strokes Random Perturbations Adding random variations and perturbations to a stroke helps to create a hand-crafted look. We assign random amounts to length,radius,color and theta. Denotation: We do not generate new values for each frame as this results in animations with too much jitter.

Rendering Strokes Clipping and Rendering In order to preserve detail and silhouettes, strokes are clipped. Method before: Users specify the edge. Method now: To rely on standard image processing techniques to locate edges In our system there is no user interaction to specify edges.

Rendering Strokes No brush stroke clipping brush stroke clipping

Rendering Strokes The Process 1. Standard conversion of r,g,b values to intensity value. 2. Using Gaussian kernel to blur the intensity image. 3. The resulting blurred image is Sobel filtered. Each pixel calculated its gradient(Gx,Gy),and the value of the Sobel filter at any given pixel is: 4. Grow the line! 5. Assign color to the stroke. The color of the stroke is assign the color of the original image at the center of the stroke.

Rendering Strokes Blurred imageSobel filtered image

Rendering Strokes Use no brush stroke clippingUse brush stroke clipping

Rendering Strokes Using Brush Texture Brush strokes may also be rendered with textured brush images that have r,g,b and alpha components. Each component of the color of the brush stroke texture is multiplied by the color assigned to the stroke.

Rendering Strokes Using Brush Texture

Brush Stroke Orientation An artist may not want to have all the strokes drawn in the same direction! The orientation can be approximated automatically by drawing strokes normal to the gradient direction(of the intensity image) The same Gaussian kernel used in the edge finding process was used for smoothing the image for the gradient calculation.

Brush Stroke Orientation A greater kernel width is used for the orientation calculation.

Brush Stroke Orientation Exception When the magnitude of gradient is near zero, we can’t rely on the gradient direction to be useful!! Solution We modify the gradient field so that brush strokes in a region of constant color smoothly interpolate the directions defined at the region’s boundaries.

Brush Stroke Orientation

Frame-to-Frame Coherence Input: a video clip with no a priori information about pixel movement in the scene. Process: 1. To render the first frame,we use the process described in the previous two sections. 2. Calculate the optical flow between the two images We use a gradient-based multi-resolution technique,employing a pyramid of successively low-passed versions of the gradient to help compute the optical flow.

Frame-to-Frame Coherence Use the flow field to move the strokes The optical flow technique we implemented provides subpixel positioning, and this feature is exploited by moving brush strokes to subpixel location. The optical flow vector field is used as a displacement field to move the brush strokes to new locations in a subsequent frame.

Frame-to-Frame Coherence Optical flow field that maps pixels from one frame to another.

Frame-to-Frame Coherence Problem: Not full coverage of the image with rendered brush strokes. Solution: Using Delaunay triangulation

Frame-to-Frame Coherence

There are two lists of brush strokes: 1. Old brush strokes(stroke moved and subsequently kept) 2. New brush strokes(generated in sparse region) If the new strokes sre simply painted behind the old strokes,undesirable effects can occur!

Frame-to-Frame Coherence Uniformly distributing the new strokes among the old ones

Result

報告結束