Painterly Rendering with Curved Brush Strokes of Multiple Sizes Aaron Hertzman, NYU http://mrl.nyu.edu/publications/painterly98/ (SIGGRAPH ’98) Presented by Marc Flury 11/28/2018

What is Painterly Rendering? An oxymoron? It’s a method of reproducing the artistic style and/or expression of a painting using a source image. An image space technique Sort of a glorified Photoshop™ filter 11/28/2018

An Example 11/28/2018

An Example 11/28/2018

In This Presentation… The rationale of painterly rendering using “curved brush strokes of multiple styles” Overview of implementation Examples of applying different parameters to achieve different styles 11/28/2018

Rationale Real painting requires Typical filtering methods Time Skill Artistic talent Typical filtering methods Use only one size brush stroke Can’t refine style with multiple passes Only support one style Image is “flattened” 11/28/2018

Typical Filtering - Photoshop 11/28/2018

More detail needs smaller strokes 11/28/2018

Improved Quality and Detail 11/28/2018

Advantages Much faster than painting Can be used for video (or even interactive rendering) Multiple brush sizes allow for varying detail and continuous color regions Multi-pass method is somewhat analogous to true artistic techniques (better results?) The system can be parameterized to create different “styles” 11/28/2018

General Painting Algorithm function paint (sourceImage, R1 … Rn) { canvas := a new constant color image // paint the canvas for each brush radius Ri, from largest to smallest do // apply Gaussian blur referenceImage = sourceImage * G(fσRi) // paint a layer paintLayer(canvas, referenceImage, Ri) } return canvas List of brush radii Original picture Approximation of image we want to paint Gaussian kernel – based on brush size Locates areas of image that differ and paints with brush -explained later 11/28/2018

Gauss reference image Gauss reference image source image Gauss Paint canvas Paint canvas Paint canvas 11/28/2018

General Painting Algorithm function paint (sourceImage, R1 … Rn) { canvas := a new constant color image // paint the canvas for each brush radius Ri, from largest to smallest do // apply Gaussian blur referenceImage = sourceImage * G(fσRi) // paint a layer paintLayer(canvas, referenceImage, Ri) } return canvas 11/28/2018

Layer Painting Algorithm function paintLayer(canvas, referenceImage, R) { S := a new set of strokes, initially empty D := difference(canvas, referenceImage) grid := fg R for x=0 to imageWidth stepsize grid do { for y=0 to imageHeight stepsize grid do { M := the region(x-grid/2…x+grid/2, y-grid/2…y+grid/2) areaError := sumOfError(M, D) / grid2 if (areaError > T) then { (x1, y1) := maxPoint(areaError) stroke := makeStroke(R, x1, y1, referenceImage) add stroke to S } } } paint all strokes S on canvas – random order creates a pointwise difference image using: |(r1,g1,b1) – (r2,g2,b2)| = ((r1-r2)2 + (g1-g2)2 + (b1-b2)2)1/2 Sums the error near (x,y) Finds largest error point Calculates stroke to be placed on canvas – explained later Prevents regularity – counter intuitive? 11/28/2018

Curved Brush Strokes Anti-aliased cubic B-splines Each stroke models the color gradient of the reference image Stroke Representation List of control points Color Brush Size 11/28/2018

Curved Stroke Algorithm function makeSplineStroke(x0, y0, R, refImage) { strokeColor = refImage.color(x0, y0) K := new stroke, radius R, color strokeColor add point (x0, y0) to K (x, y) := (x0, y0) (lastDx, lastDy) := (0, 0) for i=1 to maxStrokeLength do { if (i > minStrokeLength and (|refImage.color(x,y) – canvas.color(x,y)| < |refImage.color(x,y)- strokeColor)) then return K if (refImage.gradientMag(x,y) ==0) then return K (gx, gy) := refImage.gradientDirection(x, y) (dx, dy) := (-gy, gx) if (lastDx * dx + lastDy * dy < 0) then (dx, dy) = (-dx, -dy) (dx, dy) := fc * (dx,dy) + (1-fc) * (lastDx,lastDy) (dx, dy) := (dx,dy)/(dx2 + dy2)1/2 (x, y) := (x + R*dx, y + R*dy) (lastDx, lastDy) := (dx, dy) add the point (x, y) to K } return K } Prevents infinite loop Checks that approximation is still good enough Checks that there is a gradient Calculates direction of greatest change (gradient) Calculates normal Reverses direction if necessary Filters brush stroke curvature based on fc 11/28/2018

Finding Control Points θ1 (x2, y2) D1 G2 (x1, y1) D0 θ0 G0 (x0, y0) 11/28/2018

Style Parameters Approximation Threshold (T) Brush Sizes – Smallest (Ri), Number (n), Size Ratio (Ri-1/Ri) Curvature Filter (fc) Blur Factor (fσ) Min and max stroke lengths (minLength, maxLength) Opacity (α) Grid size (fg) Color Jitter (jh, js, jv, jr, jg, jb) 11/28/2018

Examples – Source Image 11/28/2018

Examples – “Impressionism” T = 100 R = (8, 4, 2) fc = 1 fσ = .5 α = 1 fg = 1 minLength = 4 maxLength = 16 11/28/2018

Examples – “Expressionism” T = 50 R = (8, 4, 2) fc = .25 fσ = .5 α = .7 fg = 1 minLength = 10 maxLength = 16 js = .5 11/28/2018

Examples – Water Color T = 200 R = (8, 4, 2) fc = 1 fσ = .5 α = .5 fg = 1 minLength = 4 maxLength = 16 jr = jg = jh = .3 11/28/2018

Examples – “Pointillism” R = (4, 2) fc = 1 fσ = .5 α = 1 fg = .5 minLength = 0 maxLength = 0 jv = jh = .3 11/28/2018

More Examples…. 11/28/2018

More Examples… 11/28/2018

More Examples… 11/28/2018