1. Painterly Rendering with Curved Brush Strokes of Multiple Sizes
Aaron Hertzman, NYU
(SIGGRAPH ’98)
Presented by Marc Flury
11/28/2018

2. What is Painterly Rendering?
An oxymoron?
It’s a method of reproducing the artistic style and/or expression of a painting using a source image.
An image space technique
Sort of a glorified Photoshop™ filter
11/28/2018

3. An Example
11/28/2018

4. An Example
11/28/2018

5. In This Presentation…
The rationale of painterly rendering using “curved brush strokes of multiple styles”
Overview of implementation
Examples of applying different parameters to achieve different styles
11/28/2018

6. Rationale Real painting requires Typical filtering methods Time Skill
Artistic talent
Typical filtering methods
Use only one size brush stroke
Can’t refine style with multiple passes
Only support one style
Image is “flattened”
11/28/2018

7. Typical Filtering - Photoshop
11/28/2018

8. More detail needs smaller strokes
11/28/2018

9. Improved Quality and Detail
11/28/2018

10. Advantages Much faster than painting
Can be used for video (or even interactive rendering)
Multiple brush sizes allow for varying detail and continuous color regions
Multi-pass method is somewhat analogous to true artistic techniques (better results?)
The system can be parameterized to create different “styles”
11/28/2018

11. General Painting Algorithm
function paint (sourceImage, R1 … Rn) {
canvas := a new constant color image
// paint the canvas
for each brush radius Ri,
from largest to smallest do
// apply Gaussian blur
referenceImage = sourceImage * G(fσRi)
// paint a layer
paintLayer(canvas, referenceImage, Ri) }
return canvas
List of brush radii
Original picture
Approximation of image we want to paint
Gaussian kernel – based on brush size
Locates areas of image that differ and paints with brush -explained later
11/28/2018

12. Gauss reference image Gauss reference image source image Gauss
Paint
canvas
Paint
canvas
Paint
canvas
11/28/2018

13. General Painting Algorithm
function paint (sourceImage, R1 … Rn) {
canvas := a new constant color image
// paint the canvas
for each brush radius Ri,
from largest to smallest do
// apply Gaussian blur
referenceImage = sourceImage * G(fσRi)
// paint a layer
paintLayer(canvas, referenceImage, Ri) }
return canvas
11/28/2018

14. Layer Painting Algorithm
function paintLayer(canvas, referenceImage, R) {
S := a new set of strokes, initially empty
D := difference(canvas, referenceImage)
grid := fg R
for x=0 to imageWidth stepsize grid do {
for y=0 to imageHeight stepsize grid do {
M := the region(x-grid/2…x+grid/2, y-grid/2…y+grid/2)
areaError := sumOfError(M, D) / grid2
if (areaError > T) then {
(x1, y1) := maxPoint(areaError)
stroke := makeStroke(R, x1, y1, referenceImage)
add stroke to S }
} }
paint all strokes S on canvas – random order
creates a pointwise difference image using:
|(r1,g1,b1) – (r2,g2,b2)| =
((r1-r2)2 + (g1-g2)2 + (b1-b2)2)1/2
Sums the error near (x,y)
Finds largest error point
Calculates stroke to be placed on canvas – explained later
Prevents regularity – counter intuitive?
11/28/2018

15. Curved Brush Strokes Anti-aliased cubic B-splines
Each stroke models the color gradient of the reference image
Stroke Representation
List of control points
Color
Brush Size
11/28/2018

16. Curved Stroke Algorithm
function makeSplineStroke(x0, y0, R, refImage) {
strokeColor = refImage.color(x0, y0)
K := new stroke, radius R, color strokeColor
add point (x0, y0) to K
(x, y) := (x0, y0)
(lastDx, lastDy) := (0, 0)
for i=1 to maxStrokeLength do {
if (i > minStrokeLength and (|refImage.color(x,y) – canvas.color(x,y)| < |refImage.color(x,y)- strokeColor)) then return K
if (refImage.gradientMag(x,y) ==0) then return K
(gx, gy) := refImage.gradientDirection(x, y)
(dx, dy) := (-gy, gx)
if (lastDx * dx + lastDy * dy < 0) then
(dx, dy) = (-dx, -dy)
(dx, dy) := fc * (dx,dy) + (1-fc) * (lastDx,lastDy)
(dx, dy) := (dx,dy)/(dx2 + dy2)1/2
(x, y) := (x + R*dx, y + R*dy)
(lastDx, lastDy) := (dx, dy)
add the point (x, y) to K }
return K }
Prevents infinite loop
Checks that approximation is still good enough
Checks that there is a gradient
Calculates direction of greatest change (gradient)
Calculates normal
Reverses direction if necessary
Filters brush stroke curvature based on fc
11/28/2018

17. Finding Control Pointsθ1
(x2, y2) D1 G2
(x1, y1) D0 θ0 G0
(x0, y0)
11/28/2018

18. Style Parameters Approximation Threshold (T)
Brush Sizes – Smallest (Ri), Number (n), Size Ratio (Ri-1/Ri)
Curvature Filter (fc)
Blur Factor (fσ)
Min and max stroke lengths (minLength, maxLength)
Opacity (α)
Grid size (fg)
Color Jitter (jh, js, jv, jr, jg, jb)
11/28/2018

19. Examples – Source Image
11/28/2018

20. Examples – “Impressionism”
T = 100
R = (8, 4, 2)
fc = 1
fσ = .5
α = 1
fg = 1
minLength = 4
maxLength = 16
11/28/2018

21. Examples – “Expressionism”
T = 50
R = (8, 4, 2)
fc = .25
fσ = .5
α = .7
fg = 1
minLength = 10
maxLength = 16
js = .5
11/28/2018

22. Examples – Water Color T = 200 R = (8, 4, 2) fc = 1 fσ = .5 α = .5
fg = 1
minLength = 4
maxLength = 16
jr = jg = jh = .3
11/28/2018

23. Examples – “Pointillism”
R = (4, 2)
fc = 1
fσ = .5
α = 1
fg = .5
minLength = 0
maxLength = 0
jv = jh = .3
11/28/2018

24. More Examples….
11/28/2018

25. More Examples…
11/28/2018

26. More Examples…
11/28/2018