1. MTA SzTAKI & Veszprém University (Hungary) Guests at INRIA, Sophia Antipolis, 2000 and 2001 Paintbrush Rendering of Images Tamás Szirányi

2. Stochastic Paintbrush Rendering Stochastic relaxation method to generate images (based on a reference image) by simulating a simple painting method, Sharp contours, well-defined segmentation areas, elaborated fine details, A priori models or interactions are not needed.

3. Painted image with raw strokes Visible contours of strokes

4. 4 Stochastic painting method Need for optimization MCMC optimization MRF and PB for segmentation Objectives

5. Result of an impressionist-like process controlled by an edge map (P. Litwinowicz, “Processing Images and Video for An Impressionist Effect”, Computer Graphics, Proc.SIGGRAPH’1997, 1997.)

6. Paintings of A. Dürer and M. Munkácsy, representing careful life-style painting: the painter tries to elaborate the painting without visible effects of the brush-strokes

11. Painting following a real-life picture

12. Painting, following a real-life picture

13. 13 Painting following an artist`s painting

14. 14 Painting following an artist`s painting

15. 15 Painting following an artist`s painting

16. 16 Painting repairing JPEG

17. 17 Painting following an artist`s painting

18. 18

19. Original with edge-map Segmentation

20. Painted (stroke is 37*9) and its edge

21. Painted (stroke is 10*3) and its edge

22. All PB strokes are accepted, then redundants are removed Only good matching is accepted A need for optimal estimation of stroke placement

23. Markov Chain Monte Carlo (MCMC) methods Ergodic Markov-chain (X 1,X 2,…. X n ) Stationary f(X) Sample series (X 1,X 2,…. X n ) generated with target density f(X) Paintbrush strokes: X is the position of a stroke, f(X) is the probability density of distortion error between the stroke and the reference image at position X

24. Metropolis-Hastings algorithm

25. Independent MH

26. Random Walk MH

27. Probability densities of distortion error of the proposed ( Y t ) and accepted (X t+1 =Y t ) strokes versus the original image when generating the strokes for ‘Barbara’ image. First, coarse (20x5), finally, fine (7x2) strokes are generated

28. Probability densities of difference btw distortion errors of the proposed/accepted strokes and the distortion on previous area of the stroke ( Diff(Y t ) ), when generating the strokes for ‘Barbara’ image. First, coarse (20x5), finally, fine (7x2) strokes are generated.

29. The characteristic variable is the Difference of distortion error instead of the error

30. accept/reject rule

31. 31 Narrowing effect of the distribution of the measured conditional probability

32. 32 Constraints for the flexible accept/reject rule

33. Random Stochastic search 35612645 Metropolis Hastings process 26311555 MethodNumber of non- redundant PB strokes # thousand Number of all drawn strokes # Number of all proposed strokes #

34. 34

35. Limiting the number of colors Coupling the neighboring strokes MRF-like segmentation Reference area is greater than the stroke’s area Halftone effects by strokes (modulation) Connection btw MRF segmentation and PB

36. En1: Difference inside the area of a brush-stroke between the original and the proposed stroke En2: Difference between the color of the stroke and the present neighboring pixels at the boundary of the stroke En= En 1 +(1- )En 2 0 < < 1.0

38. 38 En 1 En 2

40. Noisy input Painting ( = 0.6, No.Co. = 30 ) Painting  En < 0 MMD

42. 7*3 PB 13*5 PB 20*6 PB  E 1 < 0, ~MMD  E 1 < 0, ~MMD MRF ~MMD MRF

43. 43 Motion-Field u Gradient and block-matching motion fields Gradient-based Block-matching

44. 44 Examples 1/2 transformed video Edge map of transformed video 1.2. 3.

45. 45 Example 2/2a frame1: keyframe1frame2: painted motion area 1frame3: painted motion area 2 frame4: painted motion area 3frame5: keyframe2 1. and 5. are keyframes

46. 46 Example 2/2b frame1: keyframe1frame2: painted motion area 1frame3: painted motion area 2 frame4: painted motion area 3frame5: keyframe2

47. 47 Comparison OriginalCinepak

48. 48 Comparison u With Cinepak-codec CinepakPB result 23

49. 49 Thank you for the attention

50. 50 Appendix

51. 51 Segmentation by painting, when distortion error may increase

52. 52 Energy calculus in optimization of MRF segmentation Distortion between the original input color and the proposed random value Distortion among the neighboring pixels

53. Independent MH Since the probability of acceptance of Y t depends on X (t), the resulting sample is not independent and identically distributed (iid). X (t) is irreducible and aperiodic (thus ergodic) iff g is almost everywhere positive on the support of f. The above algorithm produces a uniformly ergodic chain if there exists a constant M such that

54. Random Walk MH RW-MH does not enjoy uniform ergodicity properties, but there are necessary and sufficient conditions for geometric ergodicity, e.g. the log- concave case, when