1. Data-Driven Handwriting Synthesis in a Conjoined Manner Hsin-I Chen, Tse-Ju Lin, Xiao-Feng Jian, I-Chao Shen, Bing-Yu Chen National Taiwan University, University of British Columbia 1 1 1 2 1 1 2

2. Handwriting Font is Popular Personal Website Invitation letter Personal Decoration

3. Handwriting Synthesis “The grown-ups are certainly very, very old”, he said to himself, as he continued on his journey. “The grown-ups are certainly very, very old”, he said to himself, as he continued on his journey Segoe print Lucida Handwriting

4. Different shape of the same character he little prince he tippler

5. Previous work - Handwriting Style From forensic science viewpoint, Important factors of handwriting style: Elements of styles Elements of execution Natural variance between each writing Other effects Alcohol, emotion, etc.

6. Previous work - Handwriting Synthesis Handwriting Synthesized HandwritingSynthesized Handwriting Synthesized Wang et al., IJDAR 2005Chang and Shin, IJDAR 2012 Lin and Wang, PR 2007

7. Contribution A statistical learning approach to synthesize non- existent paragraph: A data-analysis stage A character grouping method A data-driven optimization framework

8. Approach pipeline Data CollectionParameterizationShape Model Paragraph Synthesis Word Synthesis Character Synthesis

9. Approach pipeline Data CollectionParameterization Shape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

10. Data Collection  We should collect at least two instances for each letter.  We should cover more commonly used letter pairs(“aa”, “ab”, “ac”,…).  The collection sheet should not over be constrained. 10

11. Data Collection

12. Approach pipeline – Parameterization Data collectionParameterizationShape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

13. Character Parameterization HandwrittenReconstructed Control points

14. Approach pipeline – Shape Model Data collectionParameterizationShape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

15. Shape Model Shape coefficientDisplacement

16. Approach pipeline – Character Synthesis Data collectionParameterizationShape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

17. Collected dataSynthesized data

18. Approach pipeline – Word Synthesis Data collectionParameterizationShape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

19. Character Grouping  We group letters to obtain more conjoining information  Example: Criteria Synthesis target: Data set: How to connect ‘r’ and ‘i’ ?

20. Character Grouping  The probability we connect two neighboring letters Ending group Starting group

21. Word Synthesis Smoothness term Data term Boundary constraint

22. Word Synthesis Synthesized Result Connected

23. Approach pipeline – Paragraph synthesis Data collectionParameterizationShape Model Paragraph Synthesis Word SynthesisCharacter Synthesis

24. Paragraph Synthesis  Line angle, word height and word angle

25. User Study : Visual discrimination

27. 65 Subjects

28. User Study : Similarity test Handwritten sample Our result

29. User Study : Similarity test Handwritten paragraph Synthesized paragraph without layout info Synthesized paragraph with layout info

30. Visual Result Handwritten paragraph Our synthesized paragraph

31. Visual Result Style 1 Style 2 Style 3

32. Comparison II Handwritten samples Synthesized result by Lin et al., PR2007 Handwritten samples Our result

33. Comparison I Original writingWang et al., IJDAR 2005Ours

34. Conclusion We present a data-driven optimization approach to synthesize non-existed paragraph : We analyzing cursiveness property in the data. A novel trajectory optimization for synthesizing conjoining character. The user study and comparison results show that our approach successfully imitate one’s handwriting style.

35. Q & A Thank you http://graphics.csie.ntu.edu.tw/~fensi/