1. Attention-Directing Composition of Ying Cao Rynson W.H. Lau Elements Look Over Here: Antoni B. Chan City University of Hong Kong

2. Composition of manga elements (i.e., subjects and balloons ) Motivation © Tsugumi Ohba, Takeshi Obata / Shueisha Inc.

3. Manga artist guides viewer’s eyes through the page Motivation © Tsugumi Ohba, Takeshi Obata / Shueisha Inc. Artist’s Guiding Path (AGP) Viewer Attention

4. Motivation One of the most difficult steps in manga production No existing comic creation programs for automatic composition

5. Goal 1.Rabbit, I came here for gold, 2. and I'm gonna get it! 3. I gotcha, you rabbit! I'll show you! Close-up Fast Long Medium Close-up Medium Big Close-up Medium You can't do this to me! Eureka! Gold at last! 1 2 3 4 5 Talk

6. Related Work Balloon Placement [Kurlander et al. 1996] [Chun et al. 2006]

7. Eye-tracking Techniques for Comic Analysis and Creation [Jain et al. 2012][Toyoura et al. 2012] Related Work

8. Computational Manga [Qu et al. 2006] [Qu et al. 2008] [Cao et al. 2012] Related Work

9. Overview Input Storyboard Layout Probabilistic Graphical Model Input Infer Learn Data

10. Probabilistic Graphical Model

11. Data Acquisition 80 manga pages from three manga series Annotation [Motion state] [Shot type] [Panel Shape] [Subject] [Balloon] Eye movements of viewers

12. Probabilistic Graphical Model Abstracts artist’s guiding path as a latent variable Used for composition synthesis Connects artist’s guiding path, composition and viewer attention in a probabilisitc network Artist’s Guiding Path Composition Viewer Attention

13. Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

14. Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

15. Artist’s Guiding Path actual AGP underlying AGP

16. Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

17. Panel Properties and Local Composition Model

18. Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

19. Subject Placement

20. Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

21. Balloon Placement

22. Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

23. Viewer Attention

24. Complete Model

25. Interactive Composition Synthesis

26. Generate a composition, subject to user-specified semantics Layout Generation + Composition Synthesis 1.Rabbit, I came here for gold, 2. and I'm gonna get it! 3. I gotcha, you rabbit! I'll show you! Input: subject & script Close-up Fast shot type & motion state Talk inter-subject constraint

27. Layout Generation … Database of labeled pages [# of panels ] [# of subjects] [# of balloons] [shot type] [motion state]

28. Composition Synthesis Configurations of elements A Maximum A Posteriori inference framework Constraint-based likelihoodConditional prior Input elements & semantics + Layout Constraints

29. Constraint-based likelihood

31. Results and Evaluation

32. Comparison to Heuristic Method for Balloon Placement Our approachHeuristic [Chun et al. 2006]

33. Comparison to Heuristic Method for Balloon Placement Our approachHeuristic [Chun et al. 2006]

34. Comparison to Manual Method User study: How well does our approach facilitate manga composition, as compared to existing manual tool? Participant preference votingTime for one composition

35. Eye-tracking Experiment and Analysis Is our approach effective in directing viewer attention, as compared with the alternatives? To measure fixation consistency across different viewers Similarity metrics:1.Inlier percent 2.Root Mean Squared Distance (RMSD)

36. Eye-tracking Experiment and Analysis 1.Inlier percent Viewer A Saliency Map Viewer B Inliers Classification

37. RMSD, 2.RMSD Viewer A Viewer B Eye-tracking Experiment and Analysis

39. Comparison to Existing Manga Pages OriginalSynthesized © Tsugumi Ohba, Takeshi Obata / Shueisha Inc.

40. Summary A novel model for representing dependency among the artist’s guiding path, composition and viewer attention A novel approach for joint composition of subjects and balloons Enable easy and quick creation of attention-directing compositions

41. Limitations & Future Work Consider visual appearance of manga elements Extend to other graphic design tasks Understand readers’ behaviors

42. Thanks