Ying Cao Antoni B. ChanRynson W.H. Lau City University of Hong Kong
Background Manga layout is crucial for manga production, with unique styles ©AYOYAMA Gosho / Shogakukan Inc. Manga pages Their layouts
Background Effective manga layout can benefit –Storytelling –Attention guidance –Visual attractiveness It is a difficult task
Goal To create high-quality manga layout with ease Resulting layout Semantics Artworks
Challenge Not a well-studied problem Our solution: data-driven strategy to learn stylistic aspects from existing manga pages No explicit rules
Related Work General layout problem: global optimization [ Yu et al. 2011][Merrell et al. 2011]
Related Work Comic layout: heuristic rules or templates [Kurlander et al. 1996] [Shamir et al. 2006] [Preu et al. 2007]
Related Work Computational Manga [Qu et al. 2006] [Qu et al. 2008]
Overview
Manga Database 4,000 scanned manga pages from two manga series Panel annotation Page clustering One manga series 3-panel pages10-panel pages 4-panel pages …
Overview
Style Models Represent stylistic aspects of manga layout Learned from manga examples 3) Panel shape … 2) Panel importance (size) ) Layout structure (i.e., spatial arrangement of panels) …
A probabilistic generative model: Synthesize novel plausible layout structures Layout structure Model
Root ©AYOYAMA Gosho / Shogakukan Inc. Layout structure Model Generative process: recursive spatial division R1R2R3 C1 C2C1 R2R1 C3C2C1
Layout structure Model Parameterization: spatial division instance ©AYOYAMA Gosho / Shogakukan Inc.
Layout structure Model Probabilistic graphical model Parameterization: spatial division instance
Layout structure Model Sample splitting configuration Probabilistic graphical model
Layout structure Model Sample splitting configuration Probabilistic graphical model Sample
Layout structure Model Layout structures sampled from our modelTraining example
Panel clustering Width Heigh t Panel Importance SizeImportance Shape ? A shape-to-importance classifier
Panel Shape Variation Model Captures panel shape variability Active Shape Model [Cootes et al. 1995] … … …
Overview
Semantic Specification Single-panel semantics Inter-panel semantics Image geometry Group of related panels 3 Importance
Overview
Initial Layout Generation A layout structure Maximum a posteriori (MAP) inference Our generative model Existing ones matches resembles
Initial Layout Generation Likelihood term Penalize panel-wise mismatch in aspect ratio & importance Single-panel Likelihood Image geometry panel geometry
Initial Layout Generation Likelihood term Inter-panel Likelihood
Initial Layout Generation Likelihood term Inter-panel Likelihood Measure the smoothness of path through panels
Initial Layout Generation Likelihood term Inter-panel Likelihood Align group boundary with layout boundary
Initial Layout Generation Estimate optimal initial layout Exact MAP inference is computationally expensive … Generative Model Maximum Posteriori
Layout Optimization Unoptimized
Layout Optimization Energy function Collinearity constraint Boundary constraint Regularization term
Layout Optimization
Results (1) (2) (3) (2) (1) (2) (3) (1)
Comparison with existing manga page Input Our result Existing manga page (3) (1) (3) (2) (3) ©AYOYAMA Gosho / Shogakukan Inc.
Layouts of different styles (1) (2) (3) (1) (3) (2) Input Style of Fairy Tail Style of Detective Conan
Layouts of Western comic style
User Study 10 participants: manual tool + our tool 10 Evaluators: pairwise comparison
Summary First attempt to computationally reproduce layout styles of manga A data-driven approach for automatic generation of stylistic manga layout Easy and quick production of professional-looking and stylistically rich manga layouts
Limitations & Future Work Story pacing Art composition & balloon placement Generic framework for other layout problems