Attention-Directing Composition of Ying Cao Rynson W.H. Lau Elements Look Over Here: Antoni B. Chan City University of Hong Kong.

Slides:

Advertisements

Similar presentations

Three-Step Database Design

Advertisements

Visual Literacy.

Data Mining and the Web Susan Dumais Microsoft Research KDD97 Panel - Aug 17, 1997.

Yansong Feng and Mirella Lapata

Ying Cao Antoni B. ChanRynson W.H. Lau City University of Hong Kong.

1.Accuracy of Agree/Disagree relation classification. 2.Accuracy of user opinion prediction. 1.Task extraction performance on Bing web search log with.

MTP – Stage 1 Sanobar Nishat. Outline  Peculiarities of the mobile visualization context  Different aspects of mobile visualization design  Map-based.

MULTIMEDIA DEVELOPMENT 4.3 : AUTHORING TOOLS. At the end of the lesson, students should be able to: 1. Describe different types of authoring tools Learning.

NUS CS5247 Motion Planning for Camera Movements in Virtual Environments By Dennis Nieuwenhuisen and Mark H. Overmars In Proc. IEEE Int. Conf. on Robotics.

4/15/2017 Using Gaussian Process Regression for Efficient Motion Planning in Environments with Deformable Objects Barbara Frank, Cyrill Stachniss, Nichola.

A Probabilistic Model for Component-Based Shape Synthesis Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, Vladlen Koltun Stanford University.

Computer-Aided Design Chapter 7. Computer-Aided Design (CAD) Use of computer systems to assist in the creation, modification, analysis, and optimization.

Asa MacWilliams Lehrstuhl für Angewandte Softwaretechnik Institut für Informatik Technische Universität München Dec Software.

Connection Constrained 3D Collage Zhe Huang 1, Jiang Wang, Rynson W.H. Lau, Hongbo Fu City University of Hong Kong 1. Author’s

Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin Nov

Transfer Learning for WiFi-based Indoor Localization

Funding Networks Abdullah Sevincer University of Nevada, Reno Department of Computer Science & Engineering.

Introduction to Data-driven Animation Jinxiang Chai Computer Science and Engineering Texas A&M University.

Animating Human Athletes By J.K. Hodgkins and W.L. Wooten Arjun Rihan CS 99K: Digital Actors.

Chapter 2 Web Site Design Principles Principles of Web Design, Third Edition.

© De Montfort University, Principles of Graphics Design Howell Istance School of Computing Technology De Montfort University.

NON-FUNCTIONAL PROPERTIES IN SOFTWARE PRODUCT LINES: A FRAMEWORK FOR DEVELOPING QUALITY-CENTRIC SOFTWARE PRODUCTS May Mahdi Noorian

VIDEO PRODUCTION.

Input: Original intensity image. Target intensity image (i.e. a value sketch). Using Value Images to Adjust Intensity in 3D Renderings and Photographs.

The Eye-Tracking Butterfly: Morphing the SMI REDpt Eye-Tracking Camera into an Interactive Device. James Cunningham & James D. Miles California State University,

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.

Open source administration software for education next generation student system Using the Kuali Student Configurable User Interaction Model & Framework.

© 2011 Autodesk Freely licensed for use by educational institutions. Reuse and changes require a note indicating that content has been modified from the.

1 Shengdong Zhao Department of Computer Science University of Toronto July 9, 2008 earPod: Efficient, Hierarchical, Eyes-free Menu Selection.

The Hidden “C” in GIS – Maps Gone Bad Rod Bassler, GIS Coordinator North Dakota State Water Commission 2004.

Eye Tracking in the Design and Evaluation of Digital Libraries

Chapter 2 Web Site Design Principles

Storyboards Jim Henson and Frank Oz looking at storyboards for The Dark Crystal (1982)

Web Site Design Principles

OPTIMIZATION OF FUNCTIONAL BRAIN ROIS VIA MAXIMIZATION OF CONSISTENCY OF STRUCTURAL CONNECTIVITY PROFILES Dajiang Zhu Computer Science Department The University.

SPONSORED BY Data-driven Segmentation and Labeling of Freehand Sketches Zhe Huang, Hongbo Fu, Rynson W.H. Lau City University of Hong Kong.

Metro Transit-Centric Visualization for City Tour Planning Pio Claudio and Sung-Eui Yoon.

Selected Topics in Software Engineering - Distributed Software Development.

Chapter 2 Web Site Design Principles Principles of Web Design, Third Edition.

A Bootstrapping Method for Building Subjectivity Lexicons for Languages with Scarce Resources Author: Carmen Banea, Rada Mihalcea, Janyce Wiebe Source:

인지구조기반 마이닝 소프트컴퓨팅 연구실 박사 2 학기 박 한 샘 2006 지식기반시스템 응용.

Semantic Wordfication of Document Collections Presenter: Yingyu Wu.

WORLD NAVIGATION Don’t Fall Asleep Through These Topics  Tile Graphs  Points of Visibility  NavMesh  Path Smoothing  Hierarchical Pathfinding.

Eye Tracking In Evaluating The Effectiveness OF Ads Guide : Dr. Andrew T. Duchowski.

Automatic Storytelling in Comics

Look Over Here: Attention-Directing Composition of Manga Elements Ying Cao Rynson W.H. Lau Antoni B. Chan SIGGRAPH

DeepDive Model Dongfang Xu Ph.D student, School of Information, University of Arizona Dec 13, 2015.

Chapter 4: Digital Tools That Support Learning SarahBeth Walker.

Concepts and Realization of a Diagram Editor Generator Based on Hypergraph Transformation Author: Mark Minas Presenter: Song Gu.

HFE 760 Virtual Environments Winter 2000 Jennie J. Gallimore

Lecture 5: 11/5/1435 Computer Animation Lecturer/ Kawther Abas CS- 375 Graphics and Human Computer Interaction.

Screen Readers Cannot See (Ontology Based Semantic Annotation for Visually impaired Web users) Yeliz Yesilada, Simon Harper, Carole Goble and Robert Stevens.

How to Read a Graphic Novel. Graphic Novel or Comic Book? When most people hear the phrase “graphic novel” they think “comic books.” Superheroes like.

Introduction to Animation In animation, a series of images are rapidly changed to create an illusion of movement.

-Subject, Form, Content -Principles and Elements of Design.

Chapter 2 Web Site Design Principles

Visual Information Retrieval

Graphic Organizers.

How fast is a ‘rapid method’?

Compositional Human Pose Regression

Warm-Up What makes reading easy for you?

Composition The placement or arrangement of visual elements or ingredients in a work of art, as distinct from the subject of a work. The term composition.

Ontology Evolution: A Methodological Overview

Yeliz Yesilada Robert Stevens Carole Goble

CSc4730/6730 Scientific Visualization

Information Design and Visualization

Information Visualization (Part 1)

Data Warehousing Data Mining Privacy

Chapter 2 Web Site Design Principles

Occlusion and smoothness probabilities in 3D cluttered scenes

Presentation transcript:

Attention-Directing Composition of Ying Cao Rynson W.H. Lau Elements Look Over Here: Antoni B. Chan City University of Hong Kong

Composition of manga elements (i.e., subjects and balloons ) Motivation © Tsugumi Ohba, Takeshi Obata / Shueisha Inc.

Manga artist guides viewer’s eyes through the page Motivation © Tsugumi Ohba, Takeshi Obata / Shueisha Inc. Artist’s Guiding Path (AGP) Viewer Attention

Motivation One of the most difficult steps in manga production No existing comic creation programs for automatic composition

Goal 1.Rabbit, I came here for gold, 2. and I'm gonna get it! 3. I gotcha, you rabbit! I'll show you! Close-up Fast Long Medium Close-up Medium Big Close-up Medium You can't do this to me! Eureka! Gold at last! Talk

Related Work Balloon Placement [Kurlander et al. 1996] [Chun et al. 2006]

Eye-tracking Techniques for Comic Analysis and Creation [Jain et al. 2012][Toyoura et al. 2012] Related Work

Computational Manga [Qu et al. 2006] [Qu et al. 2008] [Cao et al. 2012] Related Work

Overview Input Storyboard Layout Probabilistic Graphical Model Input Infer Learn Data

Probabilistic Graphical Model

Data Acquisition 80 manga pages from three manga series Annotation [Motion state] [Shot type] [Panel Shape] [Subject] [Balloon] Eye movements of viewers

Probabilistic Graphical Model Abstracts artist’s guiding path as a latent variable Used for composition synthesis Connects artist’s guiding path, composition and viewer attention in a probabilisitc network Artist’s Guiding Path Composition Viewer Attention

Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Artist’s Guiding Path actual AGP underlying AGP

Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Panel Properties and Local Composition Model

Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Subject Placement

Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Balloon Placement

Subject Placement Balloon Placement Viewer Attention (eye-gaze path) Local Composition in Panel Panel Properties Artist’s Guiding Path Probabilistic Graphical Model

Viewer Attention

Complete Model

Interactive Composition Synthesis

Generate a composition, subject to user-specified semantics Layout Generation + Composition Synthesis 1.Rabbit, I came here for gold, 2. and I'm gonna get it! 3. I gotcha, you rabbit! I'll show you! Input: subject & script Close-up Fast shot type & motion state Talk inter-subject constraint

Layout Generation … Database of labeled pages [# of panels ] [# of subjects] [# of balloons] [shot type] [motion state]

Composition Synthesis Configurations of elements A Maximum A Posteriori inference framework Constraint-based likelihoodConditional prior Input elements & semantics + Layout Constraints

Constraint-based likelihood

Results and Evaluation

Comparison to Heuristic Method for Balloon Placement Our approachHeuristic [Chun et al. 2006]

Comparison to Heuristic Method for Balloon Placement Our approachHeuristic [Chun et al. 2006]

Comparison to Manual Method User study: How well does our approach facilitate manga composition, as compared to existing manual tool? Participant preference votingTime for one composition

Eye-tracking Experiment and Analysis Is our approach effective in directing viewer attention, as compared with the alternatives? To measure fixation consistency across different viewers Similarity metrics:1.Inlier percent 2.Root Mean Squared Distance (RMSD)

Eye-tracking Experiment and Analysis 1.Inlier percent Viewer A Saliency Map Viewer B Inliers Classification

RMSD, 2.RMSD Viewer A Viewer B Eye-tracking Experiment and Analysis

Comparison to Existing Manga Pages OriginalSynthesized © Tsugumi Ohba, Takeshi Obata / Shueisha Inc.

Summary A novel model for representing dependency among the artist’s guiding path, composition and viewer attention A novel approach for joint composition of subjects and balloons Enable easy and quick creation of attention-directing compositions

Limitations & Future Work Consider visual appearance of manga elements Extend to other graphic design tasks Understand readers’ behaviors

Thanks