Automatic Storytelling in Comics

Intrinsic Needs of Human Being! Communication Storytelling Today’s world is information-saturated Computers are becoming a larger and larger part of people’s lives (Intel 2005) More and more people are documenting their lives with digital images, video, music, and recordings (Imation 2003) As storage space gets cheaper, and people record more and more and more, they need some way to automate the categorization of all of these digital memories

Recall, share and preserve (gaming) experience Virtual community such as forums, blogs Video, screenshots, comics or other art formats Narrative, summary, story Search “Lineage Story” or “Lineage Comics”… Around 26,600 files in YouTube and 724,000 items in Google http://www.lineage2.com/community/fanart.html http://himewikia.blogspot.com/ …

Example:

Example:

Narrative In short, People Like Stories. Or, put more accurately, People Remember Stories So we provide a summarization tool to aid storytelling of end-users But why Comics…

Comic A storytelling medium Vocabulary of comics Visual language Panel layout Border shape Panel size Bleed, splash page Speech balloon and sound effects Motion lines Visual language Draw attention to important events Reference: http://scottmccloud.com/2-print/index.html (McCloud, Scott)

Comic Generation Tools Automation support The computer is charged with a greater role in the creation of the comic Authoring support Tools offer the most agency in creating panels and layouts of any size and shape Reference: http://dm.lcc.gatech.edu/~jalderman/comics/

Comic Chat (1996) conversations as sequences of comic panels. Comic Chat visually represents conversations as sequences of comic panels. Automation support: Changing posture and facial expression of users’ avatars in chat room Placing word balloons in natural reading order Zooming “camera” to frame the people talking Parsing text into panels procedurally. Panels are all in the same size Reference: http://kurlander.net/DJ/Projects/ComicChat/resources.html

Microsoft Chat 2.5 Authoring support: Chat room: Comic mode Emotion Wheel allows users to choose the facial expression and body language Massage

Microsoft Chat Character Editor Character Icon Character Contents New Pose

Example:

Video Manga (1999-2003) It provides a summary of video clips that allows users to quickly browse a long video in comic book format. Manga is named after a Japanese word for "comic book" A video is represented with different-sized keyframes that packed in a comic book format Video Manga allows users to quickly browse a long video The visual summaries support printing function Reference: http://www.fxpal.com/?p=manga

Automation support: Authoring support: A video is segmented based on the color features of each frame The segments are clustered according to their similarities We have introduced an importance score to rank the segments A segment is considered to be important if it is long and rare The keyframes are extracted from highly ranked segments The keyframes are sized according to their scores The frame-packing algorithm puts the keyframes in a comic book format Authoring support: Handling captions to its frames manually typed in by users

Video Manga : Summary of staff meeting

Generating Comics From 3D Interactive Computer Graphics (2006) Automation support: Recognizing scenes Converting to visual depiction Comic layout Image processing Reference: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1626186

Automation support Recognizing scenes Converting to visual depiction Time and space Separating scenes when The interaction level is low The location changes Converting to visual depiction Transforming each scene into a sequence of images depicting the main event Using idioms to depict interactions Change of scene Shooting conversation

Comic layout Image processing B (Big): peak interaction images and change-of-scene S (small): action-to-action pair or FP viewpoint shots F (fixed): images with speech balloon N (neutral): all others Image processing Mean shift Laplacian K-means

Example:

Comic Life (2005) Application for creating comics: Authoring support: loading images into comic panels Authoring support: Simple and Easy-to-use interface Drag in your pictures, captions, sound effects (‘ka-blam!') and speech balloons and it’s done! Layout templates Filters like hand-drawn, painted and night vision More freedom to create comic but still some restrictions Reference: http://plasq.com/comiclife-win

Comic Life 1.3

Example:

Manga Studio Authoring support: Application for creating comics: drawing, laying out comic and styling comics Authoring support: High versatility and flexibility to create comics A large collection of special effects Sketch, ink, and color artworks without the need for any other graphics applications Interface similar to Photoshop Reference: http://my.smithmicro.com/mac/manga/index.html

Our Proposal Automation support Authoring support User interface Frame selection Layout computing Authoring support Comic layout Graphic Effects Internet support User interface

Comic Gene (2009) Automation support: Authoring support: It automatically summarize players’ interactions (logs and screenshots) in virtual world into comics. Automation support: Recording log and shots (Addon) Frame selection Comic layout computing Rendering comics Authoring support: Editing log files and screenshots Comic layout Option setting Changing size and position of the shots Reference: http://mmnet.iis.sinica.edu.tw/publication_detail.html?key=chan09_comics

Prototype …How to improve!

1. Automation Support Automatically summarize logs and frames into comics Log: events Frame: video, photos, screenshots and so on Definition of important events Frequency, weight Significance score

Frame selection Layout computing How many images does a comic strip need? Is the criterion of significance score enough? Layout computing Algorithm Detection

2. Authoring Support Page properties Important event/frames How many images in a comic/page/row/… ? Important event/frames Comic layout Panel layout Border shape, bleed Panel size Speech balloon Sound effects

Graphic effects Internet support Templates Cartoon-like design Cartoon-like filter Motion line Internet support Uploading comics to individual album or blog

5. User Interface Design User-friendly Features Navigation bar Easy-to-use Click, wheel Drag-n-drop

UI Example: Videolyzer Navigation bar

UI Example: Comic Life Drag-n-drop

UI Proposal: Comic Gene Drag-n-drop

Thank you