2

Many picture books and graphic novels with dynamic and interesting styles For commercial reasons, publishers create derivative digital products including animation and games When digitising these books for animation or games backgrounds and scenes are just static images, which appear instantaneously or through fading The entertainment industry is asking for alternatives

Commercial opportunity as Case Study: Bo children's book series Bo = transparent piglet aimed at ages 4-6 teach children how to deal with emotions multiple traditional painting techniques Recreate scenes from the book the way that the artist made them unique recreation of the visual style to visualise the progression of the story/game (as the player continues, more and more is revealed)

Tools to allow artists to quickly and intuitively create and animate hand-painted scenes capturing the way an artist creates his picture books allowing traditional hand-painted strokes to be generated digitally Real-time animated recreation of hand-painted scenes running on mobile devices animated recreation of captured scenes (e.g., stroke by stroke) optimisation for mobile devices Integration in existing (game) engine Unity3D

6

Physical simulation

Non-photorealistic Rendering (NPR)

Raster graphics

Trade-off Accuracy indistinguishable from reality Performance speed of recreating captured data Animatability ability to animate stroke by stroke drawing from start to finish Changeability flexibility when designing (e.g., moving drawn strokes) Artistic freedom

Immediate Mode Rendering (IMR) < PC graphics industry forward facing polygon -> rendered -> shaded -> textured -> Z-buffer -> frame buffer (most) android devices Tile-Based Deferred Rendering (TBDR) < PowerVR architecture tile based (parallel) + deferred rendering (hidden surface removal) all Apple mobile devices

Original plan Raster graphics and TBDR Bad benchmarking test: ARM + PowerVR architecture write-to-texture operations not efficient for pixel-based strokes Mesh-based Strokes Combination of NPR and raster graphics Creation: strokes are flat meshes approximating the contour of brush strokes Colouring using textures: GPU shader program Allows for fast rendering of meshes on the GPU

14

Main characteristics Rubber paint roller for background elements Paper cut stencils for adding details Scanned paintings for animals and insects Blending for the final composition

Sampling of input points (< digital tablet) Pressure determines width Boundaries determine geometry of brush footprint Fading out of paint Use uv-coordinates as a length-parameter to determine start and end of the stroke as a shader has no context

General fading out supported by base stroke Background consist of transparant, slowly fading strokes new parameters: fade offset + fade multiplier Unique strokes create wide texture using two dollops of paint for each stroke randomly select narrow band from texture

Pattern of white dots when rolling back and forth in a fanning pattern due to the relief created by the paint itself as thin layers of dried paint on the surface of the roller prevent paint in between from touching the paper Employ a black and white mask white representing spots where the paint should adhere, black where it doesn’t add mask as texture and multiply

Create effect of individual blades of grass artist: achieved by ripping of a corner of paper, dipping it in paint and dragging it over the canvas tool: mask with transparent inside and grass-like outside is stretched over the length of the stroke

Using masks and additional parameters Oily brush Stencil roller brush Stamp brushes

21

Management of brushes, preview canvas…

Painting, timeline, recording functionality…

24

60 strokes 4500 vertices 2400 triangles

150 strokes 3600 vertices 2300 triangles

iPad2 Initially frame rate dropped from 60 fps to 5 fps due to overdraw of transparent strokes: cover significant amount of the screen and, hence, there is much overlap

Render-to-texture (RTT) render to image first i.o. rendering to screen drawback: background is flattened RTT for strokes that have been drawn completely and need not to be animated anymore do this for every new strokes Consistent frame rate of 60 fps