Procedural Character Animation for Interactive Systems (Games) Billy Hamilton

About this Presentation I am currently working on a procedural animation system using Unreal Engine 4, a proprietary game engine used in many high-budget games. This presentation compares the animation tools provided in Unreal with procedural animation techniques published in academic research. Unreal – Industry standard techniques, traditional, practical. Academia – State of the art, experimental, not always practical.

Basics of Character Animation A character is controlled by a skeleton (rig), made up of bones (joints), which animate (rotate) over time. The mesh as a whole may also move.

Traditional Approach to Character Animation animationmethods.files.wordpress.com Store an animation sequence, consisting of a series of key frames. Play back the sequence, interpolating between key frames. https://animationmethods.files.wordpress.com/2012/11/newrunningposes.png

Traditional Approach to Character Animation Produce sequence (key frames) either (a) by hand; or (b) with motion capture. graphics.berkeley.edu

Game Characters Animation for Special requirements: Respond to player input. Respond to unexpected events. Realtime performance. Traditional approach: Store a library of animation sequences, play them in response to player input and other game events.

Procedural Generation of Game Content Use of computer algorithms to create content traditionally created by humans. Classifications: online (i.e., runtime) vs. offline random vs. deterministic

Motivations for Procedural Approach to Character Animation Motivations include: Reduce costs (time, money). Improve performance (in some cases). More realistic animation. Increase range of possible actions.

1. Motion Graphs

Overview of Motion Graphs You have a library of animation clips (motion capture). Play clips to animate a character. When you finish one clip, transition to the start of a new clip. A directed graph describes the valid transitions from one clip to the next.

Transitions on Motion Graphs Cover the transitions between clips by blending the start and end of each clip (interpolating the bone rotations). However, some transitions look bad. The poses are not similar enough. A good motion graph must avoid bad transitions.

AnimationMontage Motion Graphs in Unreal Engine 4 Graph is hand authored. Provide explicit code to choose between multiple possible next clips when a transition point is reached. Designer manually chooses good transitions and avoids bad transitions. Plan out required transitions before recording motion capture data.

Example of an AnimationMontage

Automatic Generation of Motion Graphs This has been a focus of academic research. Approach: (1) Record a long clip of actor doing various motions. (2) Compare all frames to find difference between every possible pair of poses – O(N2), but offline. (3) Add transitions at local minima that meet some minimum criteria for similarity. Gleicher 2008

Navigation on a Generated Motion Graph No hand-written rules. Choose a desired end state, search the graph for a series of animation clips that will get you there. Sometimes results in strange animations. Can be improved by annotating animations with information about when they should be used. Automatic classification of motion is a research area.

Navigation on a Generated Motion Graph Another approach is to select motions that minimize the distance from a desired path. The path can be input by the user or generated by a pathfinding algorithm. Kovar et al. 2002

Issues with Automatic Generation of Motion Graphs Graph may be poorly connected: Loops Dead ends Unreachable motions Still need to produce and store every required transition and variation for your animation system. Might miss required transitions if graph is not planned before motion capture session.

No good transition from A to B. Player Input with Motion Graphs Natural conflict between smooth transitions and quick response to player input. Commercial games prefer to use a poor transition rather than imposing a delay on the player. Return as often as possible to standard “idle” pose. No good transition from A to B.

2. Animation Blending

Overview of Animation Blending Using a library of existing clips limits the range of actions a character can perform. Mix together two (or more) animation clips to create a new animation. Make animation library more versatile without requiring additional motion capture.

Blend Spaces Animation Blending in Unreal Engine 4 One-dimensional blend space. Animation clips are plotted as points on a line. Clips are interpolated between points. walk backward idle walk run

Blend Spaces Animation Blending in Unreal Engine 4 run left- front run run right- front walk run left strafe left idle strafe right run right walk back run back Two-dimensional blend space. run left- back run right- back

Blend Spaces Animation Blending in Unreal Engine 4 forwards motion healthy lateral motion Three-dimensional blend space. injured

Limitations of Blend Spaces The number of required animation clips is high. All animations must be designed to work together.

Style Components Blending Animation Separate an adverb (“sneakily”) from a verb (“walk”). Reapply the captured style to a different animation. Approaches: Signal processing Compare to neutral reference clip Independent Component Analysis Shapiro et al. 2006 Shapiro, A., Cao, Y. and Faloutsos, P., 2006, June. Style components. In Proceedings of Graphics Interface 2006 (pp. 33-39). Canadian Information Processing Society.

Bone-by-Bone Animation Blending Run multiple animations on a bone-by-bone basis. Lower body jumps, upper body shoots a gun. Reduce number of animation clips required by breaking them into parts that can be combined. Limitations: Realistic motion often involves entire body. Cannot do anything that will interfere with body parts that are independently animated.

Pose Graphs in Unreal Engine 4 Graph which controls how animations are blended each frame. (“Blend tree.”) Animation programmer must design graph that meets specific needs of each character. No standard form. in Unreal Engine 4 https://answers.unrealengine.com/storage/temp/18948-screenshot+(5).png

Hierarchical Character Animation Order Controller 1 World offset 2 Idle motion 3 Locomotion 4 Animation 5 Reach 6 Grab 7 Gaze 8 Breathing 9 Constraint 10 Eye saccades 11 Blink 12 Head 13 Face 14 General parameters 15 Override Character Animation Academic animation system similar to a blend tree. Series of controllers in a strict order of precedence. Higher- numbered controllers override earlier ones. Difficult to define a universal hierarchy – need exceptions. Shapiro 2011

3. Physics-Based Animation

Overview of Physics-Based Animation The previous techniques are example-based. Physics-based techniques generate new motion without using existing clips – in theory. Common for passive objects such as rag dolls, but still rare for living creatures in commercial games. Rag doll from Dark Souls.

Challenges and Advantages of Physics-Based Animation Challenges All motion must be applied through forces and torques – more difficult than poses. Must simulate biomechanics of the body. Hard to control the style of motion. Advantages Inherently realistic. Not limited by database. Can simulate motion that is impossible in motion capture (dangerous stunts, imaginary animals).

Learning Approaches to Physics-Based Animation Recently impressive results have been achieved using machine learning. Some approaches use motion capture data to train the animation system. See Liu et al. 2015 for an example which combines a physics-based animation approach with a motion graph approach. www.youtube.com/watch?v=QJbCfhRkcyg Liu et al. 2015

Summary of Procedural Character Animation for Computer Games Approach Unreal Engine 4 Academic Research Motion graphs Hand-authored Automatic generation Motion blending Blend spaces Algorithms to separate components of motion Blend trees Complex and hand-authored Physics-based animation Mainly passive objects Biomechanical models, locomotion systems.

Sources Amaya, K., Bruderlin, A. and Calvert, T., 1996, May. Emotion from motion. In Graphics interface (Vol. 96, pp. 222-229). Bruderlin, A. and Williams, L., 1995, September. Motion signal processing. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques (pp. 97-104). ACM. Champland, A., 2012. Procedural Characters and the Coming Animation Technology Revolution. AiGameDev.com. Demofox.com, 2012, September. Anatomy of a Skeletal Animation System. Geijtenbeek, T., Pronost, N., Egges, A. and Overmars, M.H., 2011. Interactive Character Animation using Simulated Physics. In Eurographics (STARs) (pp. 127-149). Geijtenbeek, T., van de Panne, M. and van der Stappen, A.F., 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics (TOG), 32(6), p.206. Gleicher, M., 2008, October. Motion Synthesis by Example. Slides for lecture, distinguished lecture series, Computer Sciences Department, University of Utah. Gleicher, M., Shin, H.J., Kovar, L. and Jepsen, A., 2008, August. Snap- together motion: assembling run-time animations. In ACM SIGGRAPH 2008 classes (p. 52). ACM. Hendrikx, M., Meijer, S., Van Der Velden, J. and Iosup, A., 2013. Procedural content generation for games: A survey. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 9(1), p.1. Kovar, L., Gleicher, M. and Pighin, F., 2002, July. Motion graphs. In ACM transactions on graphics (TOG) (Vol. 21, No. 3, pp. 473-482). ACM. Liu, L., Panne, M.V.D. and Yin, K., 2016. Guided learning of control graphs for physics-based characters. ACM Transactions on Graphics (TOG), 35(3), p.29. Shapiro, A., 2011, November. Building a character animation system. In International Conference on Motion in Games (pp. 98-109). Springer Berlin Heidelberg. Shapiro, A., Cao, Y. and Faloutsos, P., 2006, June. Style components. In Proceedings of Graphics Interface 2006 (pp. 33-39). Canadian Information Processing Society.