1. Phase-Functioned Neural Networks for Character Control
Daniel Holden, Taku Komura, Jun Saito
Presented by Kelly Griffioen and Edwin de Leeuw

2. Overview Introduction System overview Data acquisition & processing
Phase-functioned neural network
Runtime
Results/Evaluation
Discussion

3. Introduction

4. Introduction

5. Introduction

6. System overview What is a Phase-Functioned Neural Network?
A Neural Network where the weights are generated as a
function of the phase.

7. System overview
The “phase” is the scalar variable in the range 0 to 2 ℼ representing the point in time of the current pose in the locomotion cycle.
Given the phase:
The pose of the character is far less ambiguous.
The space of poses is smaller and more convex.

8. Data acquisition & processing

9. Data acquisition & processing
Data comes from motion capture
Unstructured, long takes (~10 min)
Different sequences performed with different “gaits” and facing directions
Pose of character and trajectory extracted for each point in time
Mocap data includes obstacles to climb
Capturing motion and geometry together is difficult
Make a database of heightmaps and fit patches from it to each locomotion cycle.

10. Phase-functioned neural network Neural Network Structure
(Activation function)
Phase function =
Cubic Catmull-Rom Spline

11. Phase-functioned neural network Phase function
Why a cubic Catmull-Rom spline?
Can easily be made cyclic
Number of parameters proportional to number of control points
Varies smoothly with respect to input parameter

12. Phase-functioned neural network Training
Per frame
Stack input and output variables and phases into matrices
Normalize with mean values and standard deviations
Scale input values related to joints by 0.1
Fit terrain

13. Phase-functioned neural network Training
Training is an optimization problem with respect to the 4 parameters ɑ1, ɑ2, ɑ3 and ɑ4 of the phase function and cost function:
Derivatives calculated by the stochastic gradient descent algorithm Adam from Kingma and Ba in 2014 with a model implemented in Theano by Bergstra in 2010

14. Runtime Need to supply at each frame: PFNN Phase p
Neural network input x
Joint positions and velocities of network output y will be used as new input for next frame together with past/future trajectories

15. Runtime Input relating to future trajectories
Future elements of the trajectory are computed by blending the trajectory from the game-pad control stick and the one generated by the PFNN in the previous frame by:
𝜏 = 0.5 for blending velocities
𝜏 = 2.0 for blending facing directions
Natural looking, but still responsive character

16. Runtime Precomputation of the phase function
Constant method: precompute for n=50 locations
Increases memory by n/4
Removes computation on runtime
Linear method: precompute at 10 locations, use linear interpolation
Less memory, more accurate
More time to compute
Cubic method: evaluate the entire Catmull-Rom spline

17. Results/Evaluation

18. Results/Evaluation

19. Results/Evaluation

20. Discussion
Conventional animation systems use a state machine to produce animations. This system uses a (specific) Neural Network instead, which reduces the time spent on crafting animations ourselves. But a NN is often considered a black box.
We trade control for less complexity. Is this the future of animation systems?

21. Discussion answer
We think that it mostly depends on the context (type of system, game) of the application we want to use animation for. Looking at the results, the animations look quite realistic, and the neural network can easily be used without much user input (apart from generating the initial data from motion capture). If data is readily available, (or if you use a pre-trained network) it could be used for quickly animating characters while using only a small bit of computing power and memory.
However, the neural network can only produce the movement of one specific character, since it learned from motion capture data. This means specific motion has to be present in the data beforehand, for the network to be able to learn it.
If an animator wants a to have, for example, different movements for different races in a game or wants both a male and a female character, a new network has to be trained, from new motion capture data, for each different style of movement. Given the 30 hours of training time, that might be a reason to choose a different approach.

22. Discussion answer (2)
Furthermore, the biggest problem with using a neural network is that we cannot predict or explain the results the network outputs. This means that we lose fine-grained control over the animations, which is undesired in some circumstances, e.g. when animating cutscenes. Being able to understand neural networks to a degree where we can also predict and have control over the output is still an active research topic. As long as this problem is not solved, we do not see this way of animating completely replace current, state-machine driven animation systems.
To summarize, we think this type of animation system has a lot of benefits, but also some drawbacks. We think it can coëxist alongside other systems, where the animation system of choice depends on what type of animation is desired.