1. Cengizhan Can Phoebe de Nooijer
INFOMCANIM - Research paper presentation
Combining Recurrent Neural Networks and Adversarial Training for Human Motion Modelling, Synthesis and Control
Cengizhan Can
Phoebe de Nooijer

2. Motivation of the paper
Focus: Generative model for human motion synthesis and control

3. Motivation of the paper
Focus: Generative model for human motion synthesis and control

4. Motivation of the paper
Focus: Generative model for human motion synthesis and control
Motion synthesis
The generation through algorithms of new motion sequence

5. Motivation of the paper
Focus: Generative model for human motion synthesis and control
Motion synthesis
The generation through algorithms of new motion sequence

6. Motivation of the paper
Focus: Generative model for human motion synthesis and control
Motion synthesis
Control
The generation through algorithms of new motion sequence
How effectively and quickly animations can be changed

7. Motivation of the paper
Focus: Generative model for human motion synthesis and control
Motion synthesis
Control
The generation through algorithms of new motion sequence
How effectively and quickly animations can be changed

8. Motivation of the paper
Focus: Generative model for human motion synthesis and control
Motion synthesis
Control
Generative model
The generation through algorithms of new motion sequence
How effectively and quickly animations can be changed
A generative model learns the joint probability distribution p(x,y)
(Whereas a discriminative model learns the conditional probability distribution p(y|x))

9. Motivation of the paper
What is wrong with current approaches?

10. Motivation of the paper
What is wrong with current approaches?
Current deep RNN based methods often have difficulty obtaining good performance for long term motion generation

11. Motivation of the paper
What is wrong with current approaches?
Current deep RNN based methods often have difficulty obtaining good performance for long term motion generation
Specifically, long-term results suffer from occasional unrealistic artifacts

12. High-level overview
The key idea is to combine recurrent neural networks and adversarial training for human motion modeling

13. High-level overview
The key idea is to combine recurrent neural networks and adversarial training for human motion modeling
Constructing a generative deep learning model from a large set of prerecorded motion data

14. High-level overview
The key idea is to combine recurrent neural networks and adversarial training for human motion modeling
Constructing a generative deep learning model from a large set of prerecorded motion data
Using a “refiner network” with an adversarial loss

15. High-level overview
The key idea is to combine recurrent neural networks and adversarial training for human motion modeling
Constructing a generative deep learning model from a large set of prerecorded motion data
Using a “refiner network” with an adversarial loss
Can randomly generate an infinite number of high-quality motions with infinite length

16. Problem statement:
How can Recurrent Neural Networks for Human Motion Modelling, Synthesis and Control improved?

17. Related work and background information
Shrivastava et al.:
Adversarial network to improve realism of synthetic images using unlabeled real image data
GAN and RNN

18. Technical details of the approach
Feature Representation:
Joint angle poses
Character states

19. Technical details of the approach
Input: hidden states; current feature
Output: probabilistic distribution feature

20. Technical details of the approach
Hidden states
Back Propagation Through Time (BPTT)
Long Short Term Memory cells
Probabilistic distribution (Gaussian Mixture Model)

21. Technical details of the approach
GNN training strategies:
Adding noise
Down sampling
Optimization method
Training data sets size
Process of this model

22. Technical details of the approach
Refiner Network
Discriminative Model
Motion Regularization

23. Technical details of the approach
GAN training strategies:
Training the generative model more
Using history of refined motions
Adjusting the training strategy when one of the models is too strong
Process of this model

24. Technical details of the approach

25. Technical details of the approach
Motion model in use:
Random motion generation
Offline motion design
Online motion control
Motion denoising

26. Demo

27. Critical analysis of the approach & evaluation
Highly successful approach

28. Critical analysis of the approach & evaluation
Highly successful approach
Demo only uses ‘stick figures’

29. Critical analysis of the approach & evaluation
Highly successful approach
Demo only uses ‘stick figures’
Might not work well on aperiodic motions?

30. Critical analysis of the approach & evaluation
Highly successful approach
Demo only uses ‘stick figures’
Might not work well on aperiodic motions?
Effectiveness only researched in the area of motion synthesis and control
What about motion tracking, motion recognition etc.?

31. Possible improvements of the paper (future work)
Two largest disadvantages:

32. Possible improvements of the paper (future work)
Two largest disadvantages:
More animations
E.g. Sprinting, crawling, toe walking

33. Possible improvements of the paper (future work)
Two largest disadvantages:
More animations
E.g. Sprinting, crawling, toe walking
Not able to run on mobile applications

34. Discussion & Questions
Could future implementations significantly reduce video game development time?
Could this technique be useful for simulations?

35. Discussion & Questions What are your questions?
Could future implementations significantly reduce video game development time?
Could this technique be useful for simulations?
What are your questions?