1. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Department of Computer Science, University of Bath

2. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Motivation

3. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Overview Analysis and Re-Synthesis of Facial Dynamics –Dynamics > CG Puppet –Dynamics > us –Dynamics > Computer Vision Intuitive/Automatic methods for animation –Voice Driven Animation/Analysis –Reactive Avatars

4. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing/Resynthesising Dynamics Time Parameter Magnitude

5. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing/Resynthesising Dynamics Study effect of manipulating temporal characteristics of emotions, e.g. smiles [Emotion 2007]

6. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing/Resynthesising Dynamics

7. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing/Resynthesising Dynamics

8. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing/Resynthesising Dynamics What did we learn? –Genuine Dynamics = More likeable/Trustworthy/Employable –Fake Dynamics = Less Likeable/Trustworthy/Employable Relationship between Attractiveness and Facial Dynamics To Do! - Mock Video Game Scenario

9. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 1.Define facial regions local to sub-facial actions 2.Construct new training set of artificial images Analysing Multiple Actions

10. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Analysing Multiple Actions 3.Create a corresponding artificial training set of shape vectors

11. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Performance Analysis

12. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Image Based Transfer

13. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Performance Analysis

14. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 3D Model Transfer Animation of a 3D facial model

15. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 3D Model Transfer Dynamic knowledge easily applied to existing pipelines

16. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 3D Facial Modelling 4D Camera – Continuous 3D Scanning (Cardiff/Surrey)

17. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 3D Facial Modelling (Link to 3D Example Expression)Example

18. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker 3D Facial Modelling

19. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Voice Driven Animation Visual Speech Synthesis Non-Verbal Action Synthesis

20. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Perceptual Evaluation Motivation: How best to evaluate facial animation? –Perceptual test based on McGurk effect [ACM TAP 2005]

21. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Perceptual Evaluation Record participant responses Compare real and synthetic Subtle - participants blind to real motivation Isolate strengths and weaknesses

22. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Reactive Full Body Avatar Animation Motivation: –Animate based on an input performance –Key avatar drives other avatar behaviour

23. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Conclusions and Future Directions Humans are very sensitive to different dynamics Different interpretation depending on timing Human realism will depend on understanding dynamic behaviours Realism will also depend on the accurate modelling and resynthesis of dynamics - e.g. McGurk effect Technology now allows study of 3D dynamics - our work has only just started!

24. Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Thanks for listening! Acknowledgements: Eva Krumhuber, Antony Manstead, Lanthao Benedikt, David Marshall, Paul Rosin, Alistair Calvert, Cathy Holt, Daniel Mason, Gemma Whatling, Simon Rushton, Susan Paddock RAEng, EPSRC.