1. Muscle Based Facial Animation
Jason Jerald
April 7, 2004

2. Overview History Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
Breton, Bouville and Pele 2001
Bui and Nijholt 2003

3. Facial Muscle History 19th century - physiologist Duchenne
applied electrical currents to freshly guillotined heads to observe facial contortions
Later applied same technique to old inmates of alms houses to create artificial expressions
Recorded with photography
Psychologists Ekman and Friesden
Created the Facial Action Coding System (FACS)
Notational-based environment that determines emotional states from visible facial distortion
Individual muscles are described as Action Units (AU)
This work is commonly used in computer facial animation

4. Facial Muscle History
Platt published masters thesis on a physically based muscle–controlled facial expression model
Waters published the seminal paper on muscle based facial animation using muscle vectors
21st century – improvements on Waters model

5. Overview History Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
Breton, Bouville and Pele 2001
Bui and Nijholt 2003

6. Various Muscle Models Free form deformations Spline psuedo muscles
Deforms objects by manipulating control points arranged in a 3d cubic lattice
Surface regions corresponding to anatomical descriptions of the muscle actions are defined
Displacing control point is analogous to actuating a physically modeled muscle
More Intuitive than vector representations but cannot model furrows, bulges, and wrinkles.
Spline psuedo muscles
Deforming facial mesh in muscle-like fashion
Ignores underlying anatomy
Supports smooth and flexible deformations
Hierarchical splines allow more detail in specified regions

7. Various Muscle Models Mass-spring methods Layered spring meshes
Forces applied to elastic meshes through muscle arcs
Muscles represented as collections of functional blocks
Action units created by applying muscle forces to deform the spring network
Layered spring meshes
models skin, fatty tissue, and muscle tied to bones
Spring elements connect each mesh node and each layer
Realistic but computationally expensive
Vector representations
What this talk focuses upon

8. Overview History Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
Breton, Bouville and Pele 2001
Bui and Nijholt 2003

9. Muscle Vector Model What is needed?
Few dynamic parameters that emulate the primary characteristics of facial expression
Linear/parallel muscles that pull and sphincter muscles that squeeze
Factors determining nodal mobility are
Tensile strength of the muscle and skin
Proximity to the muscle node of attachment
Proximity to the bone
Elastic bounds of the relaxed tissue
Interaction of other muscles

10. Muscle Vectors Models the actions of muscles upon skin
Each muscle has a zone of influence
A muscle includes vector field direction, an origin, and an insertion point

11. Advantages / Disadvantages
Independent of facial mesh (facial mesh can be exchanged)
Compact representation
Expression parameters can control groups of muscles
Fast
Disadvantages
Positioning of muscles can be time consuming
Does not take curvature into account
Artifacts when a mesh vertex is under the influence of multiple muscle actions

12. Muscle Vector Parameters
Muscle attached at two points
Point of Attachment A – the root of the muscle attached to the bone
Point of insertion I into the flesh
Muscle can therefore be considered as the vector AI
flesh
Muscle
bone
With no contraction the points of attachment and insertion do not move and the muscle vector maintains its length
Acts like a magnet attracting all the vertices within its zone of influence. The skin contracts more near the muscle.

13. Muscle Vector Parameters
V is the mesh vertex
Opening angles
β is the opening angle
α is the maximum angular limit
Muscle contraction is faded as β raises to α
Radial distances as a proportion of |AI|
S is where the muscle influence starts to fade
E is where the muscle influence ends
Vertices are faded if they are in the band defined by S and E
Muscle

14. Equations C is the contraction factor (between 0 and 1)
is the fading coefficient related to the angular distance between AV and AI
is the fading coefficient related to the radial distance between V and S if V is in the fading band SE

15. Muscle Results

16. Sphincter / Mouth Muscle
Waters models the mouth with a sphincter muscle
Described from a single point around which the surface contracts as if drawn together like a string bag
Longitudinal and vertical axii allow elliptical shape
Sphincter muscle
Elliptical Sphincter muscle

17. Muscle sets

18. Facial Action Coding System (FACS)
Developed by Psychologists Ekman and Friesden in 1977
FACS is Description of facial muscles and jaw/tongue derived from analysis of facial anatomy
Notational-based system that determines emotional states from visible facial distortion
Action Units (AU) correspond to muscle vectors

19. Waters Results

20. Overview History Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
Breton, Bouville and Pele 2001
Bui and Nijholt 2003

21. Breton et al
Non-muscle parametric animation mixed with muscle vectors (jaw, eyes, eyelids, neck)
Opening of mouth with a muscular system requires distinction between lower and upper lips

22. The eyes Eyes and Eyelids Modeled as spheres and hemispheres
Simple rotations
Random blinking
Gaze looking forward when speaking
Random gaze direction when not speaking

23. Jaw and Neck Jaw Neck Single axis of rotation
Lower lips not within jaw influence
Neck
Three axii of rotation
Center of rotation is the center of the neck bounding box
Vertices of the head are fully rotated
Rotations of the neck linearly faded with distance
Jaw boundary
Neck boundary

24. Lips
Distinction between upper and lower lips must be made in order to open mouth
Distinction between upper and lower lips determined at load time

25. Breton et al 2001 results

26. Overview History Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
Breton, Bouville and Pele 2001
Bui and Nijholt 2003

27. Bui et al
Multiple muscle action artifacts removed by simulating parallelism
Division into regions
Wrinkles

28. Muscle action artifacts
The problem
Problem when a mesh vertex is under the influence of multiple muscle actions
Muscle actions are independent
Actual nodal displacement determined by a succession of muscle actions
Unnatural results occur when a vertex is shifted outside the zone of influence of adjoining muscle vectors
The solution
Combining muscle contractions done by simulating parallelism
For a vertex inside multiple muscles’ zone of influence, small units of contraction levels are applied until no more contraction to apply
Step sizes of 20% of full contraction found to have good results

29. Region Division
Allows easier rendering of special parts of the face such as lips and eyebrows
Reduces artifacts generated by displacement of vertices in regions that are not affected by a muscles contraction

30. Muscle Action Artifacts
Two muscles with no parallelism
Two muscles with parallelism
Three muscles with parallelism

31. Wrinkles
Assume muscles lie parallel to the facial skin and heights of wrinkles are the same
Height and number of wrinkles are predefined for each muscle
To make wrinkles more visible use triangular flat shading at vertex where wrinkle starts

32. Bui et al results (2003) happy neutral Sad with close up of wrinkles
surprise

33. Demo

34. Conclusion Waters is the basic model
Additional tricks can be added to improve appearance
Facial expressions can be defined by muscle groups while individual muscles can also be controlled.
Simple yet effective and fast

35. References
Breton, G., Bouville, C., and Pel, D FaceEngine a 3d Facial Animation Engine for Real Time Applications. in Proceedings of 6th International Conference on 3D Web Technology, pp
Bui, T. D., Heylen, D., and Nijholt, A Improvements on a Simple Muscle-Based 3d Face for Realistic Facial Expressions. in Proceedings of 16th International Conference on Computer Animation and Social Agents, pp
Noh, J.-Y. and Neumann, U A Survey of Facial Modeling and Animation Techniques. USC Technical Report No
Parke, F. I. and Waters, K. (1996) "Chapter 7 Skin and Muscle-Based Facial Animation." In Computer Facial Animation, pp
Waters, K A Muscle Model for Animating Three-Dimensional Facial Expression. in Proceedings of 14th Annual Conference on Computer Graphics and Interactive Techniques, pp