1. 3D Modelling & Animation
Character Animation: Lip-Sync Animation

2. Agenda Definitions Selection of sub-objects Vertex clusters
Blend shapes
Using MEL for saving selection sets
Workflow
Managing audio

3. Definitions
Lip-sync - synchronizing facial animation to audio over time
Facial animation achieved via morph-targets - (metamorphosis) - ‘morph’ general term used in computing for transforming and blending one image into another over time.
Maya uses the term ‘Blend Shapes’ to refer to what other packages and authors may call ‘morph-targets’

4. Aims of Lip-Sync Ideally convey speech and aspects of emotion
Full facial characterisation requires morph targets for facial shapes (phonemes) associated with spoken dialog and facial expressions associated with emotion and gesture
Time-consuming manual animation craft - an acquired skill
Animation semi-automated in some packages (MotionBuilder) but still relies on manual creation of facial shapes as a basis for lip-sync sequence

5. Facial shapes

6. Blend shape collection of facial shapes

7. Target influences base shape
Base Shape and Target Blend Shape
Base
Target
Slider determines amount of influence applied
Target influences base shape

11. Cluster weights

13. Phoneme Chart Phoneme Example AE cat AO M fought moon AX N alas new B
bike D
dam F
fox G
gate H
hat IY
heat K
cake KG
concur L
lock M
moon N
new OW
goat P
point S
site SH
shape T
tap UH
look UW
loot V
veer Z
zoo ZH
Pleasure

14. Creating Selection Sets with MEL

15. MEL procedure made from selection set
Paste procedure into here and press CTRL+ENTER
Call procedure with procedure name in this pane then CTRL+ENTER

17. Creating MEL procedures from echoed commands
proc selectVertices () { select -r pSphere1.vtx[298] ;
select -tgl pSphere1.vtx[297] ;
select -tgl pSphere1.vtx[296] ;
select -tgl pSphere1.vtx[295] ; }
Paste procedure into upper pane of script editor - press CTRL+ENTER
Call procedure with procedure name and press CTRL+ENTER
e.g. selectVertices CRTL+ENTER

18. Workflow Create mouth opening if required
Select vertices and create vertex clusters
Could use MEL to help re-select vertice sets
Modify vertex cluster with paint-weights tool
Duplicate base shape to create target shapes
Edit target shapes for lip-sync (phonemes) and facial expression requirements
Key-frame expressive dialog using imported audio file
Preview sequence as a PlayBlast
Composite audio and video in external package (Premiere)

19. Handling Audio Supported audio formats .wav, .aif, aiff
Import audio onto timeline - make sure track set
Audio also appears in Trax editor
Set playback to real-time to hear and scrub audio
Set the playback speed via the Timeline preference dialog
Audio rendered into a PlayBlast window
Render final sequence in video editor with associated sound track