3D Modelling & Animation Character Animation: Lip-Sync Animation

Agenda Definitions Selection of sub-objects Vertex clusters Blend shapes Using MEL for saving selection sets Workflow Managing audio

Definitions Lip-sync - synchronizing facial animation to audio over time Facial animation achieved via morph-targets - (metamorphosis) - ‘morph’ general term used in computing for transforming and blending one image into another over time. Maya uses the term ‘Blend Shapes’ to refer to what other packages and authors may call ‘morph-targets’

Aims of Lip-Sync Ideally convey speech and aspects of emotion Full facial characterisation requires morph targets for facial shapes (phonemes) associated with spoken dialog and facial expressions associated with emotion and gesture Time-consuming manual animation craft - an acquired skill Animation semi-automated in some packages (MotionBuilder) but still relies on manual creation of facial shapes as a basis for lip-sync sequence

Facial shapes

Blend shape collection of facial shapes

Target influences base shape Base Shape and Target Blend Shape Base Target Slider determines amount of influence applied Target influences base shape

Cluster weights

Phoneme Chart Phoneme Example AE cat AO M fought moon AX N alas new B bike D dam F fox G gate H hat IY heat K cake KG concur L lock M moon N new OW goat P point S site SH shape T tap UH look UW loot V veer Z zoo ZH Pleasure

Creating Selection Sets with MEL

MEL procedure made from selection set Paste procedure into here and press CTRL+ENTER Call procedure with procedure name in this pane then CTRL+ENTER

Creating MEL procedures from echoed commands proc selectVertices () { select -r pSphere1.vtx[298] ; select -tgl pSphere1.vtx[297] ; select -tgl pSphere1.vtx[296] ; select -tgl pSphere1.vtx[295] ; } Paste procedure into upper pane of script editor - press CTRL+ENTER Call procedure with procedure name and press CTRL+ENTER e.g. selectVertices CRTL+ENTER

Workflow Create mouth opening if required Select vertices and create vertex clusters Could use MEL to help re-select vertice sets Modify vertex cluster with paint-weights tool Duplicate base shape to create target shapes Edit target shapes for lip-sync (phonemes) and facial expression requirements Key-frame expressive dialog using imported audio file Preview sequence as a PlayBlast Composite audio and video in external package (Premiere)

Handling Audio Supported audio formats .wav, .aif, aiff Import audio onto timeline - make sure track set Audio also appears in Trax editor Set playback to real-time to hear and scrub audio Set the playback speed via the Timeline preference dialog Audio rendered into a PlayBlast window Render final sequence in video editor with associated sound track