University of Paris 8 Animation improvements and face creation tool for ECAs Animation improvements and face creation tool for ECAs Nicolas Ech Chafai Nicolas Ech Chafai Benjamin Dariouch Benjamin Dariouch Maurizio Mancini Maurizio Mancini Catherine Pelachaud Catherine Pelachaud

Overview aiming at improving agent's facial animation quality: aiming at improving agent's facial animation quality: we are studying some motion captured data we are studying some motion captured data we apply results to our ECA we apply results to our ECA to allow the creation of individualized ECAs: to allow the creation of individualized ECAs: we developed one tool for MPEG4 face creation we developed one tool for MPEG4 face creation we propose some refinements to MPEG4 specification we propose some refinements to MPEG4 specification

MOCAP data analysis Three main goals: Three main goals: displacement of FAPs during emotion presentation displacement of FAPs during emotion presentation synchronization between different FAPs synchronization between different FAPs FAP values during transition between consecutive emotions FAP values during transition between consecutive emotions

Collected MOCAP data 2 actors 2 actors 33 markers, 21 of them are the MPEG4 FAPs 33 markers, 21 of them are the MPEG4 FAPs 78 sequences: 78 sequences: basic movements basic movements rising eyebrows rising eyebrows smiling smiling … basic emotions basic emotions anger anger happiness happiness surprise surprise …

MOCAP problems (we discovered that) obtaining usable data is not straightforward (we discovered that) obtaining usable data is not straightforward right size and shape markers have to be used right size and shape markers have to be used cameras have to be placed properly cameras have to be placed properly data has to be translated to the needed reference system data has to be translated to the needed reference system data has to be filtered from noise data has to be filtered from noise

Data example smile smile

Video examples frown clip frown clip file: clips/coline 36 eyebrows file: clips/coline 36 eyebrows fear clip fear clip file: clips/coline 56 fear file: clips/coline 56 fear

Facial animation model FAPs displacement during basic emotions FAPs displacement during basic emotions our model was simply based on onset-apex-offset our model was simply based on onset-apex-offset

Data observed model on real data we observed other general behaviors on real data we observed other general behaviors

Results we started to introduce the ADSR model: we started to introduce the ADSR model: given a sequence of (phase,intensity,duration) where phase is from {Attack, Decay, Sustain Release}, the FAP curve is built using keyframe Hermite interpolation: given a sequence of (phase,intensity,duration) where phase is from {Attack, Decay, Sustain Release}, the FAP curve is built using keyframe Hermite interpolation:

ADSR vs real data

ADSR example clip clip

MPEG4 face tool

imports models from Poser imports models from Poser allows the selection of the areas influenced by FDPs allows the selection of the areas influenced by FDPs

Tool's features automatic selection and symmetrization automatic selection and symmetrization automatic association region name available FDPs automatic association region name available FDPs

Example exports into a data file (containing geometry + regions) readable from the Greta's player exports into a data file (containing geometry + regions) readable from the Greta's player

Example flat.avi (note: female speech) flat.avi (note: female speech)

Added new FAPUs after adding new faces some refinements for the MPEG4 player will be needed after adding new faces some refinements for the MPEG4 player will be needed

Example clip without new FAPUs clip without new FAPUs clip with new FAPUs clip with new FAPUs

Conclusions more data has to be captured in proper way more data has to be captured in proper way focus more on interaction between different FAPs and transition between sequential expressions focus more on interaction between different FAPs and transition between sequential expressions ADSR has to be fully implemented ADSR has to be fully implemented for documentation, papers for documentation, papers for Greta applications available to the other HUMAINE members please contact us for Greta applications available to the other HUMAINE members please contact us