1. Three Topics Facial Animation 2D Animated Mesh MPEG-4 Audio

2. Facial Animation A synthetic visual object to allow animate face to reproduce expression, emotion and speech pronunciation. Facial Definition Parameters (FDP) --- Define facial shape and texture Facial Animation Parameters (FAP) --- Represent the facial expression, emotion and speech pronunciation(based on 84 feature points) Facial Animation Parameter Unit (FAPU) --- Represent the spatial distance between major facial features

3. Facial Animation Two FAP parameters 1. Viseme -- A visual correlate to a phoneme -- Set 14 static visemes. e.g. #1 is “b,p”; #11 is “e”; #4 is “t,d”. Example is “bed” -- Coordination of speech and mouth movement. e.g. The shape of mouth is influenced by previous, current and next phoneme

4. Facial Animation 2. Expression -- Be animated by a value e.g. expression #1 is “Joy”, description is “ the eyebrows are relaxed, the mouth is open….” -- Multiple expressions can be animated simultaneously e.g. “joy” and “surprise”

5. 2-D Animated Meshes For nature or synthetic visual objects –Content manipulation –Animation –Augmentation (Overlay) –Transfiguration (Merge or replace with synthetic visual object)

6. 2-D Animated Meshes Representation –Partition a 2D planar region into a set of triangle patches –Vertices of triangle of patches refer to as node points –Node points of initial mesh are tracked as the VOP (Visual Object Plane) move with scene

7. 2-D Animated Meshes Dynamic mesh Mesh bit-stream consists of mesh geometry and mesh motion 1. Mesh geometry coding -- Uniform mesh: A set of rectangles, and then split each one into two triangles -- Delaunay Mesh: By coding boundary nodes first, and then interior nodes of meshes.

8. 2-D Animated Meshes 2.Mesh Motion - Each mesh node has a motion vector - Breadth-first traversal - Motion vector are encoded/decoded predictively

9. MPEG-4 Audio Definition The spatio-temporal combination of audio objects –Audio object is a single audio stream –Audio object related with mixing, effects processing, switching and delaying –Using a signal-processing language

10. MPEG-4 Audio Natural Audio Coding –Parametric Coding (Speech coding) bit-rate between 2 and 6 kbit/s –Excited linear predictive (CELP) bit-rate between 6 and 24 kbit/s –Advanced Audio Coding (AAC) bit-rate start at 16 kbit/s

11. MPEG-4 Audio Text-To-Speech (Synthesize Speech) –Translate the accessed text into a string of phoneme symbols –Retrieve corresponding synthetic units from database –Concatenate the synthetic unit to synthesize the output speech