Artificial Intelligence & Information Analysis Group (AIIA) Centre of Research and Technology Hellas INFORMATICS & TELEMATICS INSTITUTE

Profile  The Informatics and Telematics Institute is a non-profit organization under the auspices of the General Secretarial of Research and Technology of Greece. Since March 10th, 2000 it is a founding member of the Centre of Research and Technology Hellas (CERTH) also supervised by the Greek Secretariat of Research and Technology.  The Artificial Intelligence & Information Analysis Group within CERTH/ITI has been active in 2-D/3-D image processing, human-centered interfaces, and multimedia authentication for more than two decades. It is also affiliated with the Aristotle University of Thessaloniki.

Profile  Two faculty members, several postdoctoral fellows, and more than ten Ph.D. students  Participation in 17 European projects (IST, RTN/HCP, ESPRIT, ACTS, LTR/BRA, RACE, TEMPUS, AIM) and 8 national projects  Books: 6; One in 3-D Image Processing Algorithms  Chapters contributed to edited books: 14  Journal Papers: 111  Conference Papers: 273

Role Prior experience in topics related to the project:  3-D Image Processing & Graphics  Human-centered interfaces  Face analysis; facial feature extraction; face tracking; facial feature tracking; face expression recognition;  Graphical communication: audio-visual speech analysis;

Role  Research activities in synthesis tasks  Development of talking heads (virtual salesmen) with emphasis in  Texture mapping;  Synthesis of facial expressions; prototypes;  Synthesis compliant with standards MPEG-4 FDPs, FACS, etc.  Development activities: Contribution to the integration of generic tools for graphic design, and speech recognition /synthesis into the Worlds Studio Platform.

State of the art in Facial Modeling and Animation  2D and 3D morphing  Manually corresponding features  Combination of 2D morphing with 3D geometric transformations  Physics based muscle modeling  Spring Mesh Muscle  Vector Muscle  Layered Spring Mesh Muscle  Finite Element Method

 Pseudo or Simulated Muscle  Free Form Deformations  Combination of 2D morphing with 3D geometric transformations  Wires  Model Fitting  Adaptation to an example face  MPEG-4 Compatible Models  Facial Action Coding System  Example-driven deformations of the model  Speech driven heads State of the art in Facial Modeling and Animation

 HSV thresholding based on facial colors  Connected components labeling and analysis  Segmentation of the image  Moments computation  Best fit ellipse image  Spatial constraint for face contour  Initialization of the snake  Snake deformation by energy minimization  Inner face contour image Face Detection using Color Information

 Thresholding based on facial colors (segmentation)  Keep only the pixels having color similar to facial texture Initial image HSV thresholding Connected components labeling Segmented image HSV Thresholding & Connected Components Labeling

Moments Computation  Ellipses in the segmented image  Best fit ellipse Segmented image Moments Computation Best Fit Ellipse image

Model Superposition on Face Images  ICP for superposition of the model on points already defined on a face image  Mass-Spring Models to fit the face model on the face image

I(terative) C(losest) P(oint) algorithm  ICP is based on the Closest Set of Points  Closest Set of Points leads to the Quaternion  Quaternion  an easily handled vector  the basis of the ICP transformations  similar to the rotation and translation matrices  Convergence  Monotonically to the nearest local minimum  rapid during the first few iterations  globallity depends on the initial parameters

Mass-Spring Models  FEM restricted models  Simulates models as masses connected with springs  Physics based simulation four masses connected among themselves with uniform springs

Examples  Interactive definition of points on the face image  Randomly initial positioning of the face model

Examples  Fit of the model by applying the Mass-Spring Model  Fit of the model by applying the ICP algorithm

Ellipse fitting Model Superposition based on model’s contour Ellipse Image  Ellipse determination based on the model’s position

Combination of two methods  Ellipse extracted using Moments Computation  Ellipse extracted using the Model Fitting procedure  Intermediate ellipse (scale) Intermediate Image from Moments Computation Intermediate Ellipse image ellipse Image from Model Fitting

Spatial Constraint for Face Contour  Initialization of the snake  Circular deformable model Intermediate Ellipse Spatial Constraint for Face Contour Initialized Snake

Snake Deformation & Face Contour Detection  Snake Deformation by Energy Minimization  Snake Energy:  Internal Energy:  External Energy: Intermediate Ellipse Snake Deformation Deformated Snake

Future Work  Statistical analysis/synthesis of the images & facial models  Eigen-decomposition and PCA of the features  Features correspondences of the images & the model  Extraction of Face Definition Points (MPEG-4)  Extraction of Facial Expressions