RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKING Salih Burak Gokturk.

Slides:

Advertisements

Similar presentations

Real-Time Template Tracking

Advertisements

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki

Introduction to Support Vector Machines (SVM)

Active Appearance Models

DDDAS: Stochastic Multicue Tracking of Objects with Many Degrees of Freedom PIs: D. Metaxas, A. Elgammal and V. Pavlovic Dept of CS, Rutgers University.

SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.

Support Vector Machines

SVM—Support Vector Machines

Face Alignment with Part-Based Modeling

Activity Recognition Aneeq Zia. Agenda What is activity recognition Typical methods used for action recognition “Evaluation of local spatio-temporal features.

Robust 3D Head Pose Classification using Wavelets by Mukesh C. Motwani Dr. Frederick C. Harris, Jr., Thesis Advisor December 5 th, 2002 A thesis submitted.

Robust Object Tracking via Sparsity-based Collaborative Model

 INTRODUCTION  STEPS OF GESTURE RECOGNITION  TRACKING TECHNOLOGIES  SPEECH WITH GESTURE  APPLICATIONS.

Snakes with Some Math.

Snake: Active Contour Models. Department of Computer Science University of Missouri at Columbia History A seminal work in Computer vision, and imaging.

SIGGRAPH Course 30: Performance-Driven Facial Animation For Latest Version of Bregler’s Slides and Notes please go to:

Support Vector Machines (SVMs) Chapter 5 (Duda et al.)

LYU0603 A Generic Real-Time Facial Expression Modelling System Supervisor: Prof. Michael R. Lyu Group Member: Cheung Ka Shun ( ) Wong Chi Kin ( )

Face Recognition Based on 3D Shape Estimation

Snake: Active Contour Models

4EyesFace-Realtime face detection, tracking, alignment and recognition Changbo Hu, Rogerio Feris and Matthew Turk.

SIGGRAPH Course 30: Performance-Driven Facial Animation Section: Marker-less Face Capture and Automatic Model Construction Part 1: Chris Bregler, NYU Part.

Vision-based Control of 3D Facial Animation Jin-xiang Chai Jing Xiao Jessica Hodgins Carnegie Mellon University.

Face Recognition from Face Motion Manifolds using Robust Kernel RAD Ognjen Arandjelović Roberto Cipolla Funded by Toshiba Corp. and Trinity College, Cambridge.

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of nonlinear features.

The Terrapins Computer Vision Laboratory University of Maryland.

Comparing Kernel-based Learning Methods for Face Recognition Zhiguo Li

Dimensionality Reduction: Principal Components Analysis Optional Reading: Smith, A Tutorial on Principal Components Analysis (linked to class webpage)

Flow Based Action Recognition Papers to discuss: The Representation and Recognition of Action Using Temporal Templates (Bobbick & Davis 2001) Recognizing.

Irfan Essa, Alex Pentland Facial Expression Recognition using a Dynamic Model and Motion Energy (a review by Paul Fitzpatrick for 6.892)

Multimodal Interaction Dr. Mike Spann

Deformable Models Segmentation methods until now (no knowledge of shape: Thresholding Edge based Region based Deformable models Knowledge of the shape.

Use and Re-use of Facial Motion Capture M. Sanchez, J. Edge, S. King and S. Maddock.

Facial animation retargeting framework using radial basis functions Tamás Umenhoffer, Balázs Tóth Introduction Realistic facial animation16 is a challenging.

1 Recognition by Appearance Appearance-based recognition is a competing paradigm to features and alignment. No features are extracted! Images are represented.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

N n Debanga Raj Neog, Anurag Ranjan, João L. Cardoso, Dinesh K. Pai Sensorimotor Systems Lab, Department of Computer Science The University of British.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

An Introduction to Support Vector Machines (M. Law)

A Fast and Accurate Tracking Algorithm of the Left Ventricle in 3D Echocardiography A Fast and Accurate Tracking Algorithm of the Left Ventricle in 3D.

Computer Vision Michael Isard and Dimitris Metaxas.

Face Detection Using Large Margin Classifiers Ming-Hsuan Yang Dan Roth Narendra Ahuja Presented by Kiang “Sean” Zhou Beckman Institute University of Illinois.

Real-Time Detection, Alignment and Recognition of Human Faces Rogerio Schmidt Feris Changbo Hu Matthew Turk Pattern Recognition Project June 12, 2003.

Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.

CS 478 – Tools for Machine Learning and Data Mining SVM.

Sparse Bayesian Learning for Efficient Visual Tracking O. Williams, A. Blake & R. Cipolloa PAMI, Aug Presented by Yuting Qi Machine Learning Reading.

Human Activity Recognition at Mid and Near Range Ram Nevatia University of Southern California Based on work of several collaborators: F. Lv, P. Natarajan,

Support Vector Machines. Notation Assume a binary classification problem. –Instances are represented by vector x   n. –Training examples: x = (x 1,

Methods for 3D Shape Matching and Retrieval

RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKING Salih Burak Gokturk.

Feature Selection and Dimensionality Reduction. “Curse of dimensionality” – The higher the dimensionality of the data, the more data is needed to learn.

Feature Selction for SVMs J. Weston et al., NIPS 2000 오장민 (2000/01/04) Second reference : Mark A. Holl, Correlation-based Feature Selection for Machine.

Machine Learning in CSC 196K

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Wrapping Snakes For Improved Lip Segmentation Matthew Ramage Dr Euan Lindsay (Supervisor) Department of Mechanical Engineering.

Computer Vision Lecture 7 Classifiers. Computer Vision, Lecture 6 Oleh Tretiak © 2005Slide 1 This Lecture Bayesian decision theory (22.1, 22.2) –General.

Support Vector Machines Reading: Textbook, Chapter 5 Ben-Hur and Weston, A User’s Guide to Support Vector Machines (linked from class web page)

FACE RECOGNITION. A facial recognition system is a computer application for automatically identifying or verifying a person from a digital image or a.

Support Vector Machines (SVMs) Chapter 5 (Duda et al.) CS479/679 Pattern Recognition Dr. George Bebis.

1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.

Guillaume-Alexandre Bilodeau

Face Detection EE368 Final Project Group 14 Ping Hsin Lee

Recognition: Face Recognition

Outline Multilinear Analysis

The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression By: Patrick Lucey, Jeffrey F. Cohn, Takeo.

Support Vector Machines Introduction to Data Mining, 2nd Edition by

Tremor Detection Using Motion Filtering and SVM Bilge Soran, Jenq-Neng Hwang, Linda Shapiro, ICPR, /16/2018.

Combining Geometric- and View-Based Approaches for Articulated Pose Estimation David Demirdjian MIT Computer Science and Artificial Intelligence Laboratory.

Principal Component Analysis

Multimodal Caricatural Mirror

Presentation transcript:

RECOGNIZING FACIAL EXPRESSIONS THROUGH TRACKING Salih Burak Gokturk

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

MOTIVATION What does expression recognition mean? Guessing the meaning of facial deformations. What are the possible applications ? - Any interactive scenerio. - Video conferencing Why do we want to use 3-D information ? - 2-D system is very dependent on the view yet computationally simple.

Components of the recognition system Analysis -Face Tracking Intelligence -Support Vector Machine Classifier Shape Parameters

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

SNAKE MODEL Introduced by Kass and Witkin. Energy Minimization Problem: Used by Waters and Terzopoulos for tracking. Snakes are fit to important regions, and tracked from one view to the other.

MODEL BASED 3-D FACE TRACKING DeCarlo and Metaxas, ’96, very accurate tracking. Eisert and Girod, ’98, used in video-conferencing application, with efficient and accurate compression. Gokturk et. al., ’00, brings a data driven approach where the face model is learnt from stereo tracking. Model based approaches. Track all the points together with n-dimensional freedom on the shape. Based on Lukas-Tomasi-Kanade optical flow tracker.

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

PROBLEM DESCRIPTION(Tracking 1) ?

PROBLEM DESCRIPTION(Tracking 2) X(t) I(x(t)) I(t+1) TIME t+1 ? X(t+1)

PROBLEM DESCRIPTION (Recognition) X(t) [ Rigid, Open Mouth, Smile] ? Training DataClassifier Testing New DataOutput

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

ASSUMPTIONS Cameras are calibrated. The person should move slow unless the camera is fast enough for motion capture. The mesh is initialized to the first image. The user performs the expressions known to the computer

p - degrees of freedom - shape is learnt from stereo learning in our case Stereo Tracking Data Monocular Tracking Learn Shape

I l (x i (t) ) Time t: I l (x i (t+1)) Time t+1: ? - For robustness, u and v are estimated using a neighbourhood around the point. LUKAS TOMASI KANADE OPTICAL FLOW TRACKER

LUKAS TOMASI KANADE OPTICAL FLOW TRACKER EXTENDED TO 3D X(t) I(x(t)) I(t+1) TIME t+1 ? X(t+1)

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

STEREO LEARNING

FACE TRACKING - The deformation space of a particular individual is learnt - That particular individual is tracked using a mono camera - Tracked parameters : , R, T.

UNIVERSAL FACE TRACKER -The deformation space of a subset of people learnt - The shapes are aligned and PCA is applied on this set

OVERVIEW MOTIVATION PREVIOUS WORK PROBLEM DESCRIPTION THEORY OF TRACKING TRACKING VIDEOS THEORY OF SVM

Support Vector Machines (SVM) - Best discriminating hyperplane between two class of objects - Distinguish the vectors that carry the relevant information (support vectors) - if nonlinear data, map the data to high dimensional domain, then apply the SVM. Training DataClassifier Testing New DataOutput

Expected Contributions - Show that 3-D model based tracking is suitable for further applications. - Support vector machine is a suitable classifier for expression recognition What needs to be done - Choose the appropriate data input for SVM classifier. Using  vector might not help. Create more intelligent vectors in that case. - Choose an appropriate kernel (transformation function) for SVM. - Apply SVM in a one to many fashion. - Combination of intelligent features and SVM should give robust results.