1 ECE 738 Paper presentation Paper: Active Appearance Models Author: T.F.Cootes, G.J. Edwards and C.J.Taylor Student: Zhaozheng Yin Instructor: Dr. Yuhen.

Slides:



Advertisements
Similar presentations
Applications of one-class classification
Advertisements

Real-Time Template Tracking
Active Appearance Models
Feature Based Image Mosaicing
EigenFaces and EigenPatches Useful model of variation in a region –Region must be fixed shape (eg rectangle) Developed for face recognition Generalised.
Active Shape Models Suppose we have a statistical shape model –Trained from sets of examples How do we use it to interpret new images? Use an “Active Shape.
CSCE 643 Computer Vision: Lucas-Kanade Registration
1 Registration of 3D Faces Leow Wee Kheng CS6101 AY Semester 1.
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction.
Amir Hosein Omidvarnia Spring 2007 Principles of 3D Face Recognition.
Wangfei Ningbo University A Brief Introduction to Active Appearance Models.
AAM based Face Tracking with Temporal Matching and Face Segmentation Dalong Du.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
, Tim Landgraf Active Appearance Models AG KI, Journal Club 03 Nov 2008.
A 4-WEEK PROJECT IN Active Shape and Appearance Models
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
A Study of Approaches for Object Recognition
Announcements Quiz Thursday Quiz Review Tomorrow: AV Williams 4424, 4pm. Practice Quiz handout.
Active Appearance Models master thesis presentation Mikkel B. Stegmann IMM – June 20th 2000.
Appearance Models Shape models represent shape variation Eigen-models can represent texture variation Combined appearance models represent both.
Rodent Behavior Analysis Tom Henderson Vision Based Behavior Analysis Universitaet Karlsruhe (TH) 12 November /9.
Real-time Combined 2D+3D Active Appearance Models Jing Xiao, Simon Baker,Iain Matthew, and Takeo Kanade CVPR 2004 Presented by Pat Chan 23/11/2004.
Active Appearance Models Computer examples A. Torralba T. F. Cootes, C.J. Taylor, G. J. Edwards M. B. Stegmann.
Active Appearance Models Suppose we have a statistical appearance model –Trained from sets of examples How do we use it to interpret new images? Use an.
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Matching Compare region of image to region of image. –We talked about this for stereo. –Important for motion. Epipolar constraint unknown. But motion small.
Active Appearance Models based on the article: T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", presented by Denis Simakov.
Statistical Shape Models Eigenpatches model regions –Assume shape is fixed –What if it isn’t? Faces with expression changes, organs in medical images etc.
PhD Thesis. Biometrics Science studying measurements and statistics of biological data Most relevant application: id. recognition 2.
CS 485/685 Computer Vision Face Recognition Using Principal Components Analysis (PCA) M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive.
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
Active Appearance Models for Face Detection
Face Alignment Using Cascaded Boosted Regression Active Shape Models
MITRE Corporation is a federally-funded research-and- development corporation that has developed their own facial recognition system, known as MITRE Matcher.
Summarized by Soo-Jin Kim
Statistical Models of Appearance for Computer Vision
Active Shape Models: Their Training and Applications Cootes, Taylor, et al. Robert Tamburo July 6, 2000 Prelim Presentation.
Last tuesday, you talked about active shape models Data set of 1,500 hand-labeled faces 20 facial features (eyes, eye brows, nose, mouth, chin) Train 40.
Multimodal Interaction Dr. Mike Spann
CSE554AlignmentSlide 1 CSE 554 Lecture 5: Alignment Fall 2011.
1 Recognition by Appearance Appearance-based recognition is a competing paradigm to features and alignment. No features are extracted! Images are represented.
Object Detection with Discriminatively Trained Part Based Models
ECE 8443 – Pattern Recognition LECTURE 10: HETEROSCEDASTIC LINEAR DISCRIMINANT ANALYSIS AND INDEPENDENT COMPONENT ANALYSIS Objectives: Generalization of.
Parameter estimation. 2D homography Given a set of (x i,x i ’), compute H (x i ’=Hx i ) 3D to 2D camera projection Given a set of (X i,x i ), compute.
CSE554AlignmentSlide 1 CSE 554 Lecture 8: Alignment Fall 2013.
Computer Vision Lecture #10 Hossam Abdelmunim 1 & Aly A. Farag 2 1 Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt 2 Electerical.
CSE 185 Introduction to Computer Vision Face Recognition.
Computer Vision Hough Transform, PDM, ASM and AAM David Pycock
Sparse Bayesian Learning for Efficient Visual Tracking O. Williams, A. Blake & R. Cipolloa PAMI, Aug Presented by Yuting Qi Machine Learning Reading.
Paper Reading Dalong Du Nov.27, Papers Leon Gu and Takeo Kanade. A Generative Shape Regularization Model for Robust Face Alignment. ECCV08. Yan.
Multimodal Interaction Dr. Mike Spann
Features, Feature descriptors, Matching Jana Kosecka George Mason University.
Point Distribution Models Active Appearance Models Compilation based on: Dhruv Batra ECE CMU Tim Cootes Machester.
Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.
Statistical Models of Appearance for Computer Vision 主講人:虞台文.
Face detection and recognition Many slides adapted from K. Grauman and D. Lowe.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Deformation Modeling for Robust 3D Face Matching Xioguang Lu and Anil K. Jain Dept. of Computer Science & Engineering Michigan State University.
CSE 554 Lecture 8: Alignment
University of Ioannina
Fast Preprocessing for Robust Face Sketch Synthesis
Recognition: Face Recognition
Outline Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,”
Object Modeling with Layers
Outline S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine.
Brief Review of Recognition + Context
Image Registration 박성진.
Paper Reading Dalong Du April.08, 2011.
Active Appearance Models theory, extensions & cases
Presentation transcript:

1 ECE 738 Paper presentation Paper: Active Appearance Models Author: T.F.Cootes, G.J. Edwards and C.J.Taylor Student: Zhaozheng Yin Instructor: Dr. Yuhen Hu Date: Feb Note: some slides copyrighted by the original authors

2 Papers T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp , 2001 T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 Vol. 2, pp , Springer, (Best paper prize)

3 Flexible models Statistical Shape Models Active Shape Models (ASM) Combined Appearance Models Active Appearance Models (AAM) Shape modelASMAAM

4 Flexible models Shape Shape is the geometric information invariant to a particular class of transformations (translation+rotation+scaling) Appearance

5 Applications Flexible models can be used to: –Locate examples of structures in new images –Classify objects found in images –Filter images to pick out interesting features Practical problems: Face recognition, industrial inspection and medical image analysis

6 Flexible models Statistical Shape Models Active Shape Models (ASM) Combined Appearance Models Active Appearance Models (AAM) Shape modelASMAAM

7 Statistical Shape Models Given sets of training images build a statistical shape model Each shape in the training set is represented by a set of n labeled landmark points, which must be consistent from one shape to the next. Ex. The outline of a hand is represented by 72 labeled points

8 Statistical Shape Models Each shape is represented by a 2n*1 vector Using Principal Component Analysis (PCA) or eigen analysis, the shape model is where P is a 2n*t matrix whose columns are unit vectors along principle axes or basis vector b is a t*1 vector of shape parameters or weight Ex. Vary the first three parameters of the shape vector, b, one at a time

9 Aligning Two Shapes Procrustes analysis: –Find transformation which minimizes –Resulting shapes have approximately the same scale and orientation

10 Aligning a Set of Shapes Generalized Procrustes Analysis –Find the transformations T i which minimise –Where –Under the constraint that

11 Dimensionality Reduction

12 Dimensionality Reduction Data lies in subspace of reduced dim. However, for some t,

13 Statistical Shape Models Another example Shape of the facial structures with 68 points

14 Flexible models Statistical Shape Models Active Shape Models (ASM) Combined Appearance Models Active Appearance Models (AAM)

15 Active Shape Models Suppose we have a statistical shape model –Trained from sets of examples How do we use it to interpret new images? Use an “Active Shape Model” Iterative method of matching model to image

16 Active Shape Models (ASM) Assume we have an initial estimate for the pose and shape parameters (eg the mean shape).

17 Active Shape Models (ASM) Iterative algorithm –Look along normals through each model point to find the best local match for the model of the image appearance at that point (eg strongest nearby edge) –Update the pose and shape parameters to best fit the model instance to the found points –Repeat until convergence Initial pos5 th iterationsconvergence

18 ASM Search Overview Local optimization Initialize near target –Search along profiles for best match,X’ –Update parameters to match to X’.

19 Active Shape Models (ASM) Performance improvement (Multi-resolution implementation/coarse-to- fine approach) we start searching on a coarse level of a Gaussian image pyramid, and progressively refine. This leads to much faster, more accurate and more robust search.

20 Flexible models Statistical Shape Models Active Shape Models (ASM) Combined Appearance Models Active Appearance Models (AAM)

21 Combined Appearance Models Idea: Statistical Shape Model models the shape change of an object construct a similar statistical model to represented the intensity variation across a region (Think: skeleton and muscle)

22 Combined Appearance Models Method: Given a set of training images, labeled with land mark points, we can use image warping to deform each image so that the object has the mean shape, then build a statistical model of the grey-levels across the object. Ex. The central image is the mean

23 Building Appearance Models For each example extract shape vector Build statistical shape model, Shape, x = (x 1,y 1, …, x n, y n ) T

24 Building Appearance Models For each example, extract texture vector Shape, x = (x 1,y 1, …, x n, y n ) T Texture, g Warp to mean shape

25 Warping texture Problem: –Given corresponding points in two images, how do we warp one into the other? Two common solutions 1.Piece-wise linear using triangle mesh 2.Thin-plate spline interpolation

26 Interpolation using Triangles Region of interest enclosed by triangles. Moving nodes changes each triangle Just need to map regions between two triangles

27 Barycentric Co-ordinates

28 Building Texture Models For each example, extract texture vector Normalise vectors (as for eigenfaces) Build eigen-model Texture, g Warp to mean shape

29 Combined Models Shape and texture often correlated –When smile, shadows change (texture) and shape changes Learning this correlation leads to more compact (and specific) model

30 Combined Appearance Models Varying c changes both shape and texture In this paper:

31 Flexible models Statistical Shape Models Active Shape Models (ASM) Combined Appearance Models Active Appearance Models (AAM)

32 Active Appearance Models Suppose we have a statistical appearance model –Trained from sets of examples How do we use it to interpret new images? Use an “Active Appearance Model” Iterative method of matching model to image

Interpreting Images Place model in image Measure Difference Update Model Iterate

34 Active Appearance Models (AAM) AAM vs. ASM The Active Appearance Model (AAM) is a generalization of the widely used Active Shape Model approach, but uses all the information in the image region covered by the target object, rather than just that near modeled edges.

35 Quality of Match Residual difference: p : all parameters, eg Ideally find and optimize p(p|r) Cannot usually know p(r)

36 Quality of Match Usually attempt to maximize (1) This is equivalent to maximizing (2) Which is equivalent to minimizing (3)

37 Quality of Match Assuming independent Gaussian noise: (1) (2) (3)

38 Quality of Match If we assume all parameters equally likely (within certain limits) (1) Thus we need to find the parameters which minimize the sum of squares of residuals, (2)

39 Learning the Relationship For each of a training set –find best fit given landmarks, p –randomly perturb p by  p and measure (in model frame)

40 More Analytic Approach Taylor expansion: Final result in the paper: where

41 AAM Algorithm Initial estimate I m (p) Start at coarse resolution At each resolution –Measure residual error, r(p) –predict correction  p = -Rr –p  p -  p –repeat to convergence

42 Active Appearance Models (AAM) Example A face model built from 400 images. The figure below shows frames from an AAM search for a new face, each starting with the mean model displaced from the true face centre. Figure: Multi-Resolution search from displaced position

43 Problems Automatic Model Building –Require correspondences across a set –Hard to achieve reliably Reliable measure of quality of fit –Necessary for good matching –Essential for detection Model initialization –Getting good initial estimate can be hard – 10% percent of the image size and scale

44 AAM Summery Parameters An AAM contains a statistical model of the shape and grey-level appearance of the object of interest. Goals Matching to an image involves finding model parameters which minimize the difference between the image and a synthesized model example, projected into the image. The potentially large number of parameters makes this a difficult problem.

45 AAM Summery Iterations We observe that displacing each model parameter from the correct value induces a particular pattern in the residuals. In a training phase, the AAM learns a linear model of the relationship between parameter displacements and the induced residuals. During search it measures the residuals and uses this model to correct the current parameters, leading to a better fit.

46 Some other group’s AAM research Simon Baker and Iain Matthews at CMU are doing some wonderful work on analyzing and improving the AAM update algorithm. They have gone on to develop fast tracking, model building and 3D reconstruction algorithms. Awesome.