Facial Features Extraction Amit Pillay Ravi Mattani Amit Pillay Ravi Mattani

What Are We Doing !  Finding Features on a Face  Eyes  Mouth  Nose  Finding Features on a Face  Eyes  Mouth  Nose

Why Facial Feature Extraction?  Biometrics  Facial recognition system  Video Surveillance  Human Computer Interface  Biometrics  Facial recognition system  Video Surveillance  Human Computer Interface

Difficulties !  Face Variation  Physical characteristic vary  Non-uniform lighting  Face position  Face Variation  Physical characteristic vary  Non-uniform lighting  Face position

Previous Work.  Many Face Extraction Methods  Main Trend : Combine image information and knowledge of face  Ian-Gang Wang, Eric sung in their article have proposed a morphological procedure to analyze the shape of segmented face region. Several rules have been formulated for the task of locating the contour of the face.  Terrillon et al., 1998 mentions the problem of how other body parts such as neck may lead to face localization error  Haalick and Shapiro, 1993 demonstrate how morphological operations can simplify the image data while preserving their essential shape characteristics and can eliminate irrelevances.  Many Face Extraction Methods  Main Trend : Combine image information and knowledge of face  Ian-Gang Wang, Eric sung in their article have proposed a morphological procedure to analyze the shape of segmented face region. Several rules have been formulated for the task of locating the contour of the face.  Terrillon et al., 1998 mentions the problem of how other body parts such as neck may lead to face localization error  Haalick and Shapiro, 1993 demonstrate how morphological operations can simplify the image data while preserving their essential shape characteristics and can eliminate irrelevances.

Our Process

Skin Segmentation  Depends on color space  Used the finding by Yang & Waibel(1995,1996)  Normalized r-g color plane.  Took seed pixel  Classified the pixels based on whether the pixel lies within the threshold  Same process carried out for the R and G plane  Depends on color space  Used the finding by Yang & Waibel(1995,1996)  Normalized r-g color plane.  Took seed pixel  Classified the pixels based on whether the pixel lies within the threshold  Same process carried out for the R and G plane

Skin color segmentation

Morphological Image Processing  Dilation  Fills the holes  Erosion  Restores the shape of the face  Dilation  Fills the holes  Erosion  Restores the shape of the face

Morphological Image Processing

Skeletonization  Reduces binary image objects to a set of thin strokes.  Retains important information about the shape of the original object  Reduces binary image objects to a set of thin strokes.  Retains important information about the shape of the original object

Skeletonization

Contour Tracing  Certain vertices of these skeleton lines called fitting points can fit the contour of the human face.  Certain rules are then applied to deduce these fitting points by analyzing the skeleton lines.  Certain vertices of these skeleton lines called fitting points can fit the contour of the human face.  Certain rules are then applied to deduce these fitting points by analyzing the skeleton lines.

Contour Tracing  Rule 1 - The contour fitting points should be the vertices of the roughly horizontal skeleton line segments that are long enough.  Rule 2 - The left vertex will be selected as candidate for contour fitting if most of the horizontal line segments are positioned at the left of the symmetry axis and vice versa  Rule 3 - The contour points should be above a vertical position that is set at 3/4 of the height from the top of the symmetry axis  Rule 4 - The point set satisfying the above will be doubled using symmetry axis  Rule 1 - The contour fitting points should be the vertices of the roughly horizontal skeleton line segments that are long enough.  Rule 2 - The left vertex will be selected as candidate for contour fitting if most of the horizontal line segments are positioned at the left of the symmetry axis and vice versa  Rule 3 - The contour points should be above a vertical position that is set at 3/4 of the height from the top of the symmetry axis  Rule 4 - The point set satisfying the above will be doubled using symmetry axis

Contour Tracing

ROI

Feature extraction within the ROI  Edge Detection using Sobel Operator  Vertical position by horizontal integral projection  Lip line maximizes the projection  Bounded by a rectangular box  Same process is repeated for nose and eyes regions within the fixed vertical positions  Edge Detection using Sobel Operator  Vertical position by horizontal integral projection  Lip line maximizes the projection  Bounded by a rectangular box  Same process is repeated for nose and eyes regions within the fixed vertical positions

Results

Conclusion  No. of images experimented with = 30  No. of images in which features are correctly identified = 27  Percentage correctly identified = 90  Average time taken to get the output in MATLAB = secs  No. of images experimented with = 30  No. of images in which features are correctly identified = 27  Percentage correctly identified = 90  Average time taken to get the output in MATLAB = secs

Future Work  More robust and dynamic  Extended for profile views of image  More efficient code for faster execution (applicable especially for MATLAB !!!)  More robust and dynamic  Extended for profile views of image  More efficient code for faster execution (applicable especially for MATLAB !!!)

References  Frontal-view face detection and facial feature extraction using color and morphological operations by Jian-Gang Wang, Eric Sung  A Model-Based Gaze Tracking System by Rainer Stiefelhagen, Jie Yang, Alex Waibel  Digital Image Processing Using MATLAB by Gonzalez, Woods &Eddins,Prentice  Images taken from  Prof. Gaborski’s lecture slides   Frontal-view face detection and facial feature extraction using color and morphological operations by Jian-Gang Wang, Eric Sung  A Model-Based Gaze Tracking System by Rainer Stiefelhagen, Jie Yang, Alex Waibel  Digital Image Processing Using MATLAB by Gonzalez, Woods &Eddins,Prentice  Images taken from  Prof. Gaborski’s lecture slides 

Questions???