A Brief Introduction on Face Detection Mei-Chen Yeh 04/06/2010 P. Viola and M. J. Jones, Robust Real-Time Face Detection, IJCV 2004.

Object Detection Find the location of an object if it appear in an image –Does the object appear? –Where is it?

Applications

George colony

Challenges 1: view point variation Michelangelo Slide (5-10) credit: Fei-Fei Li

Challenges 2: illumination slide credit: S. Ullman

Challenges 3: occlusion Magritte, 1957

Challenges 4: scale

Challenges 5: deformation Xu, Beihong 1943

Challenges 6: background clutter Klimt, 1913

Detection Framework Viola and Jones’ face-detection algorithm –The first object detection framework to provide competitive object detection rates in real-time –Implemented in OpenCV Components –Features Haar-features Integral image (speed up the feature calculation) –Learning AdaBoost algorithm –Cascade method (speed up the detection process)

Haar-features The difference between pixels’ sum of the white and black areas Based on the face symmetry

Haar-features Too many features! –Different locations –Different scales Speed-up strategy –Fast calculation of haar-features –Selection of good features 24

Integral image Sum of pixel values in the blue area Example: Image Integral image

1 3 2 ab c d a = sum(1) b = sum(1+2) c = sum(1+3) d = sum( ) Sum(4) = ? 4 d + a – b – c Four-point calculation! A, B: 2 rectangles => C: 3 rectangles => D: 4 rectangles => 6-point 8-point 9-point

A 24x24 detection window Four types of haar features Type A The feature pool

Feature selection A very small number of features can be combined to from an effective classifier! Example: The 1 st and 2 nd features selected by AdaBoost

Feature selection A week classifier h f1f1 f2f2 f 1 Face! f 2 > θ (a threshold) => Not a Face! h = 1 if f i < θ 0 otherwise

Feature selection Idea: Combining several weak classifiers to generate a strong classifier w1w1 w2w2 w3w3 wnwn …… w1h1+w2h2+w3h3+…+wnhnw1h1+w2h2+w3h3+…+wnhn ><>< T a week classifier h 1 = 1 or 0 ~ performance of the weak classifier on the training set

Feature selection Training Dataset –4916 face images –non-face images cropped from 9500 images non-face images positive samplesnegative samples

AdaBoost Each training sample may have different importance! Focuses more on previously misclassified samples –Initially, all samples are assigned equal weights –Weights may change at each boosting round misclassified samples => increase their weights correctly classified samples => decrease their weights

AdaBoost decreased increased fifi Initial weights for each data point -∞∞ misclassified

AdaBoost ……

Learning the classifier Initialize equal weights to training samples For T rounds –normalize the weights –select the best weak classifier in terms of the weighted error –update the weights (set increased weights to misclassified samples) Linearly combine these T week classifiers to form a strong classifier

Detection procedure Scans the detector at multiple locations and scales Sub-window m m m m n n n n Image the detection window

Cascade method Most are non-face images! Rejects negative sub-windows in an early stage Strong Classifier = (W 1 xL 1 + W 2 xL 2 ) + (…)+ (…+ W n xL n ) 123 ><>< T

Summary Feature Extraction Cascade Detection Face Detection

Resources OpenCV Library C codes –apps/haartraining –samples/c Detectors –data/haarcascades

Assignment #3 Due 04/20 11:59pm Build a face detector using the OpenCV resources Send to TA –A picture of yours with detected faces –Subject: Multimedia System Design-Assignment #3-your student id-your name –File name: Assignment #3-your student id-your name For Windows users, you might want to try an older version of OpenCV

Reminder Brainstorm report (a 2-page formal paper) is due on 4/20.