Triangle-based approach to the detection of human face March 2001 PATTERN RECOGNITION Speaker Jing. AIP Lab

Outline  Introduction  Segmentation of potential face regions  Face verification  Experimental results and discussion

Introduction 1/3 Given a still or video image, detect and localize an unknown number of faces –Security mechanism (replace key, card,passwd) –Criminology (find out possible criminals) –Content-based image retrieval –video coding –video conferencing –Crowd( 大眾 ) surveillance and intelligent human-computer interfaces. Applications Problem

Introduction 2/3 Requirement * achieve the task regardless of - illumination, orientation, and camera distance Why difficult ? Human face is a dynamic object High degree of variability in appearance ( 面孔的多變性 ) * Speedy and correct detection rate

Introduction 3/3  Drawbacks of the papers until now –Free of background –Cannot detect a small face ( < 50 * 50) –Cannot detect multiple face ( >3) –Cannot handle the defocus and noise –Cannot conquer the partial occlusion of mouth or wear sunglasses –Cannot detect a face of side view

A classified algorithms

Begin the method

Overview of the system 1. Form 4-connected components 2. Find the center for each one 1. Search any 3 center that form an isosceles or right triangle 1. Normalize the size of potential face regions 1. Calculate the weight by mask function

Segmentation  4 step for segmenting the potential face –Convert the input image to a binary image –Find the blocks using 4-connected component –Search the triangle –Clip the satisfy triangle region

Step1: Convert the image  RGB Color Image –Eliminating the hue and saturation –Gray-level  binary image –Remove noise using opening operation –Eliminate holes by the closing operation Gray-level < T are labelled as black Gray-level > T are white

Step 2: Form the blocks & Searching triangle  Form the blocks by using 4-connected components algorithm  Locate the center of each block  Searching the triangle –Frontal view (isosceles triangle) –Side view (right triangle)

Step 3: Frontal view (isosceles triangle)  Isosceles triangle: D(ij)=D(jk)  Matching rule: i k j Eye to mouth mouth to mouth a b c

Clipping the region 2/4 X1=X4=Xi – 1/3 d X2=X3=Xk + 1/3 d Y1=Y2=Yi + 1/3 d Y3=Y4=Yj – 1/3 d Xi,Yi d Xk,Yk Xj,Yj x1 x2

Side view (right triangle) 3/4  Right triangle  Matching Rules: (25% derivation) a < | a-c | < 0.6 a a < | a-b | < 0.19 a a < | b-c | < 0.44 a ij k 2 1 a b c

Clipping the region 4/4 i j k d 1.2d d/4 d d/6 X1=X4=Xi-d/6 X2=X3=Xi+1.2d Y1=Y2=Yi+d/4 Y3=Y4=Yi-d

Speedup of searching  How many triangles ?  If the mouth & right eye are already known, => the left eye should be located in the near area. i j k

Face verification  3 steps in verification Step1: Normalization the potential facial areas –60 * 60 pixels Step 2: Calculating the weight by masking function Step 3 :Verification by thresholding the weight Question 1. How to generate the face mask ? Question 2. How to calculate the weight ?

Question 1. How to generate the face mask ?  Read the 10 binary training masks  Add the corresponding entries  Binarized the added mask Ex:

Question 2. How to calculate the weight  Eye and mouth are labeled as black, others as white –If the pixels in the P is equal to T Both Black: Weight + 6 Both White : Weight + 2 –White in P and black in T Weight –2 –White in T and black in P Weight - 4 P: potential facial region T: Training mask

Verification  For each potential facial regions –Threshold value is given for decision making Front view => 4000 < threhold < 5500 Side view => 2300 < threhold < 2600  Finally, eliminate the regions that –Overlap with the chosen facial region

Result—frontal view Original BinaryIsosceles triangle clipping Normalized

Result – Side View Original BinaryIsosceles triangle clipping Normalized

Experimental results  500 test images – included 450 different persons –600 faces that are used  11 faces cannot be found correctly  98% success rate

Experiment result  Scaling : 5*5 to 640*480  Light condition

Experiment Result  Distinct position  Defocus face

Experiment Result  Changed expressions

Experiment Result NoiseOcclusionSunglasses cartoonChinese doll

Experiment Result 2.5 sec28 sec Target machine: PII 233 PC

Experiment Result Multi-faces and video stream

Experiment Result False cases Too DarkRight eye being occluded

Conclusion  Manage different sizes, changed light conditions, varying pose and expression  Cope with partial occlusion problem  Detect a side-view face  In the future, using this algorithm for solving face recognition problem

My opinions  The processing time depend on the complexity of the image.  Real-time requirement was unachievable. (some images need 28 sec to process)