Scott Tan Boonping Lau Chun Hui Weng EE368Group04 Face Detection by color segmentation and template matching Scott Tan Boonping Lau Chun Hui Weng

Definition of Problem Multiple face detection in images With cluttered background, high degree of occlusion Limited scale and rotational invariance Constant lighting condition GOAL: design easily extendable to general image

Approach Color segmentation + texture filtering Multi-resolution face + eyes template matching

Face template matching Input image For each resolution… Pre-processing eyes template Eye template matching Face template matching face template Pair eye/face hits Color segmentation Skin area test color mask face size mask Clustering Clustering output from other resolutions output face positions

Color Segmentation YCbCr color space Histogram model from pixels under ref*.png masks Threshold=900 pixels (empirical) If histo(Cb,Cr)>threshold => skin pixel Problem: skin pixel sample base small, model over-specific !!

Color Segmentation (Cont.) Solution: Elliptical boundary model Build more general histogram on larger skin pixel sample base under different conditions Compromise between performance under test image condition and more general conditions

Color Segmentation: Enhancements Texture filtering Grayscale variance threshold=1000 (emp.) If region variance<=threshold => non-skin Binary operations Close, fill holes, remove small regions Aspect ratio test Unreliable under high degree of occlusion/clutter

Color Segmentation Mask Example

Template Matching Why? Flexibility of implementation Extensibility despite limited sample base Predictability under occlusion

Template Matching (Cont.) Face template built from averaging pre-processed faces Pre-processing: local mean removed => lighting condition invariance Correlation done in spatial frequency domain on subsampled image and templates => reduce processing time Soft thresholding: mean+n*standard deviation Post-processing: reject border hits

Face+Eyes Template Matching Face hit marked base of nose Eyes hit marked between eyes Pair dilated hits (E hit above F hit) Advantages: 1) reduce individual false hit rates 2) reduce thresholds: detect under occlusion 3) limited stretch/rotational invariance

Face and Eyes Templates

Skin area test Removes false and non-centered hits on edge of face For each hit, take a oval-shaped region around it and count # of skin pixels (from skin segmentation mask) If less than 75% of skin pixels, then reject Dependent on quality of segmentation mask

Clustering After skin area test, cluster remaining hits within a certain distance apart (1/2 a face distance) Hits weighted by correlation Each cluster represented by a single pixel at centroid => slight translation of detection pixel

Multi-resolution Limited scale invariance needed Use 4 different scales: 90%, 100%, 120%, 130% Vary image size vs vary template size Latter can be done ahead of time Logical OR to collapse multi-resolution template matching results Clustering algorithm to eliminates hits at different resolutions

Results Image detected-false-repeat/total % Results Score Comments Training_1 20-1-0/21=19/21 90% 19/21   Training_2 23-1-0/24=22/24 (24-0-0/24=24/24) 92% 22(24)/24 occluded face, pixel actually on face if not occluded Training_3 22-2-0/25=20/25 80% 20/25 Training_4 21-2-0/24=19/24 79% 19/24 Training_5 18-2-0/24=16/24 (20-0-0/24=20/24) 67% 16(20)/24 miss by 1 pixel only, edge of face between 2 png regions Training_6 22-0-0/24=22/24 22/24 Training_7 22-0-0/22=22/22 100% 22/22 Total 148-8-0/164=140/164 151-5-0/164=146/164 85%(89%)

Conclusion Excellent overall results, even with occluded faces, slightly rotated faces, different face sizes etc. Template matching detects 162/164 faces Methods are extendable to any general image Skin color mask is limiting factor