1. Scott Tan Boonping Lau Chun Hui Weng
EE368Group04 Face Detection by color segmentation and template matching
Scott Tan
Boonping Lau
Chun Hui Weng

2. Definition of Problem Multiple face detection in images
With cluttered background, high degree of occlusion
Limited scale and rotational invariance
Constant lighting condition
GOAL: design easily extendable to general image

3. Approach Color segmentation + texture filtering
Multi-resolution face + eyes template matching

4. Face template matching
Input image
For each resolution…
Pre-processing
eyes
template
Eye template matching
Face template matching
face
template
Pair eye/face hits
Color
segmentation
Skin area test
color
mask
face size
mask
Clustering
Clustering
output from
other resolutions
output face
positions

5. Color Segmentation YCbCr color space
Histogram model from pixels under ref*.png masks
Threshold=900 pixels (empirical)
If histo(Cb,Cr)>threshold => skin pixel
Problem:
skin pixel sample base small,
model over-specific !!

6. Color Segmentation (Cont.)
Solution:
Elliptical boundary model
Build more general histogram on larger skin pixel sample base under different conditions
Compromise between performance under test image condition and more general conditions

7. Color Segmentation: Enhancements
Texture filtering
Grayscale variance threshold=1000 (emp.)
If region variance<=threshold => non-skin
Binary operations
Close, fill holes, remove small regions
Aspect ratio test
Unreliable under high degree of occlusion/clutter

8. Color Segmentation Mask Example

9. Template Matching Why? Flexibility of implementation
Extensibility despite limited sample base
Predictability under occlusion

10. Template Matching (Cont.)
Face template built from averaging pre-processed faces
Pre-processing: local mean removed
=> lighting condition invariance
Correlation done in spatial frequency domain on subsampled image and templates
=> reduce processing time
Soft thresholding: mean+n*standard deviation
Post-processing: reject border hits

11. Face+Eyes Template Matching
Face hit marked base of nose
Eyes hit marked between eyes
Pair dilated hits (E hit above F hit)
Advantages:
1) reduce individual false hit rates
2) reduce thresholds: detect under occlusion
3) limited stretch/rotational invariance

12. Face and Eyes Templates

15. Skin area test Removes false and non-centered hits on edge of face
For each hit, take a oval-shaped region around it and count # of skin pixels (from skin segmentation mask)
If less than 75% of skin pixels, then reject
Dependent on quality of segmentation mask

17. Clustering
After skin area test, cluster remaining hits within a certain distance apart (1/2 a face distance)
Hits weighted by correlation
Each cluster represented by a single pixel at centroid => slight translation of detection pixel

19. Multi-resolution Limited scale invariance needed
Use 4 different scales: 90%, 100%, 120%, 130%
Vary image size vs vary template size
Latter can be done ahead of time
Logical OR to collapse multi-resolution template matching results
Clustering algorithm to eliminates hits at different resolutions

22. Results Image detected-false-repeat/total % Results Score Comments
Training_1
20-1-0/21=19/21
90%
19/21
Training_2
23-1-0/24=22/24
(24-0-0/24=24/24)
92%
22(24)/24
occluded face, pixel actually on face if not occluded
Training_3
22-2-0/25=20/25
80%
20/25
Training_4
21-2-0/24=19/24
79%
19/24
Training_5
18-2-0/24=16/24
(20-0-0/24=20/24)
67%
16(20)/24
miss by 1 pixel only, edge of face between 2 png regions
Training_6
22-0-0/24=22/24
22/24
Training_7
22-0-0/22=22/22
100%
22/22
Total
/164=140/164
/164=146/164
85%(89%)

29. Conclusion
Excellent overall results, even with occluded faces, slightly rotated faces, different face sizes etc.
Template matching detects 162/164 faces
Methods are extendable to any general image
Skin color mask is limiting factor