1. REAL TIME FACE DETECTION
Justin Rilling
Pooja Mhapsekar
Moinuddin Sayed
Ogom J Obinor

2. Introduction
The idea is to be able to detect faces appearing in an image.
The faces can be of different sizes and orientations.
The motivation behind implementing this on an FPGA is that image processing is inherently parallel in nature and lends itself well to an FPGA.

3. Viola and Jones’ Detector
Key Contributions
1. Integral Image – fast computation of features used by the detector.
2. Combines complex classifiers in “cascade” – focus on facial features.

4. Integral Image Major contribution of the Viola and Jones detector.
Summation of pixel values of the original image.
Value at location (x,y) = sum of values of pixels above and to the left of (x,y).
Whole image converted to integral image and a window buffer used to scan the entire image.

5. Haar Classifiers Classifiers are composed of 2-3 rectangles.
Weights and sizes associated with features – obtained through AdaBoost.
Classifier sum = ∑ (area * weights)
Integral image makes area calculation easy. Area = L4–L3–L2+L1.
Several classifiers compose a stage.

6. Stage
Sum of all Haar feature classifiers compared with the stage threshold.
Multiple stages, each stage has different number of classifiers.
Threshold obtained from AdaBoost algorithm.

7. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

8. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
426
542
Right Value
Stage Threshold
Stage Sum +
<
0.0
Left Value 1

9. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
426
542
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1

10. Face Detection Example
Stage Number
Classifier Number 1
Classifier Sum
Classifier Threshold
<
887
2045
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1

11. Face Detection Example
Stage Number
Classifier Number 1
Classifier Sum
Classifier Threshold
<
887
2045
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1

12. Face Detection Example
Stage Number
Classifier Number 2
Classifier Sum
Classifier Threshold
<
349
568
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1

13. Face Detection Example
Stage Number
Classifier Number 2
Classifier Sum
Classifier Threshold
<
349
568
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1

14. Face Detection Example
Stage Number
Classifier Number 2
Classifier Sum
Classifier Threshold
<
349
568
Right Value
Stage Threshold
Stage Sum +
<
Left Value 1
Window Buffer Failed Stage 0

15. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

16. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

17. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

18. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

19. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

20. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

21. Face Detection Example
Stage Number
Classifier Number
Classifier Sum
Classifier Threshold
<
0.0
0.0
Right Value
0.0
Stage Threshold
Stage Sum +
<
0.0
0.0
Left Value
0.0 1

22. Top Level Design Get Image Get Scaled Image Get Integral Image
Window Buffer (21x21)
Draw Rectangle
If face is det.
End of Image?
Face Detection Pipeline
YES
Monitor NO

23. Face Detection Pipeline

24. Original End Product
USB Webcam
FPGA
Display
USB
DVI

25. Updated End Product FPGA Display DVI Digital Interface
1/3 Color Camera Mod C3188A-6018
Supports several standard image data formats including YCrCb 4:2:2
640 x 480 resolution
5 V signaling
30 fps

26. Updated End Product XC95144XL CPLD FPGA 5 V Signaling 3.3 V Signaling
DVI
1/3 Color Camera Mod C3188A-6018
Supports several standard image data formats including YCrCb 4:2:2
640 x 480 resolution
5 V signaling
30 fps
Display

27. DVI Controller

28. Questions ??? References:
P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, 57(2), , 2004.
Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan Kastner, “FPGA-Based Face Detection System Using Haar Classifiers,” Proceeding of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, , 2009.

29. OpenCV Program – Lena.jpg

30. OpenCV Program – ER.jpg

31. OpenCV Program – ER.jpg
Window Buffer (2, 152, 122)

32. Face Detection Pipeline

33. OpenCV Program – ER.jpg
Window Buffer (3, 165, 20)

34. Face Detection Pipeline

35. Questions ???