REAL TIME FACE DETECTION Justin Rilling Pooja Mhapsekar Moinuddin Sayed Ogom J Obinor

Introduction The idea is to be able to detect faces appearing in an image. The faces can be of different sizes and orientations. The motivation behind implementing this on an FPGA is that image processing is inherently parallel in nature and lends itself well to an FPGA.

Viola and Jones’ Detector Key Contributions 1. Integral Image – fast computation of features used by the detector. 2. Combines complex classifiers in “cascade” – focus on facial features.

Integral Image Major contribution of the Viola and Jones detector. Summation of pixel values of the original image. Value at location (x,y) = sum of values of pixels above and to the left of (x,y). Whole image converted to integral image and a window buffer used to scan the entire image.

Haar Classifiers Classifiers are composed of 2-3 rectangles. Weights and sizes associated with features – obtained through AdaBoost. Classifier sum = ∑ (area * weights) Integral image makes area calculation easy. Area = L4–L3–L2+L1. Several classifiers compose a stage.

Stage Sum of all Haar feature classifiers compared with the stage threshold. Multiple stages, each stage has different number of classifiers. Threshold obtained from AdaBoost algorithm.

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 426 542 Right Value 0.8378106 Stage Threshold Stage Sum + < 0.8226894 0.0 Left Value 0.0337941 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 426 542 Right Value 0.8378106 Stage Threshold Stage Sum + < 0.8226894 0.0337941 Left Value 0.0337941 1

Face Detection Example Stage Number Classifier Number 1 Classifier Sum Classifier Threshold < 887 2045 Right Value 0.7488812 Stage Threshold Stage Sum + < 0.8226894 0.0337941 Left Value 0.1514132 1

Face Detection Example Stage Number Classifier Number 1 Classifier Sum Classifier Threshold < 887 2045 Right Value 0.7488812 Stage Threshold Stage Sum + < 0.8226894 0.1852073 Left Value 0.1514132 1

Face Detection Example Stage Number Classifier Number 2 Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 Stage Threshold Stage Sum + < 0.8226894 0.1852073 Left Value 0.0900493 1

Face Detection Example Stage Number Classifier Number 2 Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 Stage Threshold Stage Sum + < 0.8226894 0.2752566 Left Value 0.0900493 1

Face Detection Example Stage Number Classifier Number 2 Classifier Sum Classifier Threshold < 349 568 Right Value 0.6374819 Stage Threshold Stage Sum + < 0.8226894 0.2752566 Left Value 0.0900493 1 Window Buffer Failed Stage 0

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Face Detection Example Stage Number Classifier Number Classifier Sum Classifier Threshold < 0.0 0.0 Right Value 0.0 Stage Threshold Stage Sum + < 0.0 0.0 Left Value 0.0 1

Top Level Design Get Image Get Scaled Image Get Integral Image Window Buffer (21x21) Draw Rectangle If face is det. End of Image? Face Detection Pipeline YES Monitor NO

Face Detection Pipeline

Original End Product USB Webcam FPGA Display USB DVI

Updated End Product FPGA Display DVI Digital Interface 1/3 Color Camera Mod C3188A-6018 Supports several standard image data formats including YCrCb 4:2:2 640 x 480 resolution 5 V signaling 30 fps

Updated End Product XC95144XL CPLD FPGA 5 V Signaling 3.3 V Signaling DVI 1/3 Color Camera Mod C3188A-6018 Supports several standard image data formats including YCrCb 4:2:2 640 x 480 resolution 5 V signaling 30 fps Display

DVI Controller

Questions ??? References: P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, 57(2), 137-154, 2004. Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan Kastner, “FPGA-Based Face Detection System Using Haar Classifiers,” Proceeding of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 103-112, 2009.

OpenCV Program – Lena.jpg

OpenCV Program – ER.jpg

OpenCV Program – ER.jpg Window Buffer (2, 152, 122)

Face Detection Pipeline

OpenCV Program – ER.jpg Window Buffer (3, 165, 20)

Face Detection Pipeline

Questions ???