Since 2008 Google street view has blurred faces for privacy.

The system mostly works well, but does have both false positive and false negatives

Let us see if we could solve this problem, with what we have learned in this class. First, a small insight. Let us solve the skin detection problem, instead of the face detection problem. What features can we use? Every color pixel in an image, is represented as a combination of three colors, typically Red Green Blue, but there are other possibilities, HSV (hue, saturation, value), HSL (hue, saturation, lightness/luminance) etc. R B G

R B G

However, this rule is not perfect (next slide) B G R = If we project a skin pixel into 3D space, then you could use the following, human coded rule to classify skin pixels: If (R > 95) AND (G > 40) AND (B > 20) AND (max{R, G, B} − min{R, G, B} > 15) AND (|R − G| > 15) AND (R > G) AND (R > B) Then IsSkin However, this rule is not perfect (next slide) G = B =

The rule works pretty good here, but it has a lot of false positives, the plate on the wall, the stripy sweater, parts of the couch on the bottom left etc.

Let us try to do better, with machine learning. First we need to get training data to build our classifier. Recall that for machine learning problems, it can sometimes be useful to transform features. Instead of height and weight, use weight/ height2 Maybe there is a transformation of the color space that makes this problem easer? We could ask an expert, or just compare all color spaces using leave-one-out classification.

 In YCbCr, the Y is the brightness (luma), Cb is blue minus luma (B-Y) and Cr is red minus luma (R-Y). Y = 50 100 150 200 250 Cr Cb = Cb Cr =

Cr > 124? Cr < 174? Not Skin Cb < 90? Not Skin Cb > 140? 50 100 150 200 250 Cr Cr < 174? Not Skin Cb Cb < 90? Not Skin Cb > 140? Not Skin Skin Not Skin

50 100 150 200 250 Cr Cb RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

This is a hand coded piecewise linear model Cr ≤ 1.5862 × Cb + 20 Cr ≥ 0.3448 × Cb + 76.2069 Cr ≥ -4.5652 × Cb + 234.5652 Cr ≤ -1.15 × Cb + 301.75 Cr ≤ -2.2857 × Cb + 432.85 50 100 150 200 250 Cr Cb RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

A simple nearest neighbor algorithm in the YCbCr space works very well. How could we make it better?

Extensions We can use basic classifiers as a building blocks for more complex questions For example, if we wanted to find examples of FedEx trucks on streetview, we could build a classifier for each color in the logo (Purple, Orange and White). If a small area of the image had all three colors, we could classify truck seen. Why not just “read” the text? Consider the below…

Less we get too complacent I How well could you do on this problems? Google “Karen Zack” for more examples

Less we get too complacent II How well could you do on this problems?