1. Since 2008 Google street view has blurred faces for privacy.

2. The system mostly works well, but does have both false positive and false negatives

3. Let us see if we could solve this problem, with what we have learned in this class.
First, a small insight. Let us solve the skin detection problem, instead of the face detection problem.
What features can we use?
Every color pixel in an image, is represented as a combination of three colors, typically Red Green Blue, but there are other possibilities, HSV (hue, saturation, value), HSL (hue, saturation, lightness/luminance) etc. R B G

4. R B G

5. However, this rule is not perfect (next slide)B G
R =
If we project a skin pixel into 3D space, then you could use the following, human coded rule to classify skin pixels:
If (R > 95) AND (G > 40) AND (B > 20) AND (max{R, G, B} − min{R, G, B} > 15) AND (|R − G| > 15) AND (R > G) AND (R > B)
Then IsSkin
However, this rule is not perfect (next slide)
G =
B =

6. The rule works pretty good here, but it has a lot of false positives, the plate on the wall, the stripy sweater, parts of the couch on the bottom left etc.

7. Let us try to do better, with machine learning.
First we need to get training data to build our classifier.
Recall that for machine learning problems, it can sometimes be useful to transform features. Instead of height and weight, use weight/ height2
Maybe there is a transformation of the color space that makes this problem easer? We could ask an expert, or just compare all color spaces using leave-one-out classification.

8.  In YCbCr, the Y is the brightness (luma), Cb is blue minus luma (B-Y) and Cr is red minus luma (R-Y).
Y = 50
100
150
200
250 Cr
Cb = Cb
Cr =

9. Cr > 124? Cr < 174? Not Skin Cb < 90? Not Skin Cb > 140?50
100
150
200
250 Cr
Cr < 174?
Not Skin Cb
Cb < 90?
Not Skin
Cb > 140?
Not Skin
Skin
Not Skin

10. 50
100
150
200
250 Cr Cb
RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

11. This is a hand coded piecewise linear model
Cr ≤ × Cb + 20
Cr ≥ × Cb
Cr ≥ × Cb
Cr ≤ × Cb
Cr ≤ × Cb 50
100
150
200
250 Cr Cb
RGB-H-CbCr Skin Colour Model for Human Face Detection.Nusirwan Anwar Abdul Rahim, Kit Chong Wei, John See.MMU International Symposium on Information & Communications Technologies (M2USIC 2006),

12. A simple nearest neighbor algorithm in the YCbCr space works very well.
How could we make it better?

13. Extensions
We can use basic classifiers as a building blocks for more complex questions
For example, if we wanted to find examples of FedEx trucks on streetview, we could build a classifier for each color in the logo (Purple, Orange and White).
If a small area of the image had all three colors, we could classify truck seen.
Why not just “read” the text?
Consider the below…

15. Less we get too complacent I
How well could you do on this problems? Google “Karen Zack” for more examples

16. Less we get too complacent II
How well could you do on this problems?