1. Estimation of Skin Color Range Using Achromatic Features
Wen-Hung Liao
Department of Computer Science
National Chengchi University
November 27, 2008

2. Outline Motivation and Related Work Color Spaces
Fixed vs. Dynamic Range Approach
Experimental Results
Skin color segmentation
Hand & finger detection
Conclusion

3. Background
Previous claims: skin color is restricted to a “fixed” range in certain color coordinates:
Sobottka & Pitas: Hue:[0,50º], Saturation:[0.23,0.68]
Chai & Ngan: Cb:[77,127], Cr[137,177]
Kawato & Ohya: Decision boundary in normalized RGB space

4. Decision Boundary in Normalized RGB Space

5. Sobottka & Pitas: Fixed Hue + Saturation

6. Chai & Ngan: Fixed Cb,Cr

7. Kawato & Ohya

8. Comparative Analysis
From: Phung et al, Skin segmentation using color pixel classification: analysis and comparison, IEEE Transactions on PAMI, 2005.

9. Observation
It is true that the skin color lies in a small range, yet this range tends to shift under different lighting conditions.
Question: Is it possible to dynamically adjust the range of skin color to enhance the robustness of color-based segmentation?

10. The Proposed Solution
Use achromatic information (face detection) to help determine the range.
Limitation:
Face must be present and detected.
Suitable for vision-based human computer interface.

11. Five Classes of Color Space
Representative color space
Basic color spaces
RGB、normalized RGB
Perceptual color spaces
HSV、HIS
Orthogonal color spaces
YCbCr、YUV
Perceptually uniform color spaces
CIELab、CIELuv
Other color spaces
Mixture

12. Color Spaces Investigated
domains
RGB
Red、Green、Blue
HSV
Hue、Saturation、Value
CIELab
L、a、b
YCbCr
Y、Cb、Cr
CIELuv
L、u、v
* Dynamically set the threshold in Hue domain

13. Determining the Threshold (I)
Step 1: detecting and locating the face
Step 2: mark the cheek area
X = X0 +(W0 /5)
Y = Y0 +(H0 /2)
width = W0 /5
height = H0 /5
Step 3: obtain the hue distribution of the marked area. W0
(X0, Y0) H0

14. Determining the Threshold (II)
Step 4: assume that the histogram is peaked at A:
search to the left and right of A until
Local minimum <A/10 is uncovered
A non-zero global minimum is found
255

15. Face Detection using DSE
Directional Sobel Edges

16. Experiment: Skin Color Segmentation
Compare the performance of 5 different methods:
Dynamic threshold
Fixed threshold – fixed Hue
Kawato & Ohya – fixed Normalized RGB
Sobottka & Pitas – fixed Hue & Saturation
Chai & Ngan – fixed Cb & Cr
Material
Images captured by a low-cost webcam under different lighting conditions.
A total of 400 images (taken indoor) are manually segmented and labeled.

17. Skin Color Segmentation: Experimental Results
false positive
false negative
true negative
true positive
Dynamic Threshold
0.0736
0.1706
0.9264
0.8294
fixed Hue
0.2125
0.3361
0.7875
0.6639
fixed Normalized RGB
0.0504
0.5303
0.9496
0.4697
fixed Hue & Sat
0.0588
0.5747
0.9412
0.4253
fixed Cr & Cb
0.0857
0.2996
0.9143
0.7004

18. Best and Worst Case Performance
best TP
worst TP
Dynamic Threshold
0.9947
0.3494
fixed Hue
0.9977
0.0733
fixed Normalized RGB
0.9055
0.0002
fixed Hue & Sat
0.8891
0.0005
fixed Cr & Cb
0.9447
0.2234

19. Recall and Precision
Recall = TP/(TP+FP)
Precision = TP/(TP+FN)

20. Speed-up the Process 1. Detecting Face (After K frames)
2. Record color distribution
of cheek area
3. Tracking face
4. Local search
5. Update color distribution

21. Performance Improvement

22. Experiment: Hand Detection
Color-based hand segmentation
No post-processing
Does not involve statistical modeling and classifier

23. Plamar vs. Dorsal Side
Hue histogram
Hue histogram

24. Hand Detection: Experimental Results
Dorsal side
Dorsal side (fingers)
Plamar side
Plamar side (fingers)
Accuracy
92.65%
94.26%
90.78%
95.01% 24

25. Fingertip Detection 150 images # of fingers detected Dynamic threshold
Fixed Threshold 5
108
72% 17
11% 4 21
14% 22
15% 3 10 7% 23 2 3% 20
13% 1 1% 48
33%

26. Conclusion
Perform comparative evaluation of several color-based segmentation methods.
Propose and implement a dynamic range estimation algorithm using achromatic features.
Superior performance in terms of skin-color segmentation, hand and finger detection.
Suitable for vision-based HCI.

27. Thank you
Q & A

28. Experimental Result
Dynamic Threshold
worst TP

29. Experimental Result
Fixed Hue
worst TP 29

30. Experimental Result
Fixed Normalized RGB
worst TP 30

31. Experiment Result
Fixed Hue & Saturation
worst TP 31

32. Experiment Result
Fixed Cb & Cr
worst TP 32

33. Recall = TP/(TP+FP)
Precision = TP/(TP+FN)