1. A Recognition Method of Restricted Hand Shapes in Still Image and Moving Image Hand Shapes in Still Image and Moving Image as a Man-Machine Interface Speaker : Meng-Shun Su Adviser : Chih-Hung Lin Ten-Chuan Hsiao Ten-Chuan Hsiao Date : 2010/03/23 ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Human System Interactions, 2008 Conference on Nobuharu Yasukochi, Aya Mitome, and Rokuya Ishii, Fellow, IEEE Yokohama National University, Yokohama, Japan

2. Outline Experimental Results Experimental Results 33 Introduction Introduction 11 Method Description Method Description 22 Conclusions Conclusions 44 Hand Region Extraction Hand Region Extraction Hand Shape Recognition Hand Shape Recognition ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 2

3. Introduction Introduction 11 ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Some interfaces employ to wear magnetic sensor, use infrared camera, multi-cameras, and special input device with marker and grove. expensive & complicated Those interfaces are very expensive & complicated, and not suitable for everyone's use. 3

4. Introduction Introduction 11 ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Featuring a simple user interface, this paper presents a simple recognition algorithm of restricted hand shapes from an image taken by only a (not multi-) camera. hand region extraction process from an input imagehand shape recognition process from the extracted image. The proposed method can be divided into two parts: one is the hand region extraction process from an input image; another is the hand shape recognition process from the extracted image. 4

5. Introduction Introduction 11 ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 In the hand shape recognition process, we first make a mask image from the extracted hand region, and we recognize hand shapes based on the image with uneven hand surface by using the mask. The effectiveness of the proposed method is evaluated by recognition success rate and computation time. 5

6. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Region Extraction Hand Region Extraction RGB values of hand region and grayscale background region have the following relationship. In hand region. In background region. 6

7. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Region Extraction Hand Region Extraction Therefore, an input image (RGB: 256 values) is transformed into (511 values) 7

8. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Region Extraction Hand Region Extraction Fig.1.Distribution of reference images in RGB color space. Calculate skin color vector. is determined such that 95% of skin color pixels. 8

9. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 For recognition of finger shapes, we employ the luminosity Value in HSV color space. The value R for each pixel in a recognized image is calculated. = 門檻 = HSV 色彩空間亮度值 A. Marking a mask image Hand Shape Recognition Hand Shape Recognition 9

10. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Fig.3.Distribution of values R for pixels in recognized hand shape image Hand Shape Recognition Hand Shape Recognition 10

11. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Shape Recognition Hand Shape Recognition B. Normalization of values of pixels in a hand shape region 1)Position normalization 2)Angle normalization 3)Eliminating a wrist region 4)Normalization (size) 11

12. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Shape Recognition Hand Shape Recognition C. Hand Shape Recognition Algorithm Here we prepare an ellipse curve that crosses all the fingers. The number of fingers can be detected from the pixel value R on the ellipse curve. Then hand shapes can be recognized by the angle of fingers from List point. 12

13. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Method Description Method Description 22 Hand Shape Recognition Hand Shape Recognition C. Hand Shape Recognition Algorithm Fig.5.Recognition algorithm. 13

14. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Experimental Results Experimental Results 33 A. Recognition Rate We evaluate the recognition rate. Recognition ratio is given by: 14

15. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Experimental Results Experimental Results 33 In the case of the grayscale background, average recognition rate was 96.8%. Note that the recognition cannot be successful when the background contains skin color. That problem still remains as one of future studies. The recognition rate by the method of Ref. [3] was 87.9% for grayscale, and 83.1% for color images. The proposed algorithm achieves higher recognition rate, the reason would be that Ref. [3] did not employ normalization process. 15

16. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Experimental Results Experimental Results 33 B. Processing Speed we see that the total recognition processing time is about 30[ms] at most, which can be considered as fast enough processing time to be used as hand shape recognition system. 16

17. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1 Conclusions Conclusions 44 The processing time was around several 10 of milliseconds which can be regarded enough to recognize a hand shape. So, this method enables real-time hand shape recognition. The present algorithm could recognize 9 hand shapes at the accuracy of 96.8% for the case of grayscale backgrounds. Cannot be successful when the background contains skin color. 17

18. ©2010 STUT. CSIE. Multimedia and Information Security Lab. J205-1