Human Gesture Recognition Using Kinect Camera Presented by Carolina Vettorazzo and Diego Santo Orasa Patsadu, Chakarida Nukoolkit and Bunthit Watanapa 1

Introduction This work proposes a comparison of human gesture recognition using data mining classification methods The gestures where chosen to be the knowledge base of a smart home system which monitors and detects the fall motion of the elderly or hospital patients.

Introduction Human gesture ◦ Hands, arms, and body ◦ Movements of the head, face, and eyes Performance of recognition methods ◦ Light conditions ◦ Shadows ◦ Camera angle ◦ Occlusion

The Kinect

The Kinect - depth image A pattern of IR dots is projected from the sensor These dots are detected by the IR camera The dots will change position based on how far the objects are from the source.

The Kinect - depth image

Shotton et al, CVPR(2011)

The Kinect - Skeleton

The Kinect - applications Kinect Gesture Recognition REALTIME Kinect-based Hand Gesture Recognition

The Kinect - applications Rehabilitation. Improvement of athletes performance. Interactive surfaces. 3D modeling. Augmented reality

Methodology Data mining classification ◦ It is the process of extracting valid, previously unseen or unknown, comprehensible information from large databases ◦ Algorithms can involve artificial intelligence, machine learning, statistics, and database systems. z-score normalization ◦ improve the accuracy and efficiency of mining algorithms

Classification Methods In this study, were selected four popular data mining classification method were selected : ◦ Back Propagation Neural Network (BPNN) ◦ Support Vector Machine (SVM) ◦ Decision Tree ◦ Na ї ve Bayes To identify three human gestures: ◦ Stand ◦ Sit down ◦ Lie down

Classification Methods Process of Classification Figure 1: Overview of the proposed system

Classification Methods Process of Classification ◦ 1,200 input vectors for each of the three human gesture classes in input data ◦ 3,600 input vectors (x,y,z) for each distance setting as shown (Stand, Sit down, Lie Down). ◦ 7,200 input vectors in total for both camera distance settings (2m and 3m) ◦ 1,200 vectors for both camera distance settings (2m and 3m) ◦ The output data contain 3,600 vectors in total

Classification Methods Backpropagation Neural Network(BPNN) ◦ BPNN is a multilayer feed forward neural network, which uses backpropagation algorithm in its learning.

Classification Methods Support Vector Machine (SVM) ◦ In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data and recognize patterns

Classification Methods Decision Tree (DT) ◦ Decision Tree is used to classify data from class label

Classification Methods Naïve Bayes (NB) ◦ Is a statistical classification which predicts class membership based on conditional probabilities.

Human Gestures

Results BPNN 100% SVM 99.75% DT 93.19% NB 81.94%

Questions ???