An Effective Dynamic Gesture Recognition System Based on the Feature Vector Reduction for SURF and LCS PABLO BARROS, NESTOR JÚNIOR, JUVENAL BISNETO, BRUNO FERNANDES, BYRON BEZERRA, SÉRGIO FERNANDES. ESCOLA POLITÉCNICA DE PERNAMBUCO - UNIVERSIDADE DE PERNAMBUCO - BRASIL

RPPDI Dynamic Gesture Recognition Database  Dynamic Gesture  Frame Sequences  Represent one Gesture 

Dynamic Gesture Recognition System  System Architecture  Feature Extraction Module  Classification Module

Extraction Module  Local Contour Sequence – LCS [1]  Speed Upt Robust Features – SURF [2]  Convexity Approach  CLCS  CSURF

Local Contour Sequence - LCS  Algorithm  Identify Hand Shape  Image Segmentation  Contour Detection  Calculate Feature Vector

LCS – Segmentation  Segmentation  OTSU [3]

Contour Identification  Hand Contour Identification

LCS - Local Contour Sequence  Feature Vector Calculation  Find the top first point of the image  Order the points in clockwise.  Calculate distance of a line formed by two points, ℎ [−(−1)⁄2] and ℎ [−(+1)⁄2], and h[i].

Speed Up Robust Features - SURF  Integral Image  Find Interest Points  Describe Interest Points  Intensity  Direction  Descriptors

Convexity Approach  Minimize the hand shape  Douglas-Peucker Algorithm  Apply convex hull  Sklankys Algorithm  Calculate points distances

Convexity Approach  Douglas Peucker Algorithm  Select the two most distant points.  Verify if there is vertex near than a distance T, if there is, remove it.  Recursively do it again with all the points.

Convexity Approach  Sklanky´s Algorithm  Find a convex vertex.  Rename the other vertex in clockwise, starting with p0.  If p0, p1 and p2 turn right:  Put p0 after p2.  Update p0, p1 and p2.  Else:  Put p1 before p0.  Remove p1.  Update p0, p1 and p2.  Repeat until p0 is the initial vertex and p0, p1 and p2 turns right.  For each pair of points draw a line and find the most distant point.

Convexity Local Contour Sequence  Calculate distance  Adaptation of LCS  Use the two external points to draw the line.  Use the inner point to calculate distance.  (a) LCS. (b) SuRF interest points. (c) CLCS. (d) CSURF

Classification Module  Elman Recurrent Neural Network  Hidden Markov Model  Dynamic Time Warping

Results  Convexity Approach  Methodology  Run 30 times  Validation (1/3 for test and 2/3 for training)

Referências  [1] Meena, S A Study on Hand Gesture Recognition Technique. Master’s thesis, National Institute Of Technology, Rourkela,India  [2] Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (Jun 2008),  [3] Bao, J.; Song, A.; Guo, Y.; and Tang, H Dynamic hand gesture recognition based on surf tracking. In Electric Information and Control Engineering (ICEICE), 2011 International Conference on, 338 –341.  [4] N. Otsu. A threshold selection method from gray-level histograms. Systems, Man and Cybernetics, IEEE Transactions on, 9(1):62 –66, jan