Capturing, Encoding and Rendering Gestures using the Kinect Adan Patience Supervisor: Mr. Merhdad Ghaziasgar Co-Supervisor: Mr. R Dodds Mentor: Mr. Kenzo Abrahams

Overview ▪ Recap ▪ Proposed User Interface ▪ Object Orientated Analysis – Capture – Encode – Render – Class Relationships ▪ Object Orientated Design ▪ Project Plan ▪ Demo

Recap Introduction ▪ Develop a video chatting application using the Kinect, that could possibly be used extensively by the deaf in the near futures. ▪ Reasons – Slow internet – Cost of data ▪ Thus current videos chatting application are not viable options for conveying or receiving gestures during the conversation as per the limitations above.

User Interface

Object Orientated Analysis ▪ Capture Class – Extract meaningful information from the visual input – Use Skeletal Tracking to locate and track joints of the person of interest ▪ Encode Class – Encoding the co-ordinates of the joints into a more readable format.CSV ▪ Render Class – Function is to animate the Blender avatar using the encoded.CSV file – Mapping Kinect co-ordinate system onto the Blender avatar

Object Orientated Analysis ▪ Relationships between the Capture, Encode and Render Classes Includes

OOD of the Capture Class Attributes struct s k el e t o n Methods def u p d a t e _s k el e t o n : def p r o c e s s s_ k e l e t o n : def d r a w_ s k el e t o n :

OOD of the Encode Class Attributes struct s k el e t o n Methods def g e t _j o i n t s ( s k e l e t o n ) : def p r i n t _j o i n t s ( v e c t o r ) :

OOD of the Render Class Attributes floats x, y, z Methods def r e a d_ f i l e : def t r a n s f o rm ( f l o a t x, y, z): def a n im a te ( f l o a t x, y, z ) :

Project Plan Term1 Identify User Requirements, Analyse User Requirements, Literature Survey Become Acquainted with OpenCV and C++ Term2 Identify Objects and Classes in the OOD and analyse the Class’ methods and attributes, Design UI and Choose the Render Application Familiarise myself with the Kinect SDK and API Term3 Create the Capture and Encode Classes leveraging from the Kinect Develop the mapping function. Create UI and finalise application Term4 Test the application against the User Requirements Optimise where possible and iron out bugs