-Baljeet Aulakh -Arnold Csok -Jared Shepherd -Amandeep Singh EEC 490 Spring 2012 Kinect Fitness Trainer 1

Project Overview  Recognition of Body Movements and Exercises  Everything is Voice Controlled  Ability to Calculate Velocity And Acceleration  Write Statistics of Movements to an Output File  User Interface Implementation 2

Requirements  Hardware: Kinect + Laptop(2.66Hz above)  Software: Kinect SDK + Microsoft Speech Platform + C# 2010  Environment: Quite Room(voice) + Only one thing moving at a time 3

User Interface 4

Step by Step  Voice Recognition  Detection Algorithms  Velocity and Acceleration  Ability to Write Statistics to an Output file  Documentation and Website 5

Voice Recognition  Allow User To Control Kinect By Voice  Ease of Access to the Program  As Simple as- Start && Stop  Say Which Exercise to Count 6

How Voice Recognition Work  Initialize the Audio Source from the Sensors  Initialize Speech Recognition by Speech Recognizer  Create a Speech Recognition Engine with Exercise Names  Listen to User Speech  Respond to User Speech 7

Problem With Voice Recognition  The First Release of Language Pack Doesn't Have a Reliable Confidence Model  Kinect Tries To Match Every Audio Source It Picks Up  Problem with Matching the Right Exercise Because of This 8

Possible Solutions For Voice Recognition  Test Confidence Interval for Best Accuracy  Use Fitness Trainer in a Quiet Environment  Introduce Noise Cancelation  Use of Headset/Bluetooth  Wait for a More Reliable Language Pack 9

Detection Algorithms  Resources  Recognition algorithms  List of Recognized Body Movements  An Example  Velocity and Acceleration 10

20 Joint Skeleton System  Provided By Microsoft Research Kinect 11

Resources  20 Joint Skeleton System  Each joint gives x, y and z values  Vector Math to find the angle  Timer functionality  Voice recognition functionality  Flag Variables in programming.  Counters to store the repetition. 12

Recognition algorithms  Simple cases: Displacement of the Joints Displacement in the X, Y, Z Direction  Moderate cases: Calculating Angles Between Joints Setting Threshold For Some Angles  Complex cases: Set a Step by Step Routine using all of the above to detect a motion 13

Recognized Exercises  The Following Exercises Are Recognized: 1.Squats 2.Upper Left Punch 3.Upper Right Punch 4.Right Punch 5.Left Punch 6.Right Arm Curl 7.Left Arm Curl 14

Recognized Exercises: Continued 8.Left Kick 9.Right Kick 10.Bowling 11.Hip Abduction 12.Lateral Weight Shift 13.Hamstring Stretch  Counter For All Of These Exercises 15

An example: Bowling motion 16

Bowling final position 17

JOINT STATISTICS  Each exercise has the average acceleration and velocity of all joints calculated.  The statistics are time stamped and saved into a text file.

 Two points  Distance between them  Time to travel between them AVERAGE VELOCITY

 Initial and final position of the joint  The distance formula  A Stopwatch

INITIAL AND FINAL POSITION  Save reference position(point) of skeleton for use in all calculations in defaultPosition[20].  getDisplayPosition(data.Joints[JointID.HipCenter]) ;  getDisplayPosition(data.Joints[JointID.Spine]);  The order is very important

DISTANCE FORMULA  Joints lie on a Cartesian plane  However there is a caveat; The position is measured in pixels  Pixels Are Converted Into Centimeters with the conversion factor of 72 DPI  x2 = (x2 * 2.54) / (72);

TEXT FILE  For every joint returned from the array generated from GetValues  foreach (JointID joint in Enum.GetValues(typeof(JointID)))  JointID Joint = (JointID)i;  When int is typecasted to JointID it returns the name of the joint at that position, NOT A STRING

Display / Interface  The display on the bottom of the screen shows what exercise is being done by the user  The state: Start / Stop  Exercise counter  Exercise to be detected  Exercise that was detected 24

Known Limitations 1.Blocked by an object 2.Overlapping joints 3.Distance from the Kinect Throws Off X, Y, Z coordinates 4.Two People In Front Of Kinect 5.Joints Move During Angle Calculation  These Limitations Make The Accuracy of the Exercise Recognition Difficult  E.g. Crossed arms... 25

Future Approaches  Fix Voice Recognition problem  Have a new accurate Skeleton System  Be able to record and replay exercises  More complex body motions  Introduce more exercises  Find solutions to the Limitations for example a solution to the overlapping joints and blocking objects. 26

Conclusion  Learned about Kinect programming  Use with windows  Use of Kinect in different areas other than gaming  Learned and getting used to C# 27

Website And Facebook Page  Link To Our Webpage: “  Link To Our Facebook Page: “ Fitness-Trainer/ ” 28

Questions????? OR Suggestions? 29

End of Slides 30

Fun Time!!!!! Demonstration!!!!! 31