1. -Baljeet Aulakh -Arnold Csok -Jared Shepherd -Amandeep Singh EEC 490 Spring 2012 Kinect Fitness Trainer 1

2. Project Overview  Recognition of Body Movements and Exercises  Everything is Voice Controlled  Ability to Calculate Velocity And Acceleration  Write Statistics of Movements to an Output File  User Interface Implementation 2

3. Requirements  Hardware: Kinect + Laptop(2.66Hz above)  Software: Kinect SDK + Microsoft Speech Platform + C# 2010  Environment: Quite Room(voice) + Only one thing moving at a time 3

4. User Interface 4

5. Step by Step  Voice Recognition  Detection Algorithms  Velocity and Acceleration  Ability to Write Statistics to an Output file  Documentation and Website 5

6. Voice Recognition  Allow User To Control Kinect By Voice  Ease of Access to the Program  As Simple as- Start && Stop  Say Which Exercise to Count 6

7. How Voice Recognition Work  Initialize the Audio Source from the Sensors  Initialize Speech Recognition by Speech Recognizer  Create a Speech Recognition Engine with Exercise Names  Listen to User Speech  Respond to User Speech 7

8. Problem With Voice Recognition  The First Release of Language Pack Doesn't Have a Reliable Confidence Model  Kinect Tries To Match Every Audio Source It Picks Up  Problem with Matching the Right Exercise Because of This 8

9. Possible Solutions For Voice Recognition  Test Confidence Interval for Best Accuracy  Use Fitness Trainer in a Quiet Environment  Introduce Noise Cancelation  Use of Headset/Bluetooth  Wait for a More Reliable Language Pack 9

10. Detection Algorithms  Resources  Recognition algorithms  List of Recognized Body Movements  An Example  Velocity and Acceleration 10

11. 20 Joint Skeleton System  Provided By Microsoft Research Kinect 11

12. Resources  20 Joint Skeleton System  Each joint gives x, y and z values  Vector Math to find the angle  Timer functionality  Voice recognition functionality  Flag Variables in programming.  Counters to store the repetition. 12

13. Recognition algorithms  Simple cases: Displacement of the Joints Displacement in the X, Y, Z Direction  Moderate cases: Calculating Angles Between Joints Setting Threshold For Some Angles  Complex cases: Set a Step by Step Routine using all of the above to detect a motion 13

14. Recognized Exercises  The Following Exercises Are Recognized: 1.Squats 2.Upper Left Punch 3.Upper Right Punch 4.Right Punch 5.Left Punch 6.Right Arm Curl 7.Left Arm Curl 14

15. Recognized Exercises: Continued 8.Left Kick 9.Right Kick 10.Bowling 11.Hip Abduction 12.Lateral Weight Shift 13.Hamstring Stretch  Counter For All Of These Exercises 15

16. An example: Bowling motion 16

17. Bowling final position 17

18. JOINT STATISTICS  Each exercise has the average acceleration and velocity of all joints calculated.  The statistics are time stamped and saved into a text file.

19.  Two points  Distance between them  Time to travel between them AVERAGE VELOCITY

20.  Initial and final position of the joint  The distance formula  A Stopwatch

21. INITIAL AND FINAL POSITION  Save reference position(point) of skeleton for use in all calculations in defaultPosition[20].  getDisplayPosition(data.Joints[JointID.HipCenter]) ;  getDisplayPosition(data.Joints[JointID.Spine]);  The order is very important

22. DISTANCE FORMULA  Joints lie on a Cartesian plane  However there is a caveat; The position is measured in pixels  Pixels Are Converted Into Centimeters with the conversion factor of 72 DPI  x2 = (x2 * 2.54) / (72);

23. TEXT FILE  For every joint returned from the array generated from GetValues  foreach (JointID joint in Enum.GetValues(typeof(JointID)))  JointID Joint = (JointID)i;  When int is typecasted to JointID it returns the name of the joint at that position, NOT A STRING

24. Display / Interface  The display on the bottom of the screen shows what exercise is being done by the user  The state: Start / Stop  Exercise counter  Exercise to be detected  Exercise that was detected 24

25. Known Limitations 1.Blocked by an object 2.Overlapping joints 3.Distance from the Kinect Throws Off X, Y, Z coordinates 4.Two People In Front Of Kinect 5.Joints Move During Angle Calculation  These Limitations Make The Accuracy of the Exercise Recognition Difficult  E.g. Crossed arms... 25

26. Future Approaches  Fix Voice Recognition problem  Have a new accurate Skeleton System  Be able to record and replay exercises  More complex body motions  Introduce more exercises  Find solutions to the Limitations for example a solution to the overlapping joints and blocking objects. 26

27. Conclusion  Learned about Kinect programming  Use with windows  Use of Kinect in different areas other than gaming  Learned and getting used to C# 27

28. Website And Facebook Page  Link To Our Webpage: “http://www.baljeetaulakh.com”  Link To Our Facebook Page: “https://www.facebook.com/pages/Kinect- Fitness-Trainer/236918233072748” 28

29. Questions????? OR Suggestions? 29

30. End of Slides 30

31. Fun Time!!!!! Demonstration!!!!! 31