1. Microsoft Kinect How does a machine infer body position?

2. Disclaimer A lot of the information presented comes from reverse engineering performed by a number of agencies. So they might not reflect true ground data, which is proprietary and thus not publicly available. 1

3. Microsoft Kinect  Series of motion sensing input devices for the Xbox  Launched in November 2010  Removes the need of a game controller  User interacts through gestures and spoken commands. 2

4. Two step Process  Compute Depth Map  Apply Machine Learning 3

5. Depth Map? 4

6. So how does it manage something so cool? 5

7. Projects a speckle pattern of Infrared Laser Light Depths Inferred from the distortions seen in the pattern 6

8. ● Depth from Focus ○ What is far will be more blurry ○ Combine different focal lengths for X Axis and Y Axis ● Depth from Stereo ○ Combine image obtained from cameras in different positions ○ The speckle pattern projected from one point, observed from another (Cont) The cool combined with the Classic 7

9. We have the depth. What do we do next? Infer the Body Position, of course 8

10. Body Position Inference  Kinect starts with millions of labelled depth images  Learns a randomized decision forest, mapping depth images to body parts  This RDF takes days to be trained, but the process is made faster using parallel computing 9

11. Body Position Inference  Kinect uses the depth image and the RDF and computes a probability of a pixel belonging to each body part.  Uses mean shift algorithm to select the body parts and sketches a skeleton.  All of these computations are very fast. 10

12. Advantages  Works separately per frame as all the computations are very fast  Runs at 31 fps  Significantly more efficient and robust than body trackers  Works real-time with great results  Handles multiple players  Reported Accuracy: ~72% 11

13. References http://www.i-programmer.info/news/105-artificial-intelligence/2176-kinects-ai-breakthrough- explained.html http://users.dickinson.edu/~jmac/selected-talks/kinect.pdf http://research.microsoft.com/pubs/145347/BodyPartRecognition.pdf 12

14. Thank You Arhan Sibal Rudresh Pande