1. Human pose recognition from depth image MS Research Cambridge

2. Goals Classify pixels to body parts categories Input is based on single depth image from Kinect

3. Claim contributions Fast classification - speed up to 200 frames/ second on Xbox 360 GPU implementation High accuracy on both synthetic and real world dataset

4. Methods Based on Randomized forest (bunch of random decision trees) At the leaf node in tree t, learned distribution P(c | x) over part labels c is stored (x = testing pixel) Final classification result is the average value.

5. Randomized forest

6. Features For pixel X, define features as d I (x) is depth intensity on pixel x u,v are position offset, only 2 learned parameters Intuitively, the features represent random derivative value on 2D space

7. Features

8. Training randomized forest Each decision tree is trained separately, using different disjoint training sets Parameter (u, v, t) is associated with each node  u,v are offset, t = decision threshold Proposed a randomly selected set of parameters Follow standard decision tree training based on largest gain information to select locally optimal parameters

9. Implementation Training images are 300k synthetic body pose images 2000 training pixels per image 10000 pre-selected random parameters On default, 3 trees and 20 levels depth

10. Result

11. How will I apply on RGB video Skin silhouette can be extracted easily on ASL  No background distortion  One signer Mark pixels for class (hand / non-hands). 2 classes for now.

12. Features to use Using temporal dimension Feature will be random 3 dimension derivative Unsure about d(x) – (depth intensity in the original paper). Candidates are  Skin score  Linear combination of skin + motion score

13. Contribution (if result is success) Using randomized forest on RGB instead of depth image Apply temporal information

14. Progress so far Complete pixels marking for training set using segment cut Marking is not perfect but good enough in my opinion As of now, extract only one hand sign gestures Will work on decision tree training next week

15. Samples