Presentation is loading. Please wait.

Presentation is loading. Please wait.

Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference.

Similar presentations


Presentation on theme: "Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference."— Presentation transcript:

1 Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference on Robotics and Biomimetics, IEEE, 2012

2 Outline Introduction Related Work Proposed Method Experimental Results Conclusion 2

3 Introduction 3

4 Hand Tracking: Essential for HCI Most researchers simplify the issue by restrictions: On users’ clothing On the scene complexity On hand motion 4 Zhou Ren, Junsong Yuan,, Jingjing Meng, M, and Zhengyou Zhang, "Robust Part-Based Hand Gesture Recognition Using Kinect Sensor", IEEE TRANSACTIONS ON MULTIMEDIA, AUGUST 2013

5 Introduction In this paper: Propose a robust hand tracking method Focus on reducing restrictions Combining: Depth cues Color cues (Motion cues) 5 Refined CAMShift tracking

6 Related Work 6

7 Related work Tracking: 7 [a] fingertip ‧ Seed Point ‧ Predicted hand position [b] hand Geodesic distance GSP points Neighbor depth

8 Related work Difficulties: 8 [c] [d] -- (Red) : Side-mode ㄧ (Blue) : Frontal-mode

9 Related work [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012 [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011 [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012 [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER- WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013 9

10 Proposed Method 10

11 Flow Chart 11 Hand DetectionHand Tracking

12 Foreground Segmentation Codebook model Codeword: Motion detection(Foreground): 12 K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis. Real time foreground- background segmentation using code book model. Real-Time Imaging, 11:172–185, 2005. Down-Sampled

13 Foreground Segmentation 13

14 Histogram-based Segmentation Stretch ahead Depth histogram Stretch laterally X-projection histogram 14

15 Histogram-based Segmentation Stretch ahead Depth histogram 15 depth

16 Histogram-based Segmentation Stretch laterally X-projection histogram 16 Lower boundary Upper boundary j-th bin x

17 Histogram-based Segmentation Histogram Analysis Depth histogram & X-projection histogram Foothill algorithm: 17

18 Histogram-based Segmentation 18 max 0 1 0 1 0110 0110 01 Depth histogram X-projection histogram 000000111111011100000

19 Histogram-based Segmentation 19 Scaled x-mask X-projection histogram

20 Skin Color Feature 20 ‧ ‧ ‧

21 Integration of Features Hand Detection: 21 skin depth (stretch ahead) X-projection (stretch laterally)

22 Like mean-shift Like mean-shift 1. Back projection Choose an object → probability map → back projection 2.Mean-shift (frame-frame) CAMShift 22

23 Refined CAMShift Tracking Probability map: Weights: s1 : depth mask s2 : x-mask 23 blob size < threshold otherwise skin depth (stretch ahead) X-projection (stretch laterally)

24 Ecliptic shape representation Refined CAMShift Tracking 24 Aspect ratio: Search window for the next frame:

25 Blob refinement Refined CAMShift Tracking 25 1. Choose proper reference line 2. 3. Reduce the l of the ellipse, untill a proper aspect ratio l/w is obtained.

26 Aspect ratio based blob refinement Refined CAMShift Tracking 26

27 Tracking fast movement 27 Detection + Tracking

28 Face & Hand 28 Detection + Tracking

29 Experimental Results 29

30 Experimental Results Comparison of overall performance 30 [10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674 ‧ Training: 4.8s / 10FPS

31 A) Refined CAMShift with color cue 31

32 B) Multi-cue CAMShift without refinement 32

33 C) The proposed approach 33

34 Experimental Results Video description experimental results 34

35 Conclusion 35

36 Conclusion Focus on reducing restrictions Hand Segmentation: Depth + Skin + (Motion) Histogram analysis Hand tracking CAMShift Blob refinement 36


Download ppt "Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference."

Similar presentations


Ads by Google