Download presentation
Presentation is loading. Please wait.
Published byDale Sherman Modified over 9 years ago
1
Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference on Robotics and Biomimetics, IEEE, 2012
2
Outline Introduction Related Work Proposed Method Experimental Results Conclusion 2
3
Introduction 3
4
Hand Tracking: Essential for HCI Most researchers simplify the issue by restrictions: On users’ clothing On the scene complexity On hand motion 4 Zhou Ren, Junsong Yuan,, Jingjing Meng, M, and Zhengyou Zhang, "Robust Part-Based Hand Gesture Recognition Using Kinect Sensor", IEEE TRANSACTIONS ON MULTIMEDIA, AUGUST 2013
5
Introduction In this paper: Propose a robust hand tracking method Focus on reducing restrictions Combining: Depth cues Color cues (Motion cues) 5 Refined CAMShift tracking
6
Related Work 6
7
Related work Tracking: 7 [a] fingertip ‧ Seed Point ‧ Predicted hand position [b] hand Geodesic distance GSP points Neighbor depth
8
Related work Difficulties: 8 [c] [d] -- (Red) : Side-mode ㄧ (Blue) : Frontal-mode
9
Related work [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012 [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011 [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012 [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER- WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013 9
10
Proposed Method 10
11
Flow Chart 11 Hand DetectionHand Tracking
12
Foreground Segmentation Codebook model Codeword: Motion detection(Foreground): 12 K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis. Real time foreground- background segmentation using code book model. Real-Time Imaging, 11:172–185, 2005. Down-Sampled
13
Foreground Segmentation 13
14
Histogram-based Segmentation Stretch ahead Depth histogram Stretch laterally X-projection histogram 14
15
Histogram-based Segmentation Stretch ahead Depth histogram 15 depth
16
Histogram-based Segmentation Stretch laterally X-projection histogram 16 Lower boundary Upper boundary j-th bin x
17
Histogram-based Segmentation Histogram Analysis Depth histogram & X-projection histogram Foothill algorithm: 17
18
Histogram-based Segmentation 18 max 0 1 0 1 0110 0110 01 Depth histogram X-projection histogram 000000111111011100000
19
Histogram-based Segmentation 19 Scaled x-mask X-projection histogram
20
Skin Color Feature 20 ‧ ‧ ‧
21
Integration of Features Hand Detection: 21 skin depth (stretch ahead) X-projection (stretch laterally)
22
Like mean-shift Like mean-shift 1. Back projection Choose an object → probability map → back projection 2.Mean-shift (frame-frame) CAMShift 22
23
Refined CAMShift Tracking Probability map: Weights: s1 : depth mask s2 : x-mask 23 blob size < threshold otherwise skin depth (stretch ahead) X-projection (stretch laterally)
24
Ecliptic shape representation Refined CAMShift Tracking 24 Aspect ratio: Search window for the next frame:
25
Blob refinement Refined CAMShift Tracking 25 1. Choose proper reference line 2. 3. Reduce the l of the ellipse, untill a proper aspect ratio l/w is obtained.
26
Aspect ratio based blob refinement Refined CAMShift Tracking 26
27
Tracking fast movement 27 Detection + Tracking
28
Face & Hand 28 Detection + Tracking
29
Experimental Results 29
30
Experimental Results Comparison of overall performance 30 [10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674 ‧ Training: 4.8s / 10FPS
31
A) Refined CAMShift with color cue 31
32
B) Multi-cue CAMShift without refinement 32
33
C) The proposed approach 33
34
Experimental Results Video description experimental results 34
35
Conclusion 35
36
Conclusion Focus on reducing restrictions Hand Segmentation: Depth + Skin + (Motion) Histogram analysis Hand tracking CAMShift Blob refinement 36
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.