Presentation is loading. Please wait.

Presentation is loading. Please wait.

Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference.

Published byDale Sherman Modified over 9 years ago

Similar presentations

Presentation on theme: "Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference."— Presentation transcript:

1 Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference on Robotics and Biomimetics, IEEE, 2012

2 Outline Introduction Related Work Proposed Method Experimental Results Conclusion 2

3 Introduction 3

4 Hand Tracking: Essential for HCI Most researchers simplify the issue by restrictions: On users’ clothing On the scene complexity On hand motion 4 Zhou Ren, Junsong Yuan,, Jingjing Meng, M, and Zhengyou Zhang, "Robust Part-Based Hand Gesture Recognition Using Kinect Sensor", IEEE TRANSACTIONS ON MULTIMEDIA, AUGUST 2013

5 Introduction In this paper: Propose a robust hand tracking method Focus on reducing restrictions Combining: Depth cues Color cues (Motion cues) 5 Refined CAMShift tracking

6 Related Work 6

7 Related work Tracking: 7 [a] fingertip ‧ Seed Point ‧ Predicted hand position [b] hand Geodesic distance GSP points Neighbor depth

8 Related work Difficulties: 8 [c] [d] -- (Red) : Side-mode ㄧ (Blue) : Frontal-mode

9 Related work [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012 [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011 [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012 [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER- WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013 9

10 Proposed Method 10

11 Flow Chart 11 Hand DetectionHand Tracking

12 Foreground Segmentation Codebook model Codeword: Motion detection(Foreground): 12 K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis. Real time foreground- background segmentation using code book model. Real-Time Imaging, 11:172–185, 2005. Down-Sampled

13 Foreground Segmentation 13

14 Histogram-based Segmentation Stretch ahead Depth histogram Stretch laterally X-projection histogram 14

15 Histogram-based Segmentation Stretch ahead Depth histogram 15 depth

16 Histogram-based Segmentation Stretch laterally X-projection histogram 16 Lower boundary Upper boundary j-th bin x

17 Histogram-based Segmentation Histogram Analysis Depth histogram & X-projection histogram Foothill algorithm: 17

18 Histogram-based Segmentation 18 max 0 1 0 1 0110 0110 01 Depth histogram X-projection histogram 000000111111011100000

19 Histogram-based Segmentation 19 Scaled x-mask X-projection histogram

20 Skin Color Feature 20 ‧ ‧ ‧

21 Integration of Features Hand Detection: 21 skin depth (stretch ahead) X-projection (stretch laterally)

22 Like mean-shift Like mean-shift 1. Back projection Choose an object → probability map → back projection 2.Mean-shift (frame-frame) CAMShift 22

23 Refined CAMShift Tracking Probability map: Weights: s1 : depth mask s2 : x-mask 23 blob size < threshold otherwise skin depth (stretch ahead) X-projection (stretch laterally)

24 Ecliptic shape representation Refined CAMShift Tracking 24 Aspect ratio: Search window for the next frame:

25 Blob refinement Refined CAMShift Tracking 25 1. Choose proper reference line 2. 3. Reduce the l of the ellipse, untill a proper aspect ratio l/w is obtained.

26 Aspect ratio based blob refinement Refined CAMShift Tracking 26

27 Tracking fast movement 27 Detection + Tracking

28 Face & Hand 28 Detection + Tracking

29 Experimental Results 29

30 Experimental Results Comparison of overall performance 30 [10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674 ‧ Training: 4.8s / 10FPS

31 A) Refined CAMShift with color cue 31

32 B) Multi-cue CAMShift without refinement 32

33 C) The proposed approach 33

34 Experimental Results Video description experimental results 34

35 Conclusion 35

36 Conclusion Focus on reducing restrictions Hand Segmentation: Depth + Skin + (Motion) Histogram analysis Hand tracking CAMShift Blob refinement 36

Download ppt "Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference."

Similar presentations

Hand Gesture for Taking Self Portrait Shaowei Chu and Jiro Tanaka University of Tsukuba Japan 12th July 15 minutes talk.

Hand Gesture for Taking Self Portrait Shaowei Chu and Jiro Tanaka University of Tsukuba Japan 12th July 15 minutes talk.
Wen-Hung Liao Department of Computer Science National Chengchi University November 27, 2008 Estimation of Skin Color Range Using Achromatic Features.

Wen-Hung Liao Department of Computer Science National Chengchi University November 27, 2008 Estimation of Skin Color Range Using Achromatic Features.
Robust Part-Based Hand Gesture Recognition Using Kinect Sensor

Robust Part-Based Hand Gesture Recognition Using Kinect Sensor
Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual Reality and Visualization.

Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual Reality and Visualization.
Real-Time Hand Gesture Recognition with Kinect for Playing Racing Video Games 2014 International Joint Conference on Neural Networks (IJCNN) July 6-11,

Real-Time Hand Gesture Recognition with Kinect for Playing Racing Video Games 2014 International Joint Conference on Neural Networks (IJCNN) July 6-11,
Xin Zhang, Zhichao Ye, Lianwen Jin, Ziyong Feng, and Shaojie Xu

Xin Zhang, Zhichao Ye, Lianwen Jin, Ziyong Feng, and Shaojie Xu
Multi-scenario Gesture Recognition Using Kinect Student ： Sin- Jhu YE Student Id ： MA020206 Computer Engineering & Computer Science University of Louisville.

Multi-scenario Gesture Recognition Using Kinect Student ： Sin- Jhu YE Student Id ： MA Computer Engineering & Computer Science University of Louisville.
Introduction To Tracking

Introduction To Tracking
F ACE TRACKING EE 7700 Name: Jing Chen Shaoming Chen.

F ACE TRACKING EE 7700 Name: Jing Chen Shaoming Chen.
A Robust Method of Detecting Hand Gestures Using Depth Sensors Yan Wen, Chuanyan Hu, Guanghui Yu, Changbo Wang Haptic Audio Visual Environments and Games.

A Robust Method of Detecting Hand Gestures Using Depth Sensors Yan Wen, Chuanyan Hu, Guanghui Yu, Changbo Wang Haptic Audio Visual Environments and Games.
Facial feature localization Presented by: Harvest Jang Spring 2002.

Facial feature localization Presented by: Harvest Jang Spring 2002.
Robust Object Tracking via Sparsity-based Collaborative Model

Robust Object Tracking via Sparsity-based Collaborative Model
Multiple People Detection and Tracking with Occlusion Presenter: Feifei Huo Supervisor: Dr. Emile A. Hendriks Dr. A. H. J. Stijn Oomes Information and.

Multiple People Detection and Tracking with Occlusion Presenter: Feifei Huo Supervisor: Dr. Emile A. Hendriks Dr. A. H. J. Stijn Oomes Information and.
Adviser ： Ming-Yuan Shieh Student ID ： M9820202 Student ： Chung-Chieh Lien VIDEO OBJECT SEGMENTATION AND ITS SALIENT MOTION DETECTION USING ADAPTIVE BACKGROUND.

Adviser ： Ming-Yuan Shieh Student ID ： M Student ： Chung-Chieh Lien VIDEO OBJECT SEGMENTATION AND ITS SALIENT MOTION DETECTION USING ADAPTIVE BACKGROUND.
Department of Electrical and Computer Engineering He Zhou Hui Zheng William Mai Xiang Guo Advisor: Professor Patrick Kelly ASLLENGE.

Department of Electrical and Computer Engineering He Zhou Hui Zheng William Mai Xiang Guo Advisor: Professor Patrick Kelly ASLLENGE.
A KLT-Based Approach for Occlusion Handling in Human Tracking Chenyuan Zhang, Jiu Xu, Axel Beaugendre and Satoshi Goto 2012 Picture Coding Symposium.

A KLT-Based Approach for Occlusion Handling in Human Tracking Chenyuan Zhang, Jiu Xu, Axel Beaugendre and Satoshi Goto 2012 Picture Coding Symposium.
Proceedings of the British Machine Vision Conference (BMVC), 2010 Qi Wang, Xilin Chen, Wen Gao Skin Color Weighted Disparity Competition for Hand Segmentation.

Proceedings of the British Machine Vision Conference (BMVC), 2010 Qi Wang, Xilin Chen, Wen Gao Skin Color Weighted Disparity Competition for Hand Segmentation.
Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA 94720.

Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA
Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.

Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.
{ Fast Disparity Estimation Using Spatio- temporal Correlation of Disparity Field for Multiview Video Coding Wei Zhu, Xiang Tian, Fan Zhou and Yaowu Chen.

{ Fast Disparity Estimation Using Spatio- temporal Correlation of Disparity Field for Multiview Video Coding Wei Zhu, Xiang Tian, Fan Zhou and Yaowu Chen.

Similar presentations

Ads by Google