Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine Journal of Information & Computational Science, 2015.

Slides:



Advertisements
Similar presentations
Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Advertisements

DONG XU, MEMBER, IEEE, AND SHIH-FU CHANG, FELLOW, IEEE Video Event Recognition Using Kernel Methods with Multilevel Temporal Alignment.
Víctor Ponce Miguel Reyes Xavier Baró Mario Gorga Sergio Escalera Two-level GMM Clustering of Human Poses for Automatic Human Behavior Analysis Departament.
Wen-Hung Liao Department of Computer Science National Chengchi University November 27, 2008 Estimation of Skin Color Range Using Achromatic Features.
Robust Part-Based Hand Gesture Recognition Using Kinect Sensor
Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual Reality and Visualization.
Xin Zhang, Zhichao Ye, Lianwen Jin, Ziyong Feng, and Shaojie Xu
Multi-scenario Gesture Recognition Using Kinect Student : Sin- Jhu YE Student Id : MA Computer Engineering & Computer Science University of Louisville.
Patch to the Future: Unsupervised Visual Prediction
Activity Recognition Aneeq Zia. Agenda What is activity recognition Typical methods used for action recognition “Evaluation of local spatio-temporal features.
A Robust Method of Detecting Hand Gestures Using Depth Sensors Yan Wen, Chuanyan Hu, Guanghui Yu, Changbo Wang Haptic Audio Visual Environments and Games.
The image based surveillance system for personnel and vehicle tracking Chairman:Hung-Chi Yang Advisor: Yen-Ting Chen Presenter: Fong-Ren Sie Date:
Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.
A KLT-Based Approach for Occlusion Handling in Human Tracking Chenyuan Zhang, Jiu Xu, Axel Beaugendre and Satoshi Goto 2012 Picture Coding Symposium.
Efficient Moving Object Segmentation Algorithm Using Background Registration Technique Shao-Yi Chien, Shyh-Yih Ma, and Liang-Gee Chen, Fellow, IEEE Hsin-Hua.
Tracking a moving object with real-time obstacle avoidance Chung-Hao Chen, Chang Cheng, David Page, Andreas Koschan and Mongi Abidi Imaging, Robotics and.
ACM Multimedia th Annual Conference, October , 2004
Feature vs. Model Based Vocal Tract Length Normalization for a Speech Recognition-based Interactive Toy Jacky CHAU Department of Computer Science and Engineering.
Real-time Hand Pose Recognition Using Low- Resolution Depth Images
Hand Signals Recognition from Video Using 3D Motion Capture Archive Tai-Peng Tian Stan Sclaroff Computer Science Department B OSTON U NIVERSITY I. Introduction.
Ensuring Home-based Rehabilitation Exercise by Using Kinect and Fuzzified Dynamic Time Warping Algorithm Qiao Zhang.
KinWrite: Handwriting-Based Authentication Using Kinect Proceedings of the 20th Annual Network & Distributed System Security Symposium, NDSS 2013 Jing.
Normalization of the Speech Modulation Spectra for Robust Speech Recognition Xiong Xiao, Eng Siong Chng, and Haizhou Li Wen-Yi Chu Department of Computer.
Temporal Event Map Construction For Event Search Qing Li Department of Computer Science City University of Hong Kong.
A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu th.
ONLINE HANDWRITTEN GURMUKHI SCRIPT RECOGNITION AND ITS CHALLENGES R. K. SHARMA THAPAR UNIVERSITY, PATIALA.
Fingertip Tracking Based Active Contour for General HCI Application Proceedings of the First International Conference on Advanced Data and Information.
3D Fingertip and Palm Tracking in Depth Image Sequences
Robust Hand Tracking with Refined CAMShift Based on Combination of Depth and Image Features Wenhuan Cui, Wenmin Wang, and Hong Liu International Conference.
Zhengyou Zhang Microsoft Research Digital Object Identifier: /MMUL Publication Year: 2012, Page(s): Professor: Yih-Ran Sheu Student.
KinectFusion : Real-Time Dense Surface Mapping and Tracking IEEE International Symposium on Mixed and Augmented Reality 2011 Science and Technology Proceedings.
Object Bank Presenter : Liu Changyu Advisor : Prof. Alex Hauptmann Interest : Multimedia Analysis April 4 th, 2013.
Human Gesture Recognition Using Kinect Camera Presented by Carolina Vettorazzo and Diego Santo Orasa Patsadu, Chakarida Nukoolkit and Bunthit Watanapa.
Miguel Reyes 1,2, Gabriel Dominguez 2, Sergio Escalera 1,2 Computer Vision Center (CVC) 1, University of Barcelona (UB) 2
A Method for Hand Gesture Recognition Jaya Shukla Department of Computer Science Shiv Nadar University Gautam Budh Nagar, India Ashutosh Dwivedi.
1 Formal Models for Expert Finding on DBLP Bibliography Data Presented by: Hongbo Deng Co-worked with: Irwin King and Michael R. Lyu Department of Computer.
Vehicle License Plate Detection Algorithm Based on Statistical Characteristics in HSI Color Model Instructor : Dr. K. R. Rao Presented by: Prasanna Venkatesh.
出處: Signal Processing and Communications Applications, 2006 IEEE 作者: Asanterabi Malima, Erol Ozgur, and Miijdat Cetin 2015/10/251 指導教授:張財榮 學生:陳建宏 學號: M97G0209.
資訊工程系智慧型系統實驗室 iLab 南台科技大學 1 A Static Hand Gesture Recognition Algorithm Using K- Mean Based Radial Basis Function Neural Network 作者 :Dipak Kumar Ghosh,
DIEGO AGUIRRE COMPUTER VISION INTRODUCTION 1. QUESTION What is Computer Vision? 2.
A New Fingertip Detection and Tracking Algorithm and Its Application on Writing-in-the-air System The th International Congress on Image and Signal.
Greedy is not Enough: An Efficient Batch Mode Active Learning Algorithm Chen, Yi-wen( 陳憶文 ) Graduate Institute of Computer Science & Information Engineering.
Human pose recognition from depth image MS Research Cambridge.
Interactive Sand Art Drawing Using RGB-D Sensor
Fingertip Detection with Morphology and Geometric Calculation Dung Duc Nguyen ; Thien Cong Pham ; Jae Wook Jeon Intelligent Robots and Systems, IEEE/RSJ.
AUTOMATIC TARGET RECOGNITION AND DATA FUSION March 9 th, 2004 Bala Lakshminarayanan.
Design of PCA and SVM based face recognition system for intelligent robots Department of Electrical Engineering, Southern Taiwan University, Tainan County,
Automated Fingertip Detection
Skeleton Based Action Recognition with Convolutional Neural Network
Su-ting, Chuang 1. Outline Introduction Related work Hardware configuration Detection system Optimal parameter estimation framework Conclusion 2.
Journal of Visual Communication and Image Representation
A Recognition Method of Restricted Hand Shapes in Still Image and Moving Image Hand Shapes in Still Image and Moving Image as a Man-Machine Interface Speaker.
Target Tracking In a Scene By Saurabh Mahajan Supervisor Dr. R. Srivastava B.E. Project.
Su-ting, Chuang 1. Outline Introduction Related work Hardware configuration Finger Detection system Optimal parameter estimation framework Conclusion.
Similarity Measurement and Detection of Video Sequences Chu-Hong HOI Supervisor: Prof. Michael R. LYU Marker: Prof. Yiu Sang MOON 25 April, 2003 Dept.
Outline Introduction Related Work System Overview Methodology Experiment Conclusion and Future Work.
Marco Maisto, Massimo Panella, Luca Liparulo, and Andrea Proietti
WLD: A Robust Local Image Descriptor Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikäinen, Xilin Chen, Wen Gao 报告人:蒲薇榄.
Kalman Filter and Data Streaming Presented By :- Ankur Jain Department of Computer Science 7/21/03.
Facial Smile Detection Based on Deep Learning Features Authors: Kaihao Zhang, Yongzhen Huang, Hong Wu and Liang Wang Center for Research on Intelligent.
Experience Report: System Log Analysis for Anomaly Detection
Eick: Introduction Machine Learning
Performance of Computer Vision
Majkowska University of California. Los Angeles
Video-based human motion recognition using 3D mocap data
PEBL: Web Page Classification without Negative Examples
Estimation of Skin Color Range Using Achromatic Features
Liyuan Li, Jerry Kah Eng Hoe, Xinguo Yu, Li Dong, and Xinqi Chu
CNN-based Action Recognition Using Adaptive Multiscale Depth Motion Maps And Stable Joint Distance Maps Junyou He, Hailun Xia, Chunyan Feng, Yunfei Chu.
Source: Pattern Recognition Letters, Article In Press, 2007
Presentation transcript:

Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine Journal of Information & Computational Science, 2015 Chengzhang Qua, Dengyi Zhang Computer School, Wuhan University Jing Tian Department of Computer Science and Engineering, University of South Carolina Speaker: Yi-Ting Chen

Outline Introduction Flowchart Proposed Method –Preprocessing and Feature Extraction –Recognition Method Experiment Conclusions 2

Introduction Handwriting in-space owns more advantages in natural interaction and has become a hot topic in the past decades. 3D cameras that can capture depth data besides traditional color data also gain a rapid growth in recent years, particularly those with Kinect. 3

Introduction The Kinect uses an infrared projector and a special microchip to track the movement of objects and individuals in three dimensions. It records RGB, depth, and skeleton information. We use the recorded depth and skeleton frames to describe the motion of the user. 4

Main Contribution In this paper, a Kinect based interaction recognition system was developed and tested on a handwriting digit dataset. There are mainly two steps in current system –The first is acquiring data during interaction. –The second is data processing and recognition. 5

Related Work The finger detecting and tracking algorithm –Based on depth image: Ye et al.[21] Feng et al.[22] Feature processing and recognition. –Feng et al.[22] 90% on English characters and 80% on Chinese –Lang et al.[20] recognition rate of 97% –Huang et al.[19] 94.6% on digit recognition 6

Reference [19] F. A. Huang, C. Y. Su, T. T. Chu, Kinect-based mid-air handwritten digit recognition using multiple segments and scaled coding [C], 2013 International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS), IEEE, 2013, [20] S. Lang, M. Block, R. Rojas, Sign language recognition using Kinect [C], Artificial Intelligence and Soft Computing, Springer, Berlin Heidelberg, 2012, [21] Z. Ye, X. Zhang, L. Jin et al., Finger-writing-in-the-air system using Kinect sensor [C], 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), IEEE, 2013, 1-4 [22] Z. Feng, S. Xu, X. Zhang et al., Real-time ¯ngertip tracking and detection using Kinect depth sensor for a new writing-in-the air system [C], Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, ACM, 2012,

Flowchart 8

9

Fingertip Tracking 10 V(t)

Fingertip Tracking We confirm the area based on enforcing the temporal continuity of the fingertip position. (a) should be in a small area centred by hand skeleton joint (b) should vary in a small range between two consecutive frames. 11

Fingertip Tracking Algorithm 12

Connect the Points 13 When connecting the consecutive points to form a writing trajectory, we may obtain a highly jagged and noisy 3D trajectory.

Normalization and Smoothing 1. Move the 3D trajectory to the original point(0, 0, 0). 2. Normalize each position Follows a normal Gaussian distribution N(0; 1) over all the 3D points in all recorded data. 3. Employ a standard Kalman filter algorithm to calculate a smoothed trajectory. 14

Feature Extraction We extract 6 types of features from each point of 3D handwritten trajectory. 15

6 Types of Features 16

Flowchart 17

Dynamic Time Warping 18 [23] E. Keogh, C. A. Ratanamahatana, Exact indexing of dynamic time warping [J], Knowledge and Information Systems, 7(3), 2005, M M M N N N

Classic Dynamic Time Warping 19 [23] E. Keogh, C. A. Ratanamahatana, Exact indexing of dynamic time warping [J], Knowledge and Information Systems, 7(3), 2005, Q l … 1 1 k … I. Boundary condition: II. Monotonicity condition: III. Step size condition:

Calculating between features Calculating the distance between two 3D handwritten features: 20

Modified Dynamic Time Warping 21 Q l … 1 1 k … I. Boundary condition: II. Monotonicity condition: III. Step size condition:

Support Vector Machine 22

Support Vector Machine 23

Experiment Data set: 0~9 3D handwriting digits collected by 10 college students. total of 6104 handwritten digit samples 24

Confused Matrix Training number: 20 Testing number: 590 for each class The mean accuracy is 99.1% 25

Precision Recall Curve Nearly 100% decision value for most of recall value. 26

Confused Digits 27

Accuracy curves for other methods For training numbers

Experiment Compare: Huang et al.[19] perform 94.6% on digit recognition –100 samples for training –230 samples for testing 29 [19] F. A. Huang, C. Y. Su, T. T. Chu, Kinect-based mid-air handwritten digit recognition using multiple segments and scaled coding [C], 2013 ISPACS, IEEE, 2013,

Conclusion A human computer interaction system based on Kinect handwritten was proposed. 3D fingertip positions could be tracked in real time with our method. Recognition framework for 3D handwritten interaction could be built based on DTW and SVM. According to the evaluation results, the proposed method offer high recognition accuracy 30

Thanks for your listening! 31