SOMM: Self Organizing Markov Map for Gesture Recognition Pattern Recognition 2010 Spring Seung-Hyun Lee G. Caridakis et al., Pattern Recognition, Vol.

Slides:



Advertisements
Similar presentations
Gestures Recognition. Image acquisition Image acquisition at BBC R&D studios in London using eight different viewpoints. Sequence frame-by-frame segmentation.
Advertisements

1 Gesture recognition Using HMMs and size functions.
Memristor in Learning Neural Networks
2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.
Rule extraction in neural networks. A survey. Krzysztof Mossakowski Faculty of Mathematics and Information Science Warsaw University of Technology.
Patch to the Future: Unsupervised Visual Prediction
AlgirdasBeinaravičius Gediminas Mazrimas Salman Mosslem.
Cognitive Computer Vision
1 CS6825: Recognition 8. Hidden Markov Models 2 Hidden Markov Model (HMM) HMMs allow you to estimate probabilities of unobserved events HMMs allow you.
Self Organizing Maps. This presentation is based on: SOM’s are invented by Teuvo Kohonen. They represent multidimensional.
HMM-BASED PATTERN DETECTION. Outline  Markov Process  Hidden Markov Models Elements Basic Problems Evaluation Optimization Training Implementation 2-D.
Expectation Maximization Method Effective Image Retrieval Based on Hidden Concept Discovery in Image Database By Sanket Korgaonkar Masters Computer Science.
CONTENT BASED FACE RECOGNITION Ankur Jain 01D05007 Pranshu Sharma Prashant Baronia 01D05005 Swapnil Zarekar 01D05001 Under the guidance of Prof.
Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.
5/30/2006EE 148, Spring Visual Categorization with Bags of Keypoints Gabriella Csurka Christopher R. Dance Lixin Fan Jutta Willamowski Cedric Bray.
Learning Programs Danielle and Joseph Bennett (and Lorelei) 4 December 2007.
Face Processing System Presented by: Harvest Jang Group meeting Fall 2002.
A Hybrid Self-Organizing Neural Gas Network James Graham and Janusz Starzyk School of EECS, Ohio University Stocker Center, Athens, OH USA IEEE World.
SOMTIME: AN ARTIFICIAL NEURAL NETWORK FOR TOPOLOGICAL AND TEMPORAL CORRELATION FOR SPATIOTEMPORAL PATTERN LEARNING.
Case Studies Dr Lee Nung Kion Faculty of Cognitive Sciences and Human Development UNIVERSITI MALAYSIA SARAWAK.
Resilient Machines Through Continuous Self-Modeling Pattern Recognition Seung-Hyun Lee Soft Computing Lab. Josh Bongard,Victor Zykov, and Hod.
Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.
Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.
Prakash Chockalingam Clemson University Non-Rigid Multi-Modal Object Tracking Using Gaussian Mixture Models Committee Members Dr Stan Birchfield (chair)
Abstract Developing sign language applications for deaf people is extremely important, since it is difficult to communicate with people that are unfamiliar.
7-Speech Recognition Speech Recognition Concepts
CVPR Workshop on RTV4HCI 7/2/2004, Washington D.C. Gesture Recognition Using 3D Appearance and Motion Features Guangqi Ye, Jason J. Corso, Gregory D. Hager.
Recognition, Analysis and Synthesis of Gesture Expressivity George Caridakis IVML-ICCS.
Algirdas Beinaravičius Gediminas Mazrimas Salman Mosslem.
資訊工程系智慧型系統實驗室 iLab 南台科技大學 1 A Static Hand Gesture Recognition Algorithm Using K- Mean Based Radial Basis Function Neural Network 作者 :Dipak Kumar Ghosh,
Research Projects 6v81 Multimedia Database Yohan Jin, T.A.
Vision-based human motion analysis: An overview Computer Vision and Image Understanding(2007)
Head Tracking in Meeting Scenarios Sascha Schreiber.
Human pose recognition from depth image MS Research Cambridge.
ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Reestimation Equations Continuous Distributions.
Action and Gait Recognition From Recovered 3-D Human Joints IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS— PART B: CYBERNETICS, VOL. 40, NO. 4, AUGUST.
Case Study 1 Semantic Analysis of Soccer Video Using Dynamic Bayesian Network C.-L Huang, et al. IEEE Transactions on Multimedia, vol. 8, no. 4, 2006 Fuzzy.
Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,
Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.
Face Image-Based Gender Recognition Using Complex-Valued Neural Network Instructor :Dr. Dong-Chul Kim Indrani Gorripati.
 Present by 陳群元.  Introduction  Previous work  Predicting motion patterns  Spatio-temporal transition distribution  Discerning pedestrians  Experimental.
Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology 1 A self-organizing map for adaptive processing of structured.
Learning video saliency from human gaze using candidate selection CVPR2013 Poster.
Pattern Recognition NTUEE 高奕豪 2005/4/14. Outline Introduction Definition, Examples, Related Fields, System, and Design Approaches Bayesian, Hidden Markov.
Supervised Learning – Network is presented with the input and the desired output. – Uses a set of inputs for which the desired outputs results / classes.
Learning Image Statistics for Bayesian Tracking Hedvig Sidenbladh KTH, Sweden Michael Black Brown University, RI, USA
Learning to Answer Questions from Image Using Convolutional Neural Network Lin Ma, Zhengdong Lu, and Hang Li Huawei Noah’s Ark Lab, Hong Kong
Student Gesture Recognition System in Classroom 2.0 Chiung-Yao Fang, Min-Han Kuo, Greg-C Lee, and Sei-Wang Chen Department of Computer Science and Information.
Bayesian Brain - Chapter 11 Neural Models of Bayesian Belief Propagation Rajesh P.N. Rao Summary by B.-H. Kim Biointelligence Lab School of.
Big data classification using neural network
Unsupervised Learning of Video Representations using LSTMs
Self-Organizing Network Model (SOM) Session 11
Deep Learning Amin Sobhani.
Data Mining, Neural Network and Genetic Programming
Data Mining, Neural Network and Genetic Programming
Convolutional Neural Fabrics by Shreyas Saxena, Jakob Verbeek
Intelligent Information System Lab
Self-Organizing Maps for Content-Based Image Database Retrieval
CSc 219 Project Proposal Raymond Fraizer.
RECURRENT NEURAL NETWORKS FOR VOICE ACTIVITY DETECTION
Video-based human motion recognition using 3D mocap data
Self organizing networks
Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules
CSSE463: Image Recognition Day 11
Source: Pattern Recognition Vol. 38, May, 2005, pp
Visual Recognition of American Sign Language Using Hidden Markov Models 문현구 문현구.
Human-object interaction
Week 7 Presentation Ngoc Ta Aidean Sharghi
Presentation transcript:

SOMM: Self Organizing Markov Map for Gesture Recognition Pattern Recognition 2010 Spring Seung-Hyun Lee G. Caridakis et al., Pattern Recognition, Vol. 31, pp , 2010.

S FT YONSEI UNIV. KOREA 16 Contents Introduction Related Work –Hidden Markov Models –Other Method Proposed Method Experiments Conclusion 1

S FT YONSEI UNIV. KOREA 16 Introduction Gesture –A motion of the body that conveys information In this paper –Focus on hand gestures 2

S FT YONSEI UNIV. KOREA 16 Introduction Taxonomy of gesture(McNeill, 1992) –Gesticulation –Speech-linked –Pantomime –Emblems –Sign Languages Other (Kendon,1992) (Quek, 1994) 3

S FT YONSEI UNIV. KOREA 16 Introduction Taxonomy by functionality 4 GesturesDefinition Symbolic gestures gestures that, within each culture, have come to have a single meani ng. Deictic gestures types of gestures most generally seen in HCI and are the gestures of pointing to entities or direction. Iconic gestures gestures used to convey information about the size, spatial relations, actions, shape or orientation of the object of discourse display. Pantominic gesturesgestures typically used to mimic an action, object or concept.

S FT YONSEI UNIV. KOREA 16 Related Work Cogan(2006) –Discrete HMM which fuse hand shape and position Hossain(2005) –Implicit/Explicit Temporal Information Encoded HMM –Discriminated attention and non-attention gestures Mantyla(2000) –On mobile devices –Utilized SOM and HMM method Starner(1998) –HMM based American Sign Language(ASL) recognition –Sentence level recognition is possible 5 Hidden Markov Model

S FT YONSEI UNIV. KOREA 16 Related Work Black and Jepson(1998) –CONDitional dENSity propagATION (CONDENSATION) algorothm Wong and Ciipolla(2006) –Sparse Bayesian classifier Hong et al.(2000) –Finite State Machines(FSM) Su(2000) –Fuzzy logic and rule-based approaches and hyper-rectangular composite Neural network(HRCNNs) Juang and Ku(2005) –Fuzzified Takagi-Sugeno-Kang(TSK) type recurrent network Yang et al.(2002) –Time Delay Neural network Huang and Huang(1998) –3D Hopfield Neural Network 6 Other method

S FT YONSEI UNIV. KOREA 16 Proposed Method Modules –Image processing : detection an tracking of hands –SOM : quantization of hand location and direction –HMM : transition probability matrix 7 Overview

S FT YONSEI UNIV. KOREA 16 Proposed Method Video based method –Creation of moving skin masks (Skin color area) –Tracking the centroid of the skin masks –Prior knowledge is required It should indicate different body parts (Left, right hand, and head) Environment –PC platform –OpenCV 8 Feature Extraction

S FT YONSEI UNIV. KOREA 16 Proposed Method Dataset Gesture instances 9

S FT YONSEI UNIV. KOREA 16 Proposed Method cf) SOM (1) continuous input space (2) discrete output space in the form of lattice (3) time-varying neighborhood function defined around winning neuron (4) decreasing learning rate parameter 10 Position Model

S FT YONSEI UNIV. KOREA 16 Proposed Method Some based representation of hand position 11 Position Model

S FT YONSEI UNIV. KOREA 16 Proposed Method Additional information: Moving direction 12 Direction Model

S FT YONSEI UNIV. KOREA 16 Proposed Method Based on Levenshtein distance(edit distance) –Measuring the amount of difference between two sequences Generalized median of data set Mj Mean Levenstein distance between members 13 Generalized Median

S FT YONSEI UNIV. KOREA 16 Proposed Method Position –Probability –Calculation of S som First state: initial probability From second state: transition probability –Unit u 14 Gesture Decoding

S FT YONSEI UNIV. KOREA 16 Proposed Method Direction –Probability –Calculation of S of –Unit u 15 Gesture Decoding

S FT YONSEI UNIV. KOREA 16 Proposed Method Similarity measurement –Problem Shorter gesture instances tend to gain an advantage by having less transitions and thus less probabilities multiplication –Measurement 16 Gesture Decoding

S FT YONSEI UNIV. KOREA 16 Proposed Method Error definition for function f SOM based approach –If data containing small error is mapped to the same node of SOM  No problem –Otherwise  Consequently, because of neighboring relation of u, error is not propagated to the next steps of the recognition process 17 Error Propagation

S FT YONSEI UNIV. KOREA 16 Experiment 30 gestures 10 repetitions each 18 Data Set

S FT YONSEI UNIV. KOREA 16 Experiment SOM clustering –Blue: close to input vector –Red: not close Recognition accuracy –Test with training data: 100% –10-fold cross validation: 93% ms for decoding a gesture –Only HMM-based classifier: 86.36% 19 Result

S FT YONSEI UNIV. KOREA 16 Conclusion Key features –SOM and HMM based automatic recognition architecture –ROI Relative hand position Moving direction Similarity of pattern Application –Sign language –Gaming environment 20

Thank you