BoVDW: Bag-of-Visual-and-Depth- Words for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Antonio Hernández-Vela.

Slides:

Advertisements

Similar presentations

3D Model Matching with Viewpoint-Invariant Patches(VIP) Reporter ：鄒嘉恆 Date ： 10/06/2009.

Advertisements

ADHD indicators modelling based on Dynamic Time Warping from RGB data: A feasibility study Antonio Hernández-Vela, Miguel Reyes, Laura Igual, Josep Moya,

Combining Detectors for Human Hand Detection Antonio Hernández, Petia Radeva and Sergio Escalera Computer Vision Center, Universitat Autònoma de Barcelona,

Evaluating Color Descriptors for Object and Scene Recognition Koen E.A. van de Sande, Student Member, IEEE, Theo Gevers, Member, IEEE, and Cees G.M. Snoek,

RGB-D object recognition and localization with clutter and occlusions Federico Tombari, Samuele Salti, Luigi Di Stefano Computer Vision Lab – University.

Limin Wang, Yu Qiao, and Xiaoou Tang

Víctor Ponce Miguel Reyes Xavier Baró Mario Gorga Sergio Escalera Two-level GMM Clustering of Human Poses for Automatic Human Behavior Analysis Departament.

Multi Hand Pose Recognition System using Kinect Depth Sensor: Application to Medical Image Navigation Oscar Lopes, Miguel Pousa, Miguel Reyes, Sergio Escalera,

One-Shot Learning Gesture Recognition Students:Itay Hubara Amit Nishry Supervisor:Maayan Harel Gal-On.

Patch to the Future: Unsupervised Visual Prediction

Activity Recognition Aneeq Zia. Agenda What is activity recognition Typical methods used for action recognition “Evaluation of local spatio-temporal features.

Juergen Gall Action Recognition.

Space-time interest points Computational Vision and Active Perception Laboratory (CVAP) Dept of Numerical Analysis and Computer Science KTH (Royal Institute.

Activity Recognition Computer Vision CS 143, Brown James Hays 11/21/11 With slides by Derek Hoiem and Kristen Grauman.

Instructor: Mircea Nicolescu Lecture 17

Probability-based Dynamic Time Warping for Gesture Recognition on RGB-D data All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Group.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Transferable Dictionary Pair based Cross-view Action Recognition Lin Hong.

Robust Object Tracking via Sparsity-based Collaborative Model

Real-Time Activity Monitoring of Inpatients Miguel Reyes, Jordi Vitrià, Petia Radeva and Sergio Escalera Computer Vision Center, Universitat Autònoma de.

Local Descriptors for Spatio-Temporal Recognition

Bag-of-features models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim

Lecture 28: Bag-of-words models

A Study of Approaches for Object Recognition

Bag-of-features models

Scale Invariant Feature Transform (SIFT)

Local Features and Kernels for Classification of Object Categories J. Zhang --- QMUL UK (INRIA till July 2005) with M. Marszalek and C. Schmid --- INRIA.

1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.

Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.

Review: Intro to recognition Recognition tasks Machine learning approach: training, testing, generalization Example classifiers Nearest neighbor Linear.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION.

Exercise Session 10 – Image Categorization

Biologically Inspired Turn Control for Autonomous Mobile Robots Xavier Perez-Sala, Cecilio Angulo, Sergio Escalera.

Bag-of-Words based Image Classification Joost van de Weijer.

Bag of Video-Words Video Representation

AdvisorStudent Dr. Jia Li Shaojun Liu Dept. of Computer Science and Engineering, Oakland University 3D Shape Classification Using Conformal Mapping In.

Vision-based Navigation and Reinforcement Learning Path Finding for Social Robots Xavier Pérez *, Cecilio Angulo *, Sergio Escalera + and Diego Pardo *

Final Exam Review CS485/685 Computer Vision Prof. Bebis.

A Thousand Words in a Scene P. Quelhas, F. Monay, J. Odobez, D. Gatica-Perez and T. Tuytelaars PAMI, Sept

Watch, Listen and Learn Sonal Gupta, Joohyun Kim, Kristen Grauman and Raymond Mooney -Pratiksha Shah.

Miguel Reyes 1,2, Gabriel Dominguez 2, Sergio Escalera 1,2 Computer Vision Center (CVC) 1, University of Barcelona (UB) 2

Marcin Marszałek, Ivan Laptev, Cordelia Schmid Computer Vision and Pattern Recognition, CVPR Actions in Context.

Nonparametric Part Transfer for Fine-grained Recognition Presenter Byungju Kim.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Svetlana Lazebnik, Cordelia Schmid, Jean Ponce

Yao, B., and Fei-fei, L. IEEE Transactions on PAMI(2012)

Spatio-temporal constraints for recognizing 3D objects in videos Nicoletta Noceti Università degli Studi di Genova.

Computer Vision Lab Seoul National University Keyframe-Based Real-Time Camera Tracking Young Ki BAIK Vision seminar : Mar Computer Vision Lab.

Mentor: Salman Khokhar Action Recognition in Crowds Week 7.

Pedestrian Detection and Localization

Bag-of-Words based Image Classification (week I) Joost van de Weijer.

Introducing the Separability Matrix for ECOC coding

Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.

First-Person Activity Recognition: What Are They Doing to Me? M. S. Ryoo and Larry Matthies Jet Propulsion Laboratory, California Institute of Technology,

Skeleton Based Action Recognition with Convolutional Neural Network

Using decision trees to build an a framework for multivariate time- series classification 1 Present By Xiayi Kuang.

Finding Clusters within a Class to Improve Classification Accuracy Literature Survey Yong Jae Lee 3/6/08.

Distinctive Image Features from Scale-Invariant Keypoints Presenter :JIA-HONG,DONG Advisor : Yen- Ting, Chen 1 David G. Lowe International Journal of Computer.

WLD: A Robust Local Image Descriptor Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikäinen, Xilin Chen, Wen Gao 报告人：蒲薇榄.

Hierarchical Motion Evolution for Action Recognition Authors: Hongsong Wang, Wei Wang, Liang Wang Center for Research on Intelligent Perception and Computing,

Face recognition using Histograms of Oriented Gradients

Another Example: Circle Detection

Bag-of-Visual-Words Based Feature Extraction

Data Driven Attributes for Action Detection

Learning Mid-Level Features For Recognition

Temporal Order-Preserving Dynamic Quantization for Human Action Recognition from Multimodal Sensor Streams Jun Ye Kai Li Guo-Jun Qi Kien.

CS 1674: Intro to Computer Vision Scene Recognition

Nguyen Ngoc Hoang, Guee-Sang Lee, Soo-Hyung Kim, Hyung-Jeong Yang

Presentation transcript:

BoVDW: Bag-of-Visual-and-Depth- Words for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Antonio Hernández-Vela 1,2 Miguel Ángel Bautista 1,2 Xavier Perez-Sala 2,3 Victor Ponce Lopez 1,2 Xavier Baro 2,4 Oriol Pujol 1,2 Cecilio Angulo 3 Sergio Escalera 1,2 1 Dept. Applied Mathematics and Analysis, Universitat de Barcelona 2 Computer Vision Center 3 CETpD, Universitat Politècnica de Catalunya 4 EIMT, Universitat Oberta de Catalunya

1.Introduction 2.Methodology 3.Results 4.Conclusion BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Outline

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Bag of (Visual) Words IntroductionMethodologyResultsConclusion

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Bag of Visual and Depth Words IntroductionMethodologyResultsConclusion In this work, we propose: Bag of Visual and Depth Words (BoVDW). A new depth descriptor. Comparison with state-of-the-art descriptors. Gesture recognition framework.

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Standard pipeline IntroductionMethodologyResultsConclusion Point detection Point description Vocabulary & Representation Classification

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis IntroductionMethodologyResultsConclusion Point detection Point description Vocab. & Represent. Classification Spatio-Temporal Interest Points (STIPs) [1] [1] I. Laptev, "On Space-Time Interest Points", (2005) in International Journal of Computer Vision, vol 64, number 2/3, pp Temporal extension of the Harris operator

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis IntroductionMethodologyResultsConclusion Point detection Point description Vocab. & Represent. Classification Viewpoint Feature Histogram (VFH)[2] [2] Rusu, R.B et al., "Fast 3D recognition and pose using the Viewpoint Feature Histogram", IROS, 2010 Figs. credit to [2] and Aitor Aldoma Histogram of angles between surface normals and viewpoint direction Camera Roll Histogram (CRH) Invariant to rotations in the roll axis of the camera!

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis IntroductionMethodologyResultsConclusion Point detection Point description Vocab. & Represent. Classification Concatenation of VFH and CRH

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis IntroductionMethodologyResultsConclusion Point detection Point description Vocab. & Represent. Classification Vocabulary building  K-means clustering Spatio-temporal pyramids … Final histogram: Concatenation of 8+1 histograms

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis IntroductionMethodologyResultsConclusion Point detection Point description Vocabulary Classification K-nearest neighbor classifier Distance function: Histogram intersection:

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Chalearn dataset IntroductionMethodologyResultsConclusion RGB-D video sequences. Organised in 20 batches: 47 sequences of 1-5 gestures each. Gestures from certain lexicon. Same actor. One-shot learning problem: Just 1 training sample for gesture.

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Results IntroductionMethodologyResultsConclusion Evaluation measurement: Levenshtein distance Depth RGB Mean Levenshtein distance (α= 1) (α= 0)

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Results (Late fusion) IntroductionMethodologyResultsConclusion Late fusion approachMean Lev. Dist. HOGHOF/VFHCRH HOG/HOF/VFHCRH Batch number Mean Levenshtein dist. (α= 0.8)

BoVDW for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Conclusion IntroductionMethodologyResultsConclusion We have presented: BoVDW approach for gesture recognition. VFHCRH, a new depth descriptor. Comparison of state-of-the-art descriptors. Analysis of Late fusion of RGB and Depth information. Future work: Test other methodologies for spatial coherence. Improve continuous gesture detection.

BoVDW: Bag-of-Visual-and-Depth-Words for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Antonio Hernández-Vela 1,2 Miguel Ángel Bautista 1,2 Xavier Perez-Sala 2,3 Victor Ponce Lopez 1,2 Xavier Baro 2,4 Oriol Pujol 1,2 Cecilio Angulo 3 Sergio Escalera 1,2 1 Dept. Applied Mathematics and Analysis, Universitat de Barcelona 2 Computer Vision Center 3 CETpD, Universitat Politècnica de Catalunya 4 EIMT, Universitat Oberta de Catalunya Thank you! Questions?