Multi Hand Pose Recognition System using Kinect Depth Sensor: Application to Medical Image Navigation Oscar Lopes, Miguel Pousa, Miguel Reyes, Sergio Escalera,

Slides:

Advertisements

Similar presentations

3D Model Matching with Viewpoint-Invariant Patches(VIP) Reporter ：鄒嘉恆 Date ： 10/06/2009.

Advertisements

ADHD indicators modelling based on Dynamic Time Warping from RGB data: A feasibility study Antonio Hernández-Vela, Miguel Reyes, Laura Igual, Josep Moya,

Combining Detectors for Human Hand Detection Antonio Hernández, Petia Radeva and Sergio Escalera Computer Vision Center, Universitat Autònoma de Barcelona,

Multiclass SVM and Applications in Object Classification

Object Recognition with Features Inspired by Visual Cortex T. Serre, L. Wolf, T. Poggio Presented by Andrew C. Gallagher Jan. 25, 2007.

Víctor Ponce Miguel Reyes Xavier Baró Mario Gorga Sergio Escalera Two-level GMM Clustering of Human Poses for Automatic Human Behavior Analysis Departament.

Real-Time Hand Gesture Recognition with Kinect for Playing Racing Video Games 2014 International Joint Conference on Neural Networks (IJCNN) July 6-11,

MIT CSAIL Vision interfaces Approximate Correspondences in High Dimensions Kristen Grauman* Trevor Darrell MIT CSAIL (*) UT Austin…

Automatic Techniques for Studying Attention-Deficit/Hyperactivity Disorder Laura Igual, Antonio Hernandez, Sergio Escalera, Miguel Reyes, Josep Moya, Joan.

Detecting Categories in News Video Using Image Features Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik.

BoVDW: Bag-of-Visual-and-Depth- Words for Gesture Recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Antonio Hernández-Vela.

Intelligent Systems Lab. Recognizing Human actions from Still Images with Latent Poses Authors: Weilong Yang, Yang Wang, and Greg Mori Simon Fraser University,

Probability-based Dynamic Time Warping for Gesture Recognition on RGB-D data All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Group.

A Robust Method of Detecting Hand Gestures Using Depth Sensors Yan Wen, Chuanyan Hu, Guanghui Yu, Changbo Wang Haptic Audio Visual Environments and Games.

Real-Time Activity Monitoring of Inpatients Miguel Reyes, Jordi Vitrià, Petia Radeva and Sergio Escalera Computer Vision Center, Universitat Autònoma de.

Potential Projects RGBD gesture recognition with the Microsoft Kinect Person recognition by parts.

Bag of Features Approach: recent work, using geometric information.

A Study of Approaches for Object Recognition

Spatio-Temporal Sequence Learning of Visual Place Cells for Robotic Navigation presented by Nguyen Vu Anh date: 20 th July, 2010 Nguyen Vu Anh, Alex Leng-Phuan.

1 Visual Information Extraction in Content-based Image Retrieval System Presented by: Mian Huang Weichuan Dong Apr 29, 2004.

GPGPU Volume Classification using SimpleOpenCL Oscar Amoros, Sergio Escalera, Anna Puig and Maria Salamó Computer Vision Center, Universitat Autònoma de.

Local Features and Kernels for Classification of Object Categories J. Zhang --- QMUL UK (INRIA till July 2005) with M. Marszalek and C. Schmid --- INRIA.

The Terrapins Computer Vision Laboratory University of Maryland.

Visual Speech Recognition Using Hidden Markov Models Kofi A. Boakye CS280 Course Project.

Bag-of-Words based Image Classification Joost van de Weijer.

Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu th.

A Thousand Words in a Scene P. Quelhas, F. Monay, J. Odobez, D. Gatica-Perez and T. Tuytelaars PAMI, Sept

Deep Learning for Speech and Language Yoshua Bengio, U. Montreal NIPS’2009 Workshop on Deep Learning for Speech Recognition and Related Applications December.

Introduction Kinect for Xbox 360, referred to as Kinect, is developed by Microsoft, used in Xbox 360 video game console and Windows PCs peripheral equipment.

Internet-scale Imagery for Graphics and Vision James Hays cs195g Computational Photography Brown University, Spring 2010.

AUTOMATIC ANNOTATION OF GEO-INFORMATION IN PANORAMIC STREET VIEW BY IMAGE RETRIEVAL Ming Chen, Yueting Zhuang, Fei Wu College of Computer Science, Zhejiang.

INTERACTING WITH SIMULATION ENVIRONMENTS THROUGH THE KINECT Fayez Alazmi Supervisor: Dr. Brett Wilkinson Flinders University Image 1Image 2 Source : 1.

REU Project RGBD gesture recognition with the Microsoft Kinect Steven Hickson.

Miguel Reyes 1,2, Gabriel Dominguez 2, Sergio Escalera 1,2 Computer Vision Center (CVC) 1, University of Barcelona (UB) 2

Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.

MESA LAB Two papers in icfda14 Guimei Zhang MESA LAB MESA (Mechatronics, Embedded Systems and Automation) LAB School of Engineering, University of California,

Svetlana Lazebnik, Cordelia Schmid, Jean Ponce

Yao, B., and Fei-fei, L. IEEE Transactions on PAMI(2012)

Albert Clapés 1,2, Alex Pardo 2, Miguel Reyes 1,2, Sergio Escalera 1,2, Oriol Pujol 1,2 Computer Vision Center (CVC) 1, University of Barcelona (UB) 2.

A New Fingertip Detection and Tracking Algorithm and Its Application on Writing-in-the-air System The th International Congress on Image and Signal.

Raviteja Vemulapalli University of Maryland, College Park.

ECE 8443 – Pattern Recognition EE 3512 – Signals: Continuous and Discrete Objectives: Spectrograms Revisited Feature Extraction Filter Bank Analysis EEG.

Introducing the Separability Matrix for ECOC coding

Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.

Face Detection Using Large Margin Classifiers Ming-Hsuan Yang Dan Roth Narendra Ahuja Presented by Kiang “Sean” Zhou Beckman Institute University of Illinois.

Action and Gait Recognition From Recovered 3-D Human Joints IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS— PART B: CYBERNETICS, VOL. 40, NO. 4, AUGUST.

Interactive Sand Art Drawing Using RGB-D Sensor

Histograms of Oriented Gradients for Human Detection(HOG)

Vision and SLAM Ingeniería de Sistemas Integrados Departamento de Tecnología Electrónica Universidad de Málaga (Spain) Acción Integrada –’Visual-based.

Week 10 Presentation Wesna LaLanne - REU Student Mahdi M. Kalayeh - Mentor.

REU Project RGBD gesture recognition with the Microsoft Kinect.

Categorical Perception 강우현. Introduction Scalable representations for visual categorization Representation for functional and affordance-based categorization.

SUN Database: Large-scale Scene Recognition from Abbey to Zoo Jianxiong Xiao *James Haysy Krista A. Ehinger Aude Oliva Antonio Torralba Massachusetts Institute.

Age-invariant Face Recognition

Saleh Ud-din Ahmad Dr. Md. Shamim Akhter

An Additive Latent Feature Model

Recognizing Deformable Shapes

Supervised Time Series Pattern Discovery through Local Importance

Calculate HOC on Depth and HOG on RGB and concatenate them

Raviteja Vemulapalli University of Maryland, College Park.

RGB-D Image for Scene Recognition by Jiaqi Guo

Improving Retrieval Performance of Zernike Moment Descriptor on Affined Shapes Dengsheng Zhang, Guojun Lu Gippsland School of Comp. & Info Tech Monash.

Brief Review of Recognition + Context

CS2310 Zihang Huang Project :gesture recognition using Kinect

Human Activity Analysis

Hand Gesture Recognition Using Hidden Markov Models

Recognizing Deformable Shapes

RGBD gesture recognition with the Microsoft Kinect

Problem Image and Volume Segmentation:

Presentation transcript:

Multi Hand Pose Recognition System using Kinect Depth Sensor: Application to Medical Image Navigation Oscar Lopes, Miguel Pousa, Miguel Reyes, Sergio Escalera, and Jordi González All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis

Index Motivation System Architecture Descriptor Gesture Recognition Dataset Application Results Conclusion All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis

Motivation All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Recognize hand poses and gestures with high visual variability. Introduce the recognition process in an automatic medical image navigation application

System architecture All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis RGBD data acquisition Trained spherical descriptors and HMM gesture recognition Medical image navigation A B C

Descriptor All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Sergio Escalera, Alicia Fornés, Oriol Pujol, Josep Lladós, and Petia Radeva, Circular Blurred Shape Model for Multiclass Symbol Recognition, Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions On, Volume 99, pp. 1-10, Digital Object Identifier: /TSMCB , pp. 1-10, IEEE Society, Hanover, USA, ISSN , IF JCR CCIA /103, 2010.

Descriptor All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Rotation invariance Multiclass SVM

Gesture recognition All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Probabilistic HMM gesture recognition using hand trajectories and hand labels

Dataset All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Medical image repository

Application All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis

Results All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Descriptor results improve state-of-the-art approaches (51 categories) MethodsPapersDepth SIFT + Texton + Color Histogram + Spin Images + 3D Bounding BoxesICRA11A64.7 Sparse Distance LearningICRA11B70.2 RGB-D Kernel DescriptorsIROS Hierarchical Matcing PursuitISER Ours ICPR Demonstrator challenge

Conclusion All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Conclusion Future work Use virtual models to learn objects as a composition of parts Use the same descriptor to learn spatial relations of object parts A new, fast, and robust 3D object descriptor Spherical descriptor improves state-of-the-art approaches Gesture recognition using HMM including hand poses and trajectories Fully-functional application using Microsoft Kinect SDK for medical image navigation

Multi Hand Pose Recognition System using Kinect Depth Sensor: Application to Medical Image Navigation Oscar Lopes, Miguel Pousa, Miguel Reyes, Sergio Escalera, and Jordi González All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Thank you!