3D Motion Data Mining Multimedia Project Multimedia and Network Lab, Department of Computer Science

Introduction  3D Motion Capture 2UTD Multimedia and Networking Lab2/19/2016

Integration  Gait Analysis  3D Motion Capture  Motion correlation and error modeling  Humanoid robotics  Game Control  Disease diagnostic  Motion Clacification 3UTD Multimedia and Networking Lab2/19/2016

Gait?  Study of Human Walk (Lower Limbs)  Terminology  Gait cycle: Begins when one foot contacts the ground and ends when that foot contacts the ground again 4UTD Multimedia and Networking Lab2/19/2016

3D Input Data (MoCap & EMG) 3D Mocap data M x 54 Matrix ( M is the total num of Frames ) 5UTD Multimedia and Networking Lab2/19/2016 EMG data

Input Data Format  Each Motion is represented by set of joint vectors  Use sliding Windows for feature extractions TibiaFootToe Windows (Time Frame) 6UTD Multimedia and Networking Lab2/19/2016

Motion Data Analysis Flow 7 Motion capture Data Collection Preprocessing Feature Extraction Data Analysis Gait Cycle Cross-Pair 24 Feature Point Geometric Trans.

Project I Content-Based Indexing(HS Grid). Project II Motion Classification using PMK Project III Motion Animation using 3D Human Locomotion Data. 8UTD Multimedia and Networking Lab2/19/2016

Project I: Content-Based Indexing WSVD IAV MoCapEMG FCMFCM Reduced dimensional vector where, c - pre-determined number of clusters M – Total number of overall windows center - center/median points for all c clusters in k-d space U - “degree of membership” for each M points with respect to each cluster. +  Input Data

Hierarchical Spatial Grid(HSG) 10UTD Multimedia and Networking Lab2/19/2016

HSG (2D) 1 2 3

Even distribution transformation  Transforming the points into corresponding rank index on each dimension  rank index (A(i))= r where, array A with non-zero values, non-increasing ordered array A_RI, A_RI(r)=A(i)

Similarity search  Given a query Q( using IAV, WSVD, and FCM)  Q  [q j,min, q j,max]  Where  Distance

Project Goal  Goal: Building Disk base Input/output functions for Hierarchical Spatial Grid Indexing Structure  Input: Spatial data(2D,3D,…)  Input query : Spatial data(2D,3D,…)  Output: related Spatial data(2D,3D,…)  Requirement:  Disk based input/output Indexing Structure  Language option: C or JAVA (prefer C)  Reference : G. Pradhan and B. Prabhakaran, " Indexing 3D Human Motion Repositories For Content-based Retrieval," IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 13, NO. 5,, SEPTEMBER 2009  14UTD Multimedia and Networking Lab2/19/2016

Project II: Pyramid Matching Kernel  Place multi-dimensional, multi-resolution grid over point sets  Consider points matched at finest resolution where they fall into same grid cell  Approximate similarity between matched points with worst case similarity at given level No explicit search for matches! Pyramid match kernel measures similarity of a partial matching between two sets:

Project II: Pyramid Matching Kernel  Pyramid Match Kernel?  measures similarity of a partial matching between two sets  Place multi-dimensional, multi-resolution grid over point sets  Consider points matched at finest resolution where they fall into same grid cell  Approximate similarity between matched points with worst case similarity at given level 2/19/2016UTD Multimedia and Networking Lab16

Pyramid match  2/19/2016UTD Multimedia and Networking Lab17

PMK : Approximate partial match similarity 2/19/2016UTD Multimedia and Networking Lab18 Number of newly matched pairs at level i Measure of difficulty of a match at level i

PMK Weights inversely proportional to bin size Normalize kernel values to avoid favoring large sets measure of difficulty of a match at level i histogram pyramids number of newly matched pairs at level i

Project II (Cont.)  Goal: Motion Classification using Pyramid Matching Kernel (PMK) for fast matching motion features.  Feature Extraction  3D Geometric feature extractions(Detail points are provided Later)  Classification: HMM or GMM  Programming Language: C, JAVA, or Matlab  Reference :  K. Grauman and T. Darrell. “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Beijing, China, October  C++ code: 20UTD Multimedia and Networking Lab2/19/2016

Project III  Goal: Motion animation Tool with selected frames. e.g., 21UTD Multimedia and Networking Lab2/19/2016

Question? Thank You ! 22UTD Multimedia and Networking Lab2/19/2016