1. 3D Motion Data Mining Multimedia Project Multimedia and Network Lab, Department of Computer Science

2. Introduction  3D Motion Capture 2UTD Multimedia and Networking Lab2/19/2016

3. Integration  Gait Analysis  3D Motion Capture  Motion correlation and error modeling  Humanoid robotics  Game Control  Disease diagnostic  Motion Clacification 3UTD Multimedia and Networking Lab2/19/2016

4. Gait?  Study of Human Walk (Lower Limbs)  Terminology  Gait cycle: Begins when one foot contacts the ground and ends when that foot contacts the ground again 4UTD Multimedia and Networking Lab2/19/2016

5. 3D Input Data (MoCap & EMG) 3D Mocap data M x 54 Matrix ( M is the total num of Frames ) 5UTD Multimedia and Networking Lab2/19/2016 EMG data

6. Input Data Format  Each Motion is represented by set of joint vectors  Use sliding Windows for feature extractions TibiaFootToe Windows (Time Frame) 6UTD Multimedia and Networking Lab2/19/2016

7. Motion Data Analysis Flow 7 Motion capture Data Collection Preprocessing Feature Extraction Data Analysis Gait Cycle Cross-Pair 24 Feature Point Geometric Trans.

8. Project I Content-Based Indexing(HS Grid). Project II Motion Classification using PMK Project III Motion Animation using 3D Human Locomotion Data. 8UTD Multimedia and Networking Lab2/19/2016

9. Project I: Content-Based Indexing WSVD IAV MoCapEMG FCMFCM Reduced dimensional vector where, c - pre-determined number of clusters M – Total number of overall windows center - center/median points for all c clusters in k-d space U - “degree of membership” for each M points with respect to each cluster. +  Input Data

10. Hierarchical Spatial Grid(HSG) 10UTD Multimedia and Networking Lab2/19/2016

11. HSG (2D) 1 2 3

12. Even distribution transformation  Transforming the points into corresponding rank index on each dimension  rank index (A(i))= r where, array A with non-zero values, non-increasing ordered array A_RI, A_RI(r)=A(i)

13. Similarity search  Given a query Q( using IAV, WSVD, and FCM)  Q  [q j,min, q j,max]  Where  Distance

14. Project Goal  Goal: Building Disk base Input/output functions for Hierarchical Spatial Grid Indexing Structure  Input: Spatial data(2D,3D,…)  Input query : Spatial data(2D,3D,…)  Output: related Spatial data(2D,3D,…)  Requirement:  Disk based input/output Indexing Structure  Language option: C or JAVA (prefer C)  Reference : G. Pradhan and B. Prabhakaran, " Indexing 3D Human Motion Repositories For Content-based Retrieval," IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 13, NO. 5,, SEPTEMBER 2009  http://multimedia.utdallas.edu/pub.jsp 14UTD Multimedia and Networking Lab2/19/2016

15. Project II: Pyramid Matching Kernel  Place multi-dimensional, multi-resolution grid over point sets  Consider points matched at finest resolution where they fall into same grid cell  Approximate similarity between matched points with worst case similarity at given level No explicit search for matches! Pyramid match kernel measures similarity of a partial matching between two sets:

16. Project II: Pyramid Matching Kernel  Pyramid Match Kernel?  measures similarity of a partial matching between two sets  Place multi-dimensional, multi-resolution grid over point sets  Consider points matched at finest resolution where they fall into same grid cell  Approximate similarity between matched points with worst case similarity at given level 2/19/2016UTD Multimedia and Networking Lab16

17. Pyramid match  http://www.cs.utexas.edu/~grauman/research/projects/pmk/pmk_projectpage.htm 2/19/2016UTD Multimedia and Networking Lab17

18. PMK : Approximate partial match similarity 2/19/2016UTD Multimedia and Networking Lab18 Number of newly matched pairs at level i Measure of difficulty of a match at level i

19. PMK Weights inversely proportional to bin size Normalize kernel values to avoid favoring large sets measure of difficulty of a match at level i histogram pyramids number of newly matched pairs at level i

20. Project II (Cont.)  Goal: Motion Classification using Pyramid Matching Kernel (PMK) for fast matching motion features.  Feature Extraction  3D Geometric feature extractions(Detail points are provided Later)  Classification: HMM or GMM  Programming Language: C, JAVA, or Matlab  Reference :  K. Grauman and T. Darrell. “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Beijing, China, October 2005.  C++ code: http://people.csail.mit.edu/jjl/libpmk/ 20UTD Multimedia and Networking Lab2/19/2016

21. Project III  Goal: Motion animation Tool with selected frames. e.g., 21UTD Multimedia and Networking Lab2/19/2016

22. Question? Thank You ! 22UTD Multimedia and Networking Lab2/19/2016