1. 3D Motion Classification Partial Image Retrieval and Download Multimedia Project Multimedia and Network Lab, Department of Computer Science

2. Sensors and 3D motion capture system 6/11/2016UTD Multimedia and Networking Lab2 Accelerometer Electromyogram Electrocardiogram

3. 3D Motion Capture 3UTD Multimedia and Networking Lab6/11/2016

4. Integration  Analysis  Gait Analysis  Motion correlation and error modeling  Humanoid robotics  Game Control  Disease diagnostic  Motion Classification  Clustering 4UTD Multimedia and Networking Lab6/11/2016

5. 3D Input Data (MoCap & EMG) 3D Mocap data M x 54 Matrix ( M is the total num of Frames ) 5UTD Multimedia and Networking Lab6/11/2016 EMG data

6. Input Data Format  Each Motion is represented by set of joint vectors  Use sliding Windows for feature extractions TibiaFootToe Windows (Time Frame) 6UTD Multimedia and Networking Lab6/11/2016

7. Image data 6/11/2016UTD Multimedia and Networking Lab7

8. Data Analysis 8 Motion capture Data Collection Preprocessing Feature Extraction Data Analysis Gait Cycle Cross-Pair 24 Feature Point Geometric Trans.

9. Project I: Object Semantics in Image Project II: Image Downloader Project III: 3D image Classification 6/11/2016UTD Multimedia and Networking Lab9

10. Project: Object Semantics in Image  Template-match based object semantics  Template-match 6/11/2016UTD Multimedia and Networking Lab10 + =

11. 6/11/2016UTD Multimedia and Networking Lab11 Image template Input Image Visual image semantic SURF(Speeded Up Robust Features) SIFT(Scale-invariant feature transfrom) HOG(Histogram of Gradients) ….

12. Project Goal  Goal: Building Visual Image Semantics using template-match based approach.  Input: Image data(2D)  Training Data : Partial Image data(2D)  Output: related Spatial data(2D, Visual Image Semantics)  Requirement:  Language option: anything 12UTD Multimedia and Networking Lab6/11/2016

13. Project: Annotated Image based Image (and Video) Downloader  DB- Flickr, Goolge Image… 6/11/2016UTD Multimedia and Networking Lab13

14. MIT Labelme project (Image tagging)  http://labelme.csail.mit.edu 6/11/2016UTD Multimedia and Networking Lab14

15. 6/11/2016UTD Multimedia and Networking Lab15 Image Tagging (Annotation) Query Image google Flickr Word based image search Download images

16. Project Goal (II)  Goal: Building Content based Image Downloader  Input: Image data(2D)  Training Data : Labelme Image DB  Output: collections of related Spatial data(2D)  Requirement:  Language option: anything  Lableme matlab toolbox: http://labelme.csail.mit.edu 16UTD Multimedia and Networking Lab6/11/2016

17. Project: 3D motion classification using Mocap data template  Mocap clustering and detection 6/11/2016UTD Multimedia and Networking Lab17 Mocap based motion template 3D image (image sequence) Object tracking Classification

18. Project Goal  Goal: Cluster each motion using any machine leaning techniques to form a set of motions to decide 3D image motion set.  Window size: 360 frames with 108 frames overlapping  Input: Image Sequence data(3D)  Training Data : Mocap data (54D)  Output: Segmented image sequence data(3D)  Requirement:  Language option: anything 6/11/2016UTD Multimedia and Networking Lab18

19. Question? Duk-Jin Kim duk-jin.kim@utdallas.edu @ECSS 4.416 Thank You ! 19UTD Multimedia and Networking Lab6/11/2016