1. Dr. Z. R. Ghassabi z.r.ghassabi@gmail.com Spring 2015 z.r.ghassabi@gmail.com Deep learning for Human action Recognition 1

2. Outline Introduction to human action recognition Introduction to deep learning Is deep learning useful for human action recognition? 2

3. Introduction to Human action recognition sensor-based human activity recognitionVision-based human activity recognition

4. Introduction to Human action recognition Segmentation Feature representation Feature Classification Usually hand-crafted: SIFT, HOG, etc…

5. The real challenge: image features

6. Representation Examples of descriptors

7. Unsupervised feature learning Until very recently, learning has not played a major role until the classification stage, at which point much of the input is lost. Now learn from data directly and No engineering/research effort Equally good if not better

8. Hierarchies in high-level vision

12. Deep Learning and AI

15. Unsupervised learning: optimizes Φ from unlabeled data distribution

17. Unsupervised feature learning Distributed representations – many-to-many relationship between concepts and variables Each concept is represented by many variables Each variable participates in the representation of many concepts Distributed color-shape representation

20. Hierarchical sparse coding

21. Unsupervised feature learning Boltzmann machine Deep Neural Networks Convolutional Neural Networks

22. Deep Neural Network for AR A key advantage of DNN is its representation of input features. DNN can model diverse activities with much less training data.

23. Deep Neural Network for AR Supervised : – Restricted Bozltman Machine (RBM) Unsupervised: – Shift-Invariant Sparse Coding RBM and Sparse Coding are fully connected DNN models. Therefore, they do not capture local dependencies of the time series signals.

24. Fully and Locally Connected NN

25. Convolutional NN

26. Advantages of applying CNN to AR Can capture Local Dependency and Scale invariance features of activity signals. variations of the same activity can be effectively captured through the extracted features.

27. CNN for sensor-based AR Consists of one or more pairs of convolution and pooling layers Local dependencies by Convolutional layers Scale-invariance by max- pooling layers

28. Activity Recognition

29. Criticism on Deep Learning Computational intensive A lot of parameters to tune