1. Using decision trees to build an a framework for multivariate time- series classification 1 Present By Xiayi Kuang

2. Outline Introduction – Goal, context & challenges Proposed approach – Overview – Clustering stage – Classification stage Experiments – Visual Surveillance Context Video sequence 2

3. Introduction Goal To build a flexible and generic system that can model possibly complex spatio-temporal events 3

4. Introduction Context 4 Computer vision applications – Ex: Automated Visual surveillance Medical domain – Ex: electronic graph classification Gesture Recognition

5. Introduction Challenges with MTS classification – Machine learning techniques cannot be directly applied to time-series data – Large dataset with many temporal attributes are hard to handle because of the number of combinatorial possibilities between the features – Invariance of the events with respect to times 5

6. Proposed Approach Complex spatio-temporal Event Multivariate time-series data: 6 Time- steps t1t2t3t…tn-1tn Attribute 1 f1 1 f2 1 f3 1 f… 1 fn-1 1 fn 1 Attribute 2 f1 2 f2 2 f3 2 f… 2 fn-1 2 fn 2 Attribute … f1 … f2 … f3 … f… … fn-1 … fn … Attribute m-1 f1 m-1 f2 m-1 f3 m-1 f… m-1 fn-1 m-1 fn m-1 Attribute m f1 m f2 m f3 m f… m fn-1 m fn m Pattern 1Pattern 2 before= event X

7. Proposed approach Framework Overview 7 Local Patterns (LP) extraction Temporal relation between LP

8. Proposed approach Theory: The Decision Tree (DT) and the ensemble of randomized trees (ERT) Top down induction Decision Trees – At each node, the ‘best split’ is chosen according to a specific distance measure (splitting criteria) – Clustering trees: Each node and leaf is a cluster – Classification trees: Each leaf is labeled with a class Ensemble of randomized trees – The ‘best split’ is chosen among X trials – 10 to 100 trees are built and we combine their predictions with a simple majority vote – Reduce overfitting and increase robustness and accuracy 8 Node 1Node 2Node 4Leaf Node3Node 5LeafNode 6Leaf

9. Proposed approach Clustering trees Goal: – To extract local patterns at each time-step How? – Each time-step in the training sample are clustered independently Algorithm – Multiple clustering trees – Split function at each node: Choose one attribute and one threshold that maximize a splitting criteria – Splitting criteria: Minimizing the intra-cluster distance and maximizing the inter-cluster distance  2 choices: Supervized clustering: The distance metric is the class entropy (each time-step of one sequence inherit the class label of its sequence) Unsupervized clustering: The distance can be the Euclidian distance (usual one) And then? – Each node is a pattern and is tagged – Each frame is labeled with the appropriate patterns 9

10. Proposed approach Clustering trees Time- steps t1t2t3t…tn-1tn Channel 1 f1 1 f2 1 f3 1 f… 1 fn-1 1 fn 1 Channel 2 f1 2 f2 2 f3 2 f… 2 fn-1 2 fn 2 Channel … f1 … f2 … f3 … f… … fn-1 … fn … Channel m-1 f1 m-1 f2 m-1 f3 m-1 f… m-1 fn-1 m-1 fn m-1 Channel m f1 m f2 m f3 m f… m fn-1 m fn m 10 t1 pattern 1 pattern 4 pattern X t2 pat. 2 pat. 8 pat. 9 pat. Y t3 pat. 1 pat. 4 pat. X pat. Y t… pat. 1 pat. 3 pat. 8 pat. X pat. Y tn-1 pat. 2 pat. 3 pat. 4 pat. 9 pat. Y tn pat. 1 pat. 4 pat. X pat. Y Preprocessing Input Data Clustering stage Multivariate time-series

11. Proposed approach Classification trees Goal: – To model the temporal relation between the local patterns Algorithm: – Ensemble of randomized trees Split function at each node: Splitting criteria: – Maximizing the Normalized information gain over X randomly selected tries We combine the trees prediction with a simple majority vote 11

12. Proposed approach Classification trees 12 t1 pattern 1 pattern 4 pattern X t2 pat. 2 pat. 8 pat. 9 pat. Y t3 pat. 1 pat. 4 pat. X pat. Y t… pat. 1 pat. 3 pat. 8 pat. X pat. Y tn-1 pat. 2 pat. 3 pat. 4 pat. 9 pat. Y tn pat. 1 pat. 4 pat. X pat. Y Event A Event B

13. Application Visual surveillance events Additional Challenges in automated visual surveillance – Segmentation and tracking are often not robust – Events are semantically complex and often subjective – Few training data available for interesting events – Events are extremely variable in length 13

14. Application video sequences Dataset: 7 events, no ground truth. Features: position, speed and size of the blobs 14

15. 15

16. Application video sequences Results 16

17. 17 Thanks 12/03/2008