GraphiCon 2008 | 1 Trajectory classification based on Hidden Markov Models Jozef Mlích and Petr Chmelař Brno University of Technology, Faculty of Information Technology Božetěchova Brno, Czech Republic GraphiCon

GraphiCon 2008 | 2 Abstract This paper presents a method for statistical modeling and classification of motion trajectories using Hidden Markov Models. Mass recordings from visual surveillance are processed to extract objects trajectories. Hidden Markov Models of classes of behaviour are created upon some annotated trajectories. In this way, information about complex object behaviour of objects can be discovered. Additionally, an experiment shows the successful application of Hidden Markov Models on trajectories of people in an underground station in Roma. Finally, a comparison of efficiency on different data sets, is discussed.

GraphiCon 2008 | 3 Overview 1.Motivation 2.Goal 3.Trajectory 4.Hidden Markov Models 5.System overview 6.Experiments 7.Conclusion

GraphiCon 2008 | 4 Motivation Data produced by surveillance cameras is a potential source of useful information. It is difficult to obtain, but highly demanded. Computer vision techniques are still underdeveloped for... Example of classification – class 1 normal situation

GraphiCon 2008 | 5 Goal The goal is to classify people behavior and to identify potentially dangerous situations. Examples: crime (vandalism, fights, turnpikes jumping) personal injury big disaster (bomb attack, underground fire) Based on their trajectory. IST Caretaker Content Analysis and Retrieval Technologies to Apply Knowledge Extraction to massive Recording object detection event detectors data mining

GraphiCon 2008 | 6 Caretaker overview – to be deleted Alarm mode Object Detectors – Person – Dog – Bicycle – Left Luggage Event detectors – Audio events (gunshots, screaming) – Object intersects area – Object Motion based events Data mining mode Knowledge discovery People counting

GraphiCon 2008 | 7 Trajectory It is the path a moving object follows through space. Represented as a sequence of spatio-temporal points. Knowledge discovery in the trajectory data leads to the linear dynamic and Markov models. The presented approach is based on: Supervised learning and classification using Hidden Markov Models.

GraphiCon 2008 | 8 Hidden Markov Models Statistical model for sequential data. Described by transition probabilities and state probability distribution function.

GraphiCon 2008 | 9 System overview A black box. Training phase and classification. Input: [x, y, dx, dy] t Output: classes probabilities and conclusion about behavior. Schema of the trajectory processing

GraphiCon 2008 | 10 Training phase Definition of all normal trajectories in scene. Data annotation. Model estimation using Baum-Welch algorithm: Setup of threshold between normal and abnormal behavior. Definition of normal trajectories in scene

GraphiCon 2008 | 11 Classification phase Computing of Viterbi likelihood. Selecting class with maximal likelihood. Thresholding. Definition of normal trajectories in scene

GraphiCon 2008 | 12 Experimental results Filtered data: Containing only well defined trajectories Accuracy about 92% Non­filtered data: containing also merged trajectories and ambiguous trajectories accuracy about 37%

GraphiCon 2008 | 13 Conclusion Method for trajectory classification. Object trajectories from surveillance recordings. Annotated trajectories to create Hidden Markov Models. Performed experiments approved the classification might be seriously successful when following conditions are met: – well defined (training) trajectories – well annotated abnormalities in trajectories – accurate object tracking technique In this way, the proposed method enables also detection of objects with unusual behaviour.