Activity Detection in Videos Riu Baring CIS 8590 Perception of Intelligent System Temple University Fall 2007

Outline Background Related Work The Model Normal Count Event Count

Activity Detection Problems A process like e.g., traffic flow, crowd formation, or financial electronic transactions is unfolding in time. We can monitor and observe the flow frequencies at many fixed time points. Typically, there are many causes influencing changes in these frequencies.

Causes for Change Possible causes for change include: a) changes due to noise; i.e., those best modeled by e.g., a Gaussian error distribution. b) periodic changes; i.e., those expected to happen over periodic intervals. c) changes not due to either of the above: these are usually the changes we would like to detect.

An Example: Building Data 3 months of “people count” 30 minutes Calit2 UC Irvine Campus

Another Example: Traffic Data 6 months of estimated vehicle count Every 5 minutes Glendale on-ramp to 101N, Los Angeles

More Examples Detecting ‘Events’, which are not pre- planned, involving large numbers of people at a particular location. Detecting ‘Fraudulent transactions’. We observe a variety of electronic transactions at many time intervals. We would like to detect when the number of transactions is significantly different from what is expected.

Related Work Keogh et al. – KDD ‘02  Quantize real-valued time-series into finite set of symbols  Then use a Markov model to detect surprising patterns in the symbol sequence Guralnik and Srivastava – KDD ‘99  Iterative likelihood-based method for segmenting a time-series into homogeneous regions Salmenkivi and Mannila (2005)  Segmenting sets of low-level time-stamped events into time- periods of relatively constant intensity using a combination of Poisson models and Bayesian estimation methods Kleinberg – KDD ‘02  method based on an infinite automaton could be used to detect bursty events in text streams

Related Work All approaches share a common goal  detection of novel and unusual data points or segments in time-series. None focuses on detection of bursty events embedded in time series of counts that reflect the normal diurnal and calendar patterns of human activity. GOAL: To automatically detect the presence of unusual events in the observation sequence.

Background Markov-modulated Processes (Scott, 1998)  Analysis of Web Surfing Behavior (Scott and Smyth, 2005)  Telephone Network Fraud Detection (Scott, 2000) Ihler et al (KDD 2006) developed a framework for building a probabilistic model of time-varying counting process in which a superposition of both time-varying but regular (periodic) and aperiodic processes were observed.

Method I A Baseline Model  Where Threshold Adequate for:  Events interspersed in the data are sufficiently few  Events are sufficiently noticeable.

Chicken and Egg Method I Baseline Model Ideal Model

Method I False Positive, Persistence, and Duration Baseline ModelBaseline Model – Lower Threshold

Method II – Ideal Model Observed Count Normal Count (Unobserved) Event Count (Unobserved )

Normal Count Modeling Periodic Count Data d(t) = {1, …, 7} Sunday = 1, … h(t) = interval i.e. half-hour

Periodic Components Poisson Process Rate Day Effect Time of Day Effect

Event Count: The Process N E Events signify times during which there are higher frequencies which are not due to periodic or noise causes. We can model this by introducing a binary latent process z(t) and assuming z(t)=1 for such events and z(t)=0 if not. P(z(t)=1|z(t-1)=0)= 1-z 00 ; P(z(t)=0|z(t-1)=0)= z 00; P(z(t)=1|z(t-1)=1)= z 11 ; P(z(t)=0|z(t-1)=1)= 1-z 11 i.e., if there is no event at time t-1, the chance of an event at time t is 1-z 00 Modeling Rare Persistent Events

Graphical Model

Priors for event probabilities Beta distributions: priors for the z’s. and z 11 analogously. This characterizes the behavior of the underlying latent process. The hyperparameters a,b are designed to model that behavior.

Priors for event probabilities Recall that N 0 (t) (the non-event process) characterizes periodic and noise changes. The event process N E (t) characterizes other changes. N E (t) is 0 if z(t)=0 and Poisson with rate γ(t) if z(t)=1. So, if there is no event, N(t)=N 0 (t). If there is an event, the frequency due to periodic or noise changes is N(t)=N 0 (t)+N E (t) The rate γ (t) is itself gamma with parameters a E and b E. Hence (by conjugacy) it is marginally negative binomial (NB) with p=(b E /(1+b E ) and n=N(t).

Gibbs Sampling Gibbs sampling works by simulating each parameter/latent variable conditional on all the rest. The λ’s are parameters and the z’s,N’s are the latent variables. The resulting simulated values have an empirical distribution similar to the true posterior distribution. It works as a result of the fact that the joint distribution of parameters is determined by the set of all such conditional distributions.

Gibbs Sampling Given z(t)=0 and the remaining parameters, Put N 0 (t)=N(t) and N E (t)=0. If z(t)=1, simulate N E (t) as negative binomial with parameters, N(t) and b E /(1+b E ). Put N 0 (t)=N(t)-N E (t). To simulate z(t), define

More of Gibbs Sampling Then, if the previous state was 0, we get:

Gibbs Sampling (Continued) Having simulated z(t), we can simulate the parameters as follows: Where ‘N day ’ denotes the number of ‘day’ units in the data, ‘N hh ’ denotes the number of ‘hh’ periods in the data.

Result: Building Data

Result: Freeway Traffic Data

Result: 2600 frames

Discussion Poisson process (nonhomogeneous). Able to detect activity at the expected frame.

Future Work Histogram of direction implementation

References 1.A. Ihler, J. Hutchins, and P. Smyth, “Adaptive event detection with time-varying Poissons process,” KDD S. L. Scott and P. Smyth, “The Markov modulated Poisson process and Markov Poisson cascade with applications to web traffic data,” Bayesian Statistics, vol. 7, pp , S. L. Scott, “Detecting network intrusion using a Markov modulated nonhomogeneous Poisson process,” S. L. Scott, “Bayesian methods and extensions for the two state Markov modulated Poisson process,” Ph.D. dissertation, Harvard University, Dept. of Statistics, 1998.