CMU July 29, 2006 Pittsburgh, PA, USA Machine Learning Algorithms for Surveillance and Event Detection Denver Dash – Intel, Corp. Terran Lane – University of New Mexico Dragos Margineantu – The Boeing Company Weng-Keen Wong – Oregon State Univ. Workshops

Workshop Sponsors The Boeing Company Intel

Event Detection Biosurveillance Example: Detect if there is a disease outbreak in a city as early as possible Approach: Monitor the total number of Emergency Department visit in the city each day.

Event Detection Obtain Emergency Department data from the past year Fit a Gaussian to this data Raise an alert when the daily number of ED visits exceeds a threshold 0 x 50 35

Event Detection An interesting event occurs when the # of ED visits per day exceeds the threshold. If it corresponds to a real disease outbreak, it is a true positive. Otherwise, it is a false positive.

“Interesting” Events and their Detection Knowledge Data Decisions Model Event Detection Process Problem/ Environment Problem/ Environment Our World

P(x) 0 x Model learned from x 1 – x 100, and on expert/prior knowledge Interesting Events With Respect To… x x x x x x x x x x ……

Events of Interest are observations with likelihood  (very small?) of occurrence with respect to The model M that is believed to have generated the observations The other observations X that are available P( x i | M, X ) =  Interesting Events with Respect To…

Complex Forms of Data Primary KeyDateTimeProdromeSexAgeHome Location Many more… 1006/1/039:12FeverM20sNE… 1016/1/0310:45DiarrheaF40sNE… 1026/1/0311:03RespiratoryF60sNE… 1036/1/0311:07DiarrheaM60sE… : : : : : : : : JAKARTA, Indonesia (AP) -- Researchers scouring swamps in the heart of Borneo island have discovered a venomous species of snake that can change its skin color, the conservation group WWF announced Tuesday. The ability to change skin color is known in some reptiles, such as the chameleon, but scientists have seen it rarely with snakes and have not yet understood this phenomenon, the group said in a statement....

Event Detection Tasks Intrusion detection / network security Security monitoring Fraud detection Biosurveillance Traffic incident detection Detection of interesting differences between images Detection of potential causes for instability in dynamic systems or control loops Quality control in manufacturing Topic detection Sensor network monitoring Aircraft / train / vehicle maintenance monitoring Fault detection Activity monitoring Supernova detection Weather modeling Data cleaning Detection of regions of increased brain activity from fMRI data And many more…

Features Shared by Most Event Detection Tasks Event detection is difficult or time consuming for human experts Interesting events are usually rare Detecting an interesting event can have a significant impact Difficult to capture all the conditions that make an event “interesting” Evaluation of algorithms is difficult

Not Typical Machine Learning Standard supervised learning approaches are unsatisfactory: –few or no positive examples, plenty of negatives –new forms of interesting events appear Standard unsupervised learning approaches are unsatisfactory: –skewed distributions –in many cases, not just looking for outliers

Standard ML Event Detection Approaches One-class classification of “normal” observations; every other instance considered a potential “important event” Unsupervised clustering + post processing Multi-Stage Event Detection: a standard ML approach + filtering of false positives + Incorporation of background knowledge

Research Questions Event Detection approaches for complex data (video, text, spatio-temporal, relational) Sensor fusion Incorporating domain knowledge into the detection models Validation and testing of Event Detection Algorithms & Tools: –Statistical tests –Testbeds for anomaly detection systems Online Event Detection Defining the “interestingness” of an event (active learning?) Explaining why an event is interesting –Effective visualization techniques Event Detection in adversarial environments

Session 1 (9:20-10:50) 9:20-10:00 Interactive Event Detection in Audio and Video Rahul Sukthankar 10: :25 Framework for Anomalous Change Detection – James Theiler, Simon Perkins 10:25-10:50 Shape Outlier Detection Using Pose Preserving Dynamic Shape Models Chan-Su Lee, Ahmed Elgammal Coffee Break (10:50-11:20) Session 2 (11:20-12:40) 11:20-12:00 Detection of Stepping-Stones: Algorithms and Confidence Bounds Shobha Venkataraman 12:00-12:20 Distributed Probabilistic Inference for Detection of Weak Network Anomalies Denver Dash 12:00-12:20 Learning Sequential Models for Detecting Anomalous Protocol Usage Lloyd Greenwald Lunch (12:40-14:05) Schedule

Session 3 (14:05-15:45) 14:05-14:45 Forecast, Detect, Intervene: Anomaly Detection for Time Series Deepak Agarwal 14:45-15:25 Bayesian Biosurveillance Greg Cooper 15:25-15:45 A Wavelet-based Anomaly Detector for Early Detection of Disease Outbreaks Thomas Lotze, Galit Shmueli, Sean Murphy, Howard Burkom Coffee Break (15:45-16:15) Session 4 (16:15-17:35) 16:15-16:45 Towards a Learning Traffic Incident Detection System Tomas Singliar, Milos Hauskrecht 16:45-17:05 Bayesian Anomaly Detection (BAD v1.0) Tim Menzies, David Allen 17:05-17:35 Discussion Panel Schedule