Presentation is loading. Please wait.

Presentation is loading. Please wait.

Probabilistic Analysis of a Large-Scale Urban Traffic Sensor Data Set Jon Hutchins, Alexander Ihler, and Padhraic Smyth Department of Computer Science.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Probabilistic Analysis of a Large-Scale Urban Traffic Sensor Data Set Jon Hutchins, Alexander Ihler, and Padhraic Smyth Department of Computer Science."— Presentation transcript:

1 Probabilistic Analysis of a Large-Scale Urban Traffic Sensor Data Set Jon Hutchins, Alexander Ihler, and Padhraic Smyth Department of Computer Science University of California, Irvine

2 People Counter – Optical Sensor Car Counter – Loop Detector Human Activity Sensors Pe MS

3 Outline Modeling human count data Scale-Up challenges Fault-tolerant model Urban Analysis

4 Outline Modeling human count data Scale-Up challenges Fault-tolerant model Urban Analysis

5 Outline Modeling human count data Scale-Up challenges Fault-tolerant model Urban Analysis

6 Outline Modeling human count data Scale-Up challenges Fault-tolerant model Urban Analysis

7 Time Series of Cumulative Counts TimeCar Count 10:0022 10:0526 10:1026 10:1519 10:2031 10:2528 10:3025

8 One Week of Freeway Observations car count

9 One Week of Freeway Observations car count

10 One Week of Freeway Observations car count

11 Sensor location car count One Week of Freeway Observations

12 Original Model hiddenobserved Observed Count

13 Original Model hiddenobserved Observed Count Normal Count Poisson Rate λ(t)

14 Original Model hiddenobserved Observed Count Normal Count Poisson Rate λ(t) Event State Event Count

15 Original Model hiddenobserved Observed Count Normal Count Poisson Rate λ(t) Event State Event Count OBSERVED COUNT NORMAL COUNT (UNOBSERVED) EVENT COUNT (UNOBSERVED) Time-varying Poisson Markov with Poisson counts

16 Original Model hiddenobserved Observed Count Normal Count Poisson Rate λ(t) Event State Event Count OBSERVED COUNT NORMAL COUNT (UNOBSERVED) EVENT COUNT (UNOBSERVED) Time-varying Poisson Markov with Poisson counts Markov Modulated Poisson Process (MMPP) e.g., see Scott (1998)

17 Time t+1 Event State Observed Count Observed Count Observed Count Event Count Event Count Event Count Poisson Rate λ(t) Normal Count Normal Count Normal Count Poisson Rate λ(t) Poisson Rate λ(t) Time t-1Time t Inference over Time hidden observed

18 Learning and Inference Bayesian Framework –Gibbs sampling to approximate parameters and hidden variables –Forward-backward algorithm –Complexity Linear in the number of time slices For Details see Ihler, Hutchins, Smyth ACM TKDD (Dec 2007)

19 Original Model

20 Urban Scale-Up Sensor LocationsMap of study area 1716 sensors + 7 months = over 100 million measurements

21 Urban Scale-Up Difficult Sensors to Analyze see Bickel et al. Statistical Science (2007)

22 Urban Scale-Up – Original Model Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153

23 Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153 car count p(E) Urban Scale-Up – Original Model

24 Urban Scale-Up - Challenges Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153 car count p(E)

25 Urban Scale-Up - Challenges Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153

26 Urban Scale-Up - Challenges Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153

27 Urban Scale-Up - Challenges Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153

28 Event FractionNumber of Sensors 0 to 10%912 10 to 20%386 20 to 50%265 50 to 100%153 Periods of clear periodic behavior missed by our model Long periods of sensor failure

29 Time t+1 Event State Observed Count Observed Count Observed Count Event Count Event Count Event Count Poisson Rate λ(t) Normal Count Normal Count Normal Count hidden observed Poisson Rate λ(t) Poisson Rate λ(t) Time t-1Time t Original Model Fault State Fault-Tolerant Model

30 Event FractionOriginal Model Number of Sensors Fault-Tolerant Model Number of Sensors 0 to 10%9601285 10 to 20%375242 20 to 50%244117 50 to 100%13772 Fault-Tolerant Model

31

32 Large-Scale Urban Study Event FractionFault-Tolerant Model Number of Sensors 0 to 10%1285 10 to 20%242 20 to 50%117 50 to 100%72

33 Large-Scale Urban Study Event FractionFault-Tolerant Model Number of Sensors 0 to 10%1285 10 to 20%242 20 to 50%117 50 to 100%72

34 Unusual activity detection as a function of day of week and time of day

35

36

37 16:30 Spatial Event

38 16:30 16:40 16:55 17:05 17:20 17:25 17:50 18:05 18:20

39

40

41

42 Model prediction of normal flow Raw flow measurements

43

44

45 Model prediction of normal flow Raw flow measurements

46 Conclusions Extended our earlier work to add a fault-tolerant component Our new model automatically learned normal and anomalous behavior for over 1700 sensors and 100 million measurements This approach has made possible analysis of a large- scale urban traffic sensor data set that was previously considered beyond analysis

Download ppt "Probabilistic Analysis of a Large-Scale Urban Traffic Sensor Data Set Jon Hutchins, Alexander Ihler, and Padhraic Smyth Department of Computer Science."

Similar presentations

EXPLORIS Montserrat Volcano Observatory Aspinall and Associates Risk Management Solutions 1 2 3 4 5 An Evidence Science approach to volcano hazard forecasting.

EXPLORIS Montserrat Volcano Observatory Aspinall and Associates Risk Management Solutions An Evidence Science approach to volcano hazard forecasting.
Bayesian Belief Propagation

Bayesian Belief Propagation
Scaling Up Graphical Model Inference

Scaling Up Graphical Model Inference
Autonomic Scaling of Cloud Computing Resources

Autonomic Scaling of Cloud Computing Resources
Background Reinforcement Learning (RL) agents learn to do tasks by iteratively performing actions in the world and using resulting experiences to decide.

Background Reinforcement Learning (RL) agents learn to do tasks by iteratively performing actions in the world and using resulting experiences to decide.
Darya Chudova, Alexander Ihler, Kevin K. Lin, Bogi Andersen and Padhraic Smyth BIOINFORMATICS Gene expression Vol. 25 no. 23 2009, pages 3114-3120.

Darya Chudova, Alexander Ihler, Kevin K. Lin, Bogi Andersen and Padhraic Smyth BIOINFORMATICS Gene expression Vol. 25 no , pages
Dynamic Bayesian Networks (DBNs)

Dynamic Bayesian Networks (DBNs)
Sensitivity of PCA for Traffic Anomaly Detection Evaluating the robustness of current best practices Haakon Ringberg 1, Augustin Soule 2, Jennifer Rexford.

Sensitivity of PCA for Traffic Anomaly Detection Evaluating the robustness of current best practices Haakon Ringberg 1, Augustin Soule 2, Jennifer Rexford.
Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.
Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution Ping Yan Tim Schoenharl Alec Pawling Greg Madey.

Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution Ping Yan Tim Schoenharl Alec Pawling Greg Madey.
1 Vertically Integrated Seismic Analysis Stuart Russell Computer Science Division, UC Berkeley Nimar Arora, Erik Sudderth, Nick Hay.

1 Vertically Integrated Seismic Analysis Stuart Russell Computer Science Division, UC Berkeley Nimar Arora, Erik Sudderth, Nick Hay.
Yanxin Shi 1, Fan Guo 1, Wei Wu 2, Eric P. Xing 1 GIMscan: A New Statistical Method for Analyzing Whole-Genome Array CGH Data RECOMB 2007 Presentation.

Yanxin Shi 1, Fan Guo 1, Wei Wu 2, Eric P. Xing 1 GIMscan: A New Statistical Method for Analyzing Whole-Genome Array CGH Data RECOMB 2007 Presentation.
Probabilistic reasoning over time So far, we’ve mostly dealt with episodic environments –One exception: games with multiple moves In particular, the Bayesian.

Probabilistic reasoning over time So far, we’ve mostly dealt with episodic environments –One exception: games with multiple moves In particular, the Bayesian.
Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation James Foulds 1, Levi Boyles 1, Christopher DuBois 2 Padhraic Smyth.

Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation James Foulds 1, Levi Boyles 1, Christopher DuBois 2 Padhraic Smyth.
1 Graphical Models in Data Assimilation Problems Alexander Ihler UC Irvine Collaborators: Sergey Kirshner Andrew Robertson Padhraic Smyth.

1 Graphical Models in Data Assimilation Problems Alexander Ihler UC Irvine Collaborators: Sergey Kirshner Andrew Robertson Padhraic Smyth.
Probabilistic Aggregation in Distributed Networks Ling Huang, Ben Zhao, Anthony Joseph and John Kubiatowicz {hling, ravenben, adj,

Probabilistic Aggregation in Distributed Networks Ling Huang, Ben Zhao, Anthony Joseph and John Kubiatowicz {hling, ravenben, adj,
1 In-Network PCA and Anomaly Detection Ling Huang* XuanLong Nguyen* Minos Garofalakis § Michael Jordan* Anthony Joseph* Nina Taft § *UC Berkeley § Intel.

1 In-Network PCA and Anomaly Detection Ling Huang* XuanLong Nguyen* Minos Garofalakis § Michael Jordan* Anthony Joseph* Nina Taft § *UC Berkeley § Intel.
UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering On-line Alert Systems for Production Plants A Conflict Based Approach.

UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering On-line Alert Systems for Production Plants A Conflict Based Approach.
Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.

Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.
1 Distributed Online Simultaneous Fault Detection for Multiple Sensors Ram Rajagopal, Xuanlong Nguyen, Sinem Ergen, Pravin Varaiya EECS, University of.

1 Distributed Online Simultaneous Fault Detection for Multiple Sensors Ram Rajagopal, Xuanlong Nguyen, Sinem Ergen, Pravin Varaiya EECS, University of.

Similar presentations

Ads by Google