1. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science November 12, 2000Memory Management for High- Performance Applications - Ph.D. defense - Emery Berger Switching Kalman Filters for Prediction and Tracking in an Adaptive Meteorological Sensing Network Victoria Manfredi, Sridhar Mahadevan, Jim Kurose SECON’05 September 28, 2005

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Introduction CASA –Collaborative Adaptive Sensing of the Atmosphere –Distributed, collaborative, adaptive radar network Where/what, when, and how to sense? Configure radars based on predicted locations of meteorological phenomena Our focus? Storm cells

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Problem Track storm cells over time Use predicted storm locations to identify future radar configurations Constraints/Assumptions –Existing meteorological algorithms that identify storms from raw radar data –Tracking only a single storm cell –Less than 30 seconds for prediction

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Outline Meteorological vs. Statistical Approaches Kalman Filter Approaches Experiments Conclusions Future work

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 Storm Tracking Extrapolation –SCIT: linear least-squares over last five points [JMWMSET98] –Titan: extrapolation plus cross-correlation [DW93] –K-means to identify storm clusters, smooth storm movements with Kalman filter [LRD03] Knowledge-intensive –Gandolf: model meteorological evolution of each storm [PHCH00] –Growth and Decay Storm Tracker: track encompassing storm instead of storm cell [WFHM98] –Ensemble Kalman Filter: project a set of points forward in time using a meteorological model [E03] Computationally Expensive Simpler

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Goal: –Good predictions –Satisfy real-time constraints Meteorological Approaches –Extrapolation –Knowledge-intensive Meteorological vs. Statistical Other Statistical Approaches –Kalman filter: linear, Gaussian, state –Switching Kalman filter: non-linear, Gaussian, state SCIT: Linear least-squares regression [JMWMSET98] Linear, Gaussian, no state (Developed at NSSL, Kurt Hondl)

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Kalman Filter (KF) t=1 t=2t=3 State Observation State transitions : x t+1 = Ax t + N[0,Q] Observations : y t+1 = Bx t+1 + N[0,R] Model (linear) dynamics of an object States, Obs: Linear function plus Gaussian noise X = [lat, long, v lat, v long ] Y = [lat, long]

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 Switching Kalman Filter (SKF) State transitions : x t+1 = A i x t + N[0,Q i ] Observations : y t+1 = B i x t+1 + N[0,R i ] Model object dynamics with set of Kalman filters Piecewise linear approximation of nonlinear path t=1 t=2 t=3 State Observation X = [lat, long, v lat, v long ] Y = [lat, long] Switch S = which Kalman filter Switch A 1, Q 1, B 1, R 1,  1,  1 A 2, Q 2, B 2, R 2,  2,  2 A 3, Q 3, B 3, R 3,  3,  3

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 t=4 t=1t=2 t=3 State Observation Kalman Filter X = [ lat, long, v lat, v long ] Y = [ lat, long] Observe Infer Predict Inference + Prediction Least-Squares Use five most recent observations only

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 t=4 Inference + Prediction t=1t=2 t=3 State Observation Switch Switching Kalman Filter X = [ lat, long, v lat, v long ] Y = [ lat, long] S = which Kalman filter Switch values unknown  inference in SKF is hard t=1: K possible states with K Kalman filters t=2: K 2 possible states … t=n: K n possible states Solution? Approximate inference: Generalized pseudo-Bayesian –Order 2: Collapse over state, switches two time steps ago Prediction –Compute most likely sequence of switches –Use corresponding KFs to infer hidden state and predict next state Observe Infer Collapse Most Likely Predict

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 Experiments Compare Kalman filter, switching Kalman filter and linear least-squares regression (SCIT [JMWMSET98] ) on tracking and predicting storm locations Data –35 storm tracks courtesy of Kurt Hondl at NSSL –Each track is a sequence of latitude and longitude coordinates –Range in length from ten to 30 data points –Identified using SCIT [JMWMSET98]

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12  Compare hand-coded parameters with learned parameters Kalman filter, switching Kalman filter parameters –What are dynamics of storm cells? –How to obtain model of dynamics? Parameter Learning Expectation-maximization to learn parameters –E-step: Assume parameters are known, compute expected values of hidden variables (state, switch) –M-step: Assume values of hidden variables are known, compute maximum likelihood parameters SKF KF hand-coded KF-EMSKF-EM learned

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Results (Not suprisingly) On nonlinear track, switching Kalman filter performs better

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 Results On linear tracks, both methods perform similarly

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Results MethodAverage 1-Step RMSE Latitude Longitude Average 2-Step RMSE Latitude Longitude KF0.22480.19230.33590.2871 KF-EM0.19670.16800.26800.2389 SKF0.19140.17020.26420.2421 SKF-EM0.25770.20700.39480.3110 Least- Squares 0.21140.21070.30300.3081 0.1° lat = 6.9 miles 0.1° long  6.9 miles

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 # of KFsTime Required for 1-Step Prediction (seconds) avgmaxmin 10.0001550.0008640.000103 40.0016890.0064790.001138 80.0066550.0283750.004553 Timing  Within timing constraints

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Conclusions and Future Work Although tracks identified with least-squares method (SCIT), KF-EM and SKF have lower prediction error Can learn storm dynamics to improve prediction model Future work –Obtain more data to improve learned model Especially SKF –Incorporate meteorological information –Track multiple targets, other meteorological phenomena –Combine decision-making with prediction –Add higher layers to the SKF

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Thank You. Questions?

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 References [JMWMSET98] J. Johnson, P. MacKeen, A. Witt, E. Mitchell, G. Stumpf, M. Eilts, and K.. Thomas. The storm cell identification and tracking algorithm: An enhanced WSR-88D algorithm. Weather and Forecasting, 13:263-276, 1998. [DW93] M. Dixon and G. Weiner. TITAN: Thunderstorm identification, tracking analysis and nowcasting a radar based methodology. J. Atmos. Ocean. Tech., 10:785-797, 1993. [LRD03] V. Lakshamanan, R. Rabin, and V. DeBrunner. Multiscale strom identification and forecast. Journal of Atmospheric Research, 367-380, 2003. [PHCH00] C.Pierce, P. Hardaker, C. Collier, and C. Haggett. GANDOLF: A system for generating automated nowcasts of covective precipitation. Meteorol. Appl., 7:341- 360, 2000. [WFHM98] M. Wolfson, B. Forman, R. Hallowell, and M. Moore. The growth and decay storm tracker. American Meteorological Society 79th Annual Conference, 1999. [E03] G. Evensen. The ensemble Kalman filter: Theoretical formulation and practical implementatioon. Ocean Dynamics, 53:343-367, 2003.

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Generalized Pseudo-Bayesian Values of switch variables are unknown  inference in SKF is hard –Time step 1: K possible states with K Kalman filters –Time step 2: K 2 possible states – … –Time step n: K n possible states Solution? Approximate inference –Generalized pseudo-Bayesian –Variational –Sampling –Viterbi

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Generalized Pseudo-Bayesian Order two generalized pseudo-Bayesian algorithm –Collapse over everything two time steps ago –x = mean, V = covariance, W = switch probability (x j, V j ) = Collapse(x ij, V ij, W i ) x j = ∑ i W i x ij V j = ∑ i W i V ij + ∑ i W i (x ij -x j )(x ij -x j ) T Covariance depends on observations through x

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Linear Least-Squares Regression Given a set of points, find best fit line Assumes constant covariance Solve Ax=b for coefficient vector x If too many equations, problem is over-constrained Error: difference between what model says response value should be and actual value –Ax - b Minimize squared vertical distance to best fit line –||Ax -b|| 2 So instead solve A T Ax=A T b for coefficient vector x

23. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 Kalman Filter (KF) Assume A = Identity and Q = zero matrix –Then for all t, x t+1 = x t This can be used to derive the recursive least- squares update equations Implies least-squares assumes constant covariance while KF does not State transitions : x t+1 = Ax t + N[0,Q] Observations : y t+1 = Bx t+1 + N[0,R]

24. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 Results

25. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 25 t=4 t=5 Inference + Prediction t=1t=2 t=3 State Observation Kalman Filter Observe Infer Predict X = [ lat, long, v lat, v long ] X = [ lat, long]

26. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 26 t=4 t=5 Inference + Prediction t=1t=2 t=3 State Observation Switch Switching Kalman Filter Least-Squares Use five most recent observations only Observe Infer Predict Collapse X = [ lat, long, v lat, v long ]