1. Matthew Greenstein | METEO 485 | Apr. 26, 2004 Using Neural Networks and Lagged Climate Indices to Predict Monthly Temperature and Precipitation Anomalies

2. Overview To correlate monthly temperature and precipitation anomalies with a number of climate indices lagged several months To use neural networks because they simulate non-linear interactions between variables (as opposed to linear regression)

3. Overview 1.Introduction to neural networks 2.Data collection Temperature and precipitation anomalies Climate indices 3.Methods of attack (“how to”) 4.Results 5.Discussion 6.Future ideas

4. Neural Networks

5. Creates categorical and numerical forecasts Uses categorical and numerical predictors

6. Neural Networks Layered regression equations Predictors are linearly regressed (weighted) to create the hidden layer of intermediate forecasts Hidden layer forecasts used as predictors to produce either another hidden layer (and so on) or a final forecast

7. Neural Networks Layered regression captures non-linear relationships, i.e. mimics whatever equation best fits the data You don’t need to know form of equation ahead of time Each dot is a node Human brain: 10 billion nodes Neural net: 10 – 1000 nodes

8. Neural Networks Training a network 1.Training data (66% of dataset) run through neural net / forecasts generated 2.Error calculated  skill scores 3.Neural net tuned (weights changed) to improve scores 4.Repeat fixed number of times (epochs) or until weights stop changing

9. Neural Networks Training a network Learning rate: how much weights are changed compared to error slope Momentum: use a portion of previous weight change for less “jumpiness”

10. Neural Networks Training a network Decay: eliminates useless weights / interactions

11. Neural Networks WEKA Waikato Environment for Knowledge Analysis (University of Waikato, New Zealand) Weka: flightless bird with an inquisitive nature found only in New Zealand Set values of learning rate, momentum, number of nodes, & epochs to fit data well without overfitting Overfitting = fit too perfectly to training data  performs poorly on new data

12. Data Collection What data is needed? Monthly anomalies 6 regions of the U.S. (NW, SW, NC, SC, NE, SE) Temperature and precipitation U.S. Climate Division data since 1895 available Climate indices Monthly values lagged 2, 3, & 4 months Since 1948 available

13. Anomaly Data Divide country into 6 pieces (NW/SW/NC/SC/NE/SE)

14. Anomaly Data Obtained average monthly anomaly data for the U.S. Climate Divisions in each of the 6 regions Dataset from Jeremy Ross Averaged using GrADS Monthly, 1950 – present °F, inches

15. Climate Index Data Obtained from CDC’s climate indices page: http://www.cdc.noaa.gov/ClimateIndices/ From 1950 – present SOI, PNA, NAO, EPO, MEI, Nino3, Nino1+2, Nino3.4, Nino4, AO, NOI, WP, NP, QBO

16. Climate Index Data Some years & months missing! No SOI until 1951 No AO until 1958 No PNA for June & July No EPO for Aug & Sept WEKA throws out cases with missing data  No forecasts were made for Aug – Jan !! Need to re-run without PNA and EPO to get a neural net that can be used during any month

17. Data Processing Excel file Row for each month (Jan 1950 – Dec 2000) Columns of month; each anomaly; and each index lagged 2, 3, and 4 months

18. Data Processing Conversion to ARFF / Conversion to ARFF / Attribute-Relation File Format Save as a CSV Fix blanks:,, replaced by,?, Change file extension:.csv .arff

19. Method I Following procedure followed for each anomaly (NE T, NE P, SE T, SE P, SW T, SW P, NW T, NW P) Build neural nets Vary learning rate (L), momentum (M), layers, epochs Decay Indices and month  predict anomaly Takes a long time to try many possibilities

20. Method I Skill scores Calculated with remaining 34% of dataset Many scores provided 2 used 1.Correlation coefficient (r) 2.Root relative squared error Relative to error if prediction = average of actual values Outliers are penalized strongly

21. Results I NE Temperature Linear regression: r = 0.1067, RRSE = 102.25% Neural nets: Neural Net Layers 8,4,29,6 Momentum0.3 Learning Rate 0.3 Epochs500 R-0.0205-0.1190 RRSE100.25%100.30%

22. Results I SE Temperature Linear regression: r = 0.0352, RRSE = 104.78% Neural nets: Neural Net Layers 4,215,7 Momentum0.3 Learning Rate 0.2 Epochs500 R0.0372-0.0389 RRSE100.13%100.74%

23. Results I SW Temperature Linear regression: r = 0.036, RRSE = 103.40% Neural nets: Neural Net Layers 9,3 Momentum0.20.4 Learning Rate 0.2 Epochs500 R0.0630.025 RRSE99.99%99.97%

24. Results I NW Temperature Linear regression: r = 0.011, RRSE = 103.88% Neural nets: Neural Net Layers Auto9,5 Momentum0.050.8 Learning Rate 0.2 Epochs500 R0.1760.224 RRSE98.66%99.08%

25. Results I NE Precipitation Linear regression: r = 0.073, RRSE = 101.044% Neural nets: Neural Net Layers 5,3 Momentum0.50.2 Learning Rate 0.20.1 Epochs5002000 R0.0610.115 RRSE99.732%99.999%

26. Results I SE Precipitation Linear regression: r = 0.063, RRSE = 104.14% Neural nets: Neural Net Layers 8,3Auto Momentum0.70.5 Learning Rate 0.2 Epochs500 R0.06510.1179 RRSE99.85%101.05%

27. Results I SW Precipitation Linear regression: r = 0.187, RRSE = 98.83% Neural nets: Neural Net Layers 9,5 Momentum0.50.3 Learning Rate 0.2 Epochs500 R0.2800.289 RRSE92.25%96.12%

28. Results I NW Precipitation Linear regression: r = 0.091, RRSE = 101.49% Neural nets: Neural Net Layers 9,5Auto Momentum0.3 Learning Rate 0.50.6 Epochs500 R0.0520.098 RRSE99.91%102.71%

29. Results I Putrid results !! Not worth trying NC/SC… away from oceans RRSE ~ 100%, r ~ 0.10 No big improvement over linear regression SW Precipitation predicted the best (although still bad)… El Nino-related?

30. Method II Predict positive or negative anomaly instead of actual value! Anomalies changed to binary (1, 0) predictands Vary indices used Does that cause significant changes? This became the most interesting part of the study Limited time available: NE T, NE P, SW P

31. Method II Skill scores Many scores provided 3 used 1.Percent Correctly Classified 2.TP (True Positive) Rate 3.TN (True Negative) Rate

32. Results II NE Temperature Neural Net Setup AutoNo MonthOnly Nino: Nino 3.4 No Nino’sNo Nino’s, SOI, MEI More Epochs Momentum0.5 Learning Rate 0.5 Epochs500 1000 Classified Correctly 56.04%54.59%51.69%54.59%53.62%56.52% TP Rate.617.628.606.617.553.628 TN Rate.513.478.442.487.522.513 Auto = WEKA automatically chooses node setup

33. Results II NE Precipitation Neural Net Setup AutoNo MonthOnly Nino: Nino 3.4 No Nino’sNo Nino’s, SOI, MEI Auto Momentum0.3 Learning Rate 0.2 Epochs500 1000 Classified Correctly 54.11%48.79%53.14%49.28%51.21%53.62% TP Rate.664.573.691.645.755.645 TN Rate.402.392.351.320.237.412 ** Changing the epochs results in overfitting!

34. Results II SW Precipitation Neural Net Setup AutoNo MonthOnly Nino: Nino 3.4 No Nino’sNo Nino’s, SOI, MEI Momentum0.1 Learning Rate 0.2 Epochs500 Classified Correctly 60.39% 62.32%61.83%59.42% TP Rate.611.646.628.841 TN Rate.596.606.298 ** Changing the epochs did not change the ‘Only Nino: Nino 3.4” value

35. Discussion NE Temperature 94 +, 113 – Predict negative  correct 54.59% Best neural net: 56.52% correctly classified NE Precipitation 110 +, 97 – Predict positive  correct 53.14% Best neural net: 54.11 % correctly classified SW Precipitation 113 +, 94 – Predict positive  correct 54.59% Best neural net: 62.32% correctly classified

36. Discussion These types of neural nets do not provide significant skill over ‘guessing’ Similar to Method I, not significant difference in skill of logistic regression versus neural nets There is some sensitivity to which variables are included in the neural net… even though the decay factor would attempt to eliminate useless interactions Different sensitivities in each region Using the ‘auto’ setting for layers produced better results

37. Discussion The study was originally supposed to predict the anomaly, but predicting the sign of the anomaly seems to show more promise Time constraints prevented a more in depth look at Method II  possible Meteo 485 project in future semesters Missing June – Sept data could have caused problems with this study

38. Future Work 1.Obtain missing PNA & EPO data 2.Build neural nets for other regions of the country for Method II 3.Use different lag times and combinations of lag times 4.Use different climate indices 5.Omit different indices from current set 6.Try other tools that WEKA offers

39. Special thanks to… Jeremy Ross  For gathering anomaly data Climate Diagnostics Center (CDC)  For climate indices Dr. George Young  Neural net info from Meteo 474 notes

40. Useful Info WEKA website with software downloads: http://www.cs.waikato.ac.nz/ml/weka/ Results data file ARFF file