1. VIPER Optimization What is optimization? How does viper’s Station and Time Period optimization work? How to interpret results? What to avoid? Tom Pagano tom.pagano@por.usda.gov 503-414-3010

2. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow

3. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Data types (3) Snowpack Precipitation Temperature Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Months (4) October November December January What is the optimal combination of variables to get the best prediction?

4. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Data types (3) Snowpack Precipitation Temperature Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Months (4) October November December January What is the optimal combination of variables to get the best prediction?

5. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Data types (3) Snowpack Precipitation Temperature Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Months (4) October November December January What is the optimal combination of variables to get the best prediction?

6. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Data types (3) Snowpack Precipitation Temperature Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Months (4) October November December January What is the optimal combination of variables to get the best prediction? Too many combination to evaluate using “brute force” (at least 24,000)

7. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Data types (3) Snowpack Precipitation Temperature Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Months (4) October November December January What is the optimal combination of variables to get the best prediction? Too many combination to evaluate using “brute force” (at least 24,000) Instead we could “search”, trying some combinations and following up promising avenues.

8. Station optimization search algorithm, simple example Target Tomichi at Gunnison Apr-July streamflow Predictors A Avalanche Oct-Mar Precip B Butte Apr 1 SWE C Cochetopa Pass Apr 1 SWE D Datother Station Apr 1 SWE

9. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D

10. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30

11. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30 Sort equations, best to worst 1. D – 30 best 2. C – 33 3. B – 36 4. A – 38 worst

12. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30 Sort equations, best to worst 1. D – 30 best 2. C – 33 3. B – 36 4. A – 38 worst Build 2 variable equations, regress, find jackknife standard error 1. D – 30 2. C – 33 3. B – 36 4. A – 38 A,D - 32 B,D - 34 C,D - 29 A,C - 37 B,C - 35 A,B - 39 Note: A,B is the same as B,A

13. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34

14. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34 Build all 3-variable combinations, growing from top 5 list, evaluate 1. C,D - 29 A,C,D - 28 B,C,D - 36 2. D - 30 3. A,D - 32 A,B,D - 35 4. C - 33 5. B,D - 34 Note: A,B,C not evaluated… No “trunk” leads us there

15. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34 Build all 3-variable combinations, growing from top 5 list, evaluate 1. C,D - 29 A,C,D - 28 B,C,D - 36 2. D - 30 3. A,D - 32 A,B,D - 35 4. C - 33 5. B,D - 34 Re-sort list, evaluate 4-variable combinations 1. A,C,D – 28 A,B,C,D - 31 2. C,D - 29 3. D - 30 4. A,D - 32 5. C - 33 Note: A,B,C not evaluated No “trunk” leads us there

16. Station optimization search algorithm, simple example Re-sort list… Optimization finished when top 5 list doesn’t change 1. A,C,D - 28 2. C,D - 29 3. D - 30 4. A,B,C,D - 31 5. A,D - 32

17. Station optimization search algorithm, simple example Re-sort list… Optimization finished when top 5 list doesn’t change 1. A,C,D - 28 2. C,D - 29 3. D - 30 4. A,B,C,D - 31 5. A,D - 32 Questions to ask: Which variables are “popular”? What would maintain consistency from month to month? Does the combination make physical sense? Is some data harder to get than others? Optimization is not a substitute for thinking!

18. Viper station optimization

19. Output

20. Input data: Year Y1 X9 1979 99.40 24.80 1980 93.10 25.80 1981 17.40 13.80 1982 63.30 23.80 1983 129.00 25.10 1984 269.60 30.00 1985 135.70 24.60 1986 92.80 21.00 1987 127.50 22.60

21. DETAILED OUTPUT RANKED REGRESSION EQUATIONS: ***** RANK 1 coef var ---- ---- 0.20602 X9 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek -0.28128 Constant #obs= 22 #pc = 1 Jr = 0.873 JStdErr = 0.448 JStdErrSS = 0.500 r = 0.896 StdErr = 0.407 StdErrSS = 0.546 JCK REG JCK REG STD REG STD REG YEAR OBSERVED COMPUTED ERROR COMPUTED ERROR 1979 4.63 4.84 0.21 4.83 0.20 1980 4.53 5.08 0.55 5.03 0.50 1981 2.59 2.55 -0.04 2.56 -0.03 1982 3.99 4.66 0.67 4.62 0.64 1983 5.05 4.88 -0.18 4.89 -0.16

22. SUMMARY OUTPUT Name of this file: 09119000_optimization.txt Number of combinations evaluated = 1 Created on 6/13/2007 3:36:56 PM by tpagano Transformation type: Cubert Analysis type: Principal Components VARIABLES: Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X9 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 ERROR COEF. USED PC'S X 0.448 0.873 22 1

23. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tables

24. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tables Recognize popular variables

25. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tables Recognize popular variables Be aware of num. obs

26. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tables Recognize popular variables Be aware of num. obs Meaningful interpretations

27. Viper time period optimization Maybe you know “slumgullion precip” should be a predictor, but don’t know if it should be oct-mar or nov-mar or ? Time optimization varies stations with in groups one at a time, looking for the optimal combination.

28. Viper time period optimization Maybe you know “slumgullion precip” should be a predictor, but don’t know if it should be oct-mar or nov-mar or ? Time optimization varies stations with in groups one at a time, looking for the optimal combination. It does not have a fancy search algorithm, it tries all combinations. Time optimization available for Z-score or PCA… Station optimization only available in PCA for now.

29. Interpretation 1.Adjust only SnotelPRCP and NRCSStrm. Leave others alone.

30. Interpretation 1.Adjust only SnotelPRCP and NRCSStrm. Leave others alone. 2.Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar)

31. Interpretation 1.Adjust only SnotelPRCP and NRCSStrm. Leave others alone. 2.Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar) 3. Evaluate all combinations for NRCSStrm between Jul and Mar (e.g. Aug-Feb is a valid combo Jun-Mar is not)

32. Interpretation 1.Adjust only SnotelPRCP and NRCSStrm. Leave others alone. 2.Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar) 3. Evaluate all combinations for NRCSStrm between Jul and Mar (e.g. Aug-Feb is a valid combo Jun-Mar is not) 4. Return best combination to interface

33. Recent additions Don’t allow the “best” month for a group be one that eliminates that group. e.g. Oct-Jan precipitation + August swe = “best” But there is no august SWE. Really it just wants precip alone. Optimize Groups in Order Optimize Groups Dependently Optimize Groups Independently When optimizing one group do you leave the others on or off? Do you want each group to try to do its best on its own, or let groups cooperate?

34. A word of caution about optimization! A station is not graded on the exams that it skips Streamflow Forecast Observed Station availability #1 #2 High error period

35. A word of caution about optimization! A station is not graded on the exams that it skips Period of record Standard error #1 32.7 #2 45.9 Streamflow Forecast Observed Station availability #1 #2 High error period Which station is really “best”? better

36. A word of caution about optimization! A station is not graded on the exams that it skips Period of record Standard error #1 32.7 #2 45.9 Standard error for overlapping period (1988-2006) #1 32.7 #2 28.8 Streamflow Forecast Observed Station availability #1 #2 High error period Which station is really “best”? better