1. Identify Parameters Important to Predictions using PPR & Identify Existing Observation Locations Important to Predictions using OPR

2. PPR Statistics for Exercise 8.1c Files are provided for 2 analyses : 1.MODE=PPR, PARGROUPS=NO – If we could obtain data on any one parameter, which should it be? 2.MODE=PPR, PARGROUPS=YES, 2 parameters per group – If we could obtain data on any pair of parameters, which should they be?

3. PPR – Exercise 8.1c Prediction is the advective transport at 100 years travel time. PercentReduc=10 What if we could collect data to reduce by 10 percent the parameter standard deviation? -PPR = percent decrease in the standard deviation of a prediction produced by a 10-percent decrease in the standard deviation of the parameter. Results for the advective-transport predictions at 100 years are shown in next slides: First – individual parameters Second – pairs of parameters x y Figure 8.15b, p. 210

4. Exercise 8.1c: PPR Individual Parameters Which parameters rank as most important to the predictions by the ppr statistic? With CSS and PSS, HK_2 and POR1&2 were ranked first. Why the difference for POR1&2??? Average ppr statistic for all predictions Figure 8.9a, p. 201 1 2 3

5. Changes in meters are small for A100z compared to A100x & A100y. But the vertical dimension is much smaller. PPR correctly represents the different dimensions. PPR Change, in meters Exercise 8.1c: PPR Individual Parameters Figure 8.9b, p. 201Figure 8.9c, p. 201

6. Exercise 8.1c: PPR Grouped Parameters Which parameter pairs would be most beneficial to simultaneously investigate? Figure 8.9d, p. 201 Any pair of: HK_1 RCH_1 VK_CB RCH_2 HK_2 Kind of surprising!

7. How is PPR calculated??? OPR and PPR statistics are based on the calculation of prediction standard deviation, a measure of prediction uncertainty

8. Predictions – Advective Travel Prediction UCODE_2005 can compute the sensitivity of the predicted travel path in three directions: X - East-West Y - North-South Z - Up-Down Using calculations described later, the variance and / or standard deviation of predictions can be determined Advective path

9. Predictions – Uncertainty Standard Deviation Measure of spread of values for a variable Involves assumptions Used in OPR & PPR statistics as a means for comparing relative predictive uncertainty The black curve presents the standard deviation in the context of a normal distribution, which may or not be the appropriate distribution for this uncertainty. Normal distribution Advective path

10. Predictions – Uncertainty Standard Deviation With additional information on parameters or with additional observations – predictive standard deviation is reduced Red bars illustrates ‘new’ predictive standard deviation The change in standard deviation makes the probability distribution more narrow. Use the difference between the red and the black bars to measure the worth of the additional data Normal distribution Advective path Normal distribution

11. Predictions – Uncertainty Standard Deviation With the omission of information about one or more observations – predictive standard deviation is increased Red bars illustrate ‘new’ predictive standard deviation The change in standard deviation makes the probability distribution wider. Use the difference between the red and the black bars to measure the worth of the omitted data Normal distribution Advective path

12. Standard deviation of a prediction standard deviation of the th simulated prediction, z’ calculated error variance from regression vector of prediction sensitivities to parameters matrix of observation sensitivities to parameters matrix of weights on observations and prior transpose the matrix parameter variance-covariance matrix s z’ s 2  z’  b X   V(b) s z’ = [s 2 ( (X T  X) -1 )] 1/2 V(b) = s 2 (X T  X) -1  z’   b   z’  T  b 

13. Standard deviation of a prediction All terms in this equation are already available weight matrix includes weights on observations and on prior information about parameters sensitivity matrix X contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters First order second moment (FOSM) method First order – linearise using first order Taylor’s series Second moment – variances and standard deviations For OPR and PPR statistics, manipulate  and X s z’ = [s 2 ( (X T  X) -1 )] 1/2  z’   b   z’  T  b 

14. Standard deviation of a prediction All terms in this equation are already available weight matrix includes weights on observations and on prior information about parameters sensitivity matrix X contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters First order second moment (FOSM) method First order – linearise using first order Taylor’s series Second moment – variances and standard deviations For OPR and PPR statistics, manipulate  and X s z’ = [s 2 ( (X T  X) -1 )] 1/2  z’   b   z’  T  b 

15. X and  X Observation part Sensitivities Weighting Prior information part

16. s z’ = s 2 ( (X T  X ) -1 ) 1/2 OPR and PPR Statistics - Theory V = s 2 (X T  X ) -1 (  j) Notation (  j) indicates that one of the following has taken place: Information has been added regarding parameter(s) j (+) Information has been added regarding observation(s) j (+) Information has been omitted regarding observation(s) j (-)  z’   b   z’  T  b  (  j)

17. X and  X Observation part Sensitivities Weighting Prior information part For PPR add Prior Information terms For OPR add or remove observation terms

18. Calculate the prediction standard deviation using calibrated model and existing observations Calculate hypothetical prediction standard deviation assuming changes in information about parameters or changes to the available observations The Parameter-Prediction (PPR) Statistic: Evaluate worth of potential new knowledge about parameters, posed in the form of prior information - add to calculations The Observation-Prediction (OPR) Statistic: Evaluate existing observation locations - omit from calculations Evaluate potential new observation locations – add to calculations OPR and PPR Statistics - Approach

19. OPR-PPR Program Encapsulates OPR and PPR statistics: Compatible with the JUPITER API and UCODE_2005 Distributed with MF2K2DX that will convert MODFLOW-2000 and MODFLOW-2005 output files into the Data-Exchange Files needed by OPR-PPR ***ask Matt Tonkin, Tiedeman, Ely, Hill (2007) Documentation for OPR-PPR, USGS Techniques & Methods 6-E2 Exercise uses the OPR and PPR methods together with the synthetic model

20. PPR Statistic Calculation The PPR statistic is defined as the percent change in prediction standard deviation caused by increased knowledge about the parameter Therefore it measures the relative importance to a prediction of potential new information on a parameter s z’ = [s 2 ( (X T  X) -1 )] 1/2 (j)  z’   b   z’  T  b  (j) ppr = [1- (s z / s z )] x 100 (j)

21. PPR Statistic - Theory Focusing on  ppr : Weights on the potential new information are ideally proportional to the uncertainty in that information But, it is not known how certain this information will be This is overcome pragmatically by calculating the weight that that reduces the parameter standard deviation by a user specified percentage.   Y,PRI    ppr Weights on existing observations and prior Weights on potential new information on parameters (j)

22. PPR Statistic - Theory Calculating weights on potential new information: User specifies the desired percent reduction (‘PercentReduc’) in the parameter standard deviation Within OPR-PPR: Add a nominal initial weight into the weight matrix  ppr for the corresponding parameter Iteratively solve the equations above until the standard deviation in that parameter is reduced by the user-specified amount Calculate s z

23. OPR Statistic Calculation The OPR statistic is defined as the percent change in prediction standard deviation caused by: the addition of one or more observations – OPR-ADD the omission of one or more observations – OPR-OMIT [1- (s z / s z )] x 100 (  i) s z’ = [s 2 ( (X T  X ) -1 )] 1/2 (  i)  z’   b   z’  T  b  (  i)

24. OPR Statistic - Theory Weights on existing observations already determined Weights on potential observations must be determined using same guiding principles   Y,PRI Weights on existing observations and prior

25. OPR Statistic - Calculation OBSOMIT STEPS: Set weight(s) for relevant observation(s) to zero Sensitivity matrix X does not need to be modified Calculate s z OBSADD STEPS: Calculate sensitivities for potential observations and append these to X Construct weights for potential observations and append these to  Y,PRI Calculate s z

26. Exercise 8.1d: OPR Statistic Use MODE=OPROMIT, OBSGROUPS=NO to analyze the individual omission of the existing head and flow observations and identify which of these observations are most important to the predictions.

27. Exercise 8.1d – OPR Statistic Results Which observations rank as most important to the predictions? Why? Use: dss – Table 7.5 (p. 148) pss – Figure 8.8 (p. 198) pcc – Information in Table 8.6 (p. 204) Figure 8.10a, p. 203 OPR

28. Exercise 8.1d – OPR Statistic Results Does analysis of the absolute increases in prediction standard deviation produce the same conclusions as did analysis of the opr statistics on the previous slide? Figure 8.10b, p. 203 Change, in meters