1. Sensitivity Analysis in GEM-SA

2. GEM-SA course - session 62 Example ForestETP vegetation model 7 input parameters 120 model runs Objective: conduct a variance-based sensitivity analysis to identify which uncertain inputs are driving the output uncertainty.

3. GEM-SA course - session 63 Exploratory scatter plots

4. GEM-SA course - session 64 Sensitivity analysis walkthrough 1.  Project  New 2. In the Files tab, click on Browse on the Inputs File row GEM-SA Demo Data / Model1 / emulator7x120inputs.txt 3. Click on Browse on the Outputs File row GEM-SA Demo Data / Model1 / out11.txt 4. Select the Options tab

5. GEM-SA course - session 65 Sensitivity analysis walkthrough 5. Change the Number of Inputs to 7. 6. Tick the calculate main effects and sum effects boxes only 7. Leave the other options unchanged Input uncertainty options: All unknown, uniform Prior mean options: Linear term for each input Generate predictions as: function realisations (correlated points)

6. GEM-SA course - session 66 Sensitivity analysis walkthrough

7. GEM-SA course - session 67 Sensitivity analysis walkthrough 8. Click OK 9. An Inputs Parameter Ranges window will appear. Click Defaults from input ranges, then OK 10.  Project  Run or use

8. GEM-SA course - session 68 Main effect plots

9. GEM-SA course - session 69 Main effect plots Fixing X 6 = 18, this point shows the expected value of the output (obtained by averaging over all other inputs). Simply fixing all the other inputs at their central values and comparing X 6 =10 with X 6 =40 would underestimate the influence of this input (The thickness of the band shows emulator uncertainty)

10. GEM-SA course - session 610 Variance of main effects Main effects for each input. Input 6 has the greatest individual contribution to the variance Main effects sum to 66% of the total variance

11. GEM-SA course - session 611 Interactions and total effects Main effects explain 2/3 of the variance Model must contain interactions Any input can have small main effect, but large interaction effect, so overall still an ‘important’ input Can ask GEM-SA to compute all pair-wise interaction effects 435 in total for a 30 input model – can take some time! Useful to know what to look for

12. GEM-SA course - session 612 Interactions and total effects For each input X i Total effect = main effect for X i + all interactions involving X i Main effects and total effects normalised by variance Total effect >> main effect implies interactions in the model Look for inputs with large total effects relative to main effects Investigate possible interactions involving those inputs

13. GEM-SA course - session 613 Interactions and total effects Total effects for inputs 4 and 7 much larger than its main effect. Implies presence of interactions

14. GEM-SA course - session 614 Interaction effects 11.  Project  Edit or 12. In Options tab, tick calculate joint effects 13. De-select all inputs under inputs to include in joint effects, select X4, X5, X6, X7

15. GEM-SA course - session 615 Interaction effects 14. Click OK 15.  Project  Run or

16. GEM-SA course - session 616 Interaction effects Note interactions involving inputs 4 and 7 Main effects and selected interactions now sum to almost 92% of the total variance

17. GEM-SA course - session 617 Exercise 1. Set up a new project using SAex1_inputs.txt for the inputs and SAex1_outputs.txt for the output 8 input parameters (uniform on [0,1]) 100 model runs 2. Estimate the main effects only for this model and identify the influential input variables 3. By comparing main effects with total effects, can you spot any interactions? 4. Estimate any suspected interactions to test your intuition!