1. Exploratory Modeling and Analysis Dr.ir Jan Kwakkel

2. knowledge about likelihoods knowledge about outcomes problematicunproblematic problematic unproblematic RISKAMBIGUITY UNCERTAINTYIGNORANCE Sterling & Scoones (2009) http://www. ecologyandsociety.org/vol14/iss2/art14/

3. knowledge about likelihoods knowledge about outcomes problematicunproblematic problematic unproblematic RISKAMBIGUITY UNCERTAINTYIGNORANCE Deep uncertainty Lempert et al (2003) Kwakkel et al (2010) 10.1504/IJTPM.2010.036918 Wicked problems Rittel & Weber (1973) Churchman (1967) Societal messes Ackhoff (1974)

4. Exploratory Modeling Bankes (1993) Bankes et al (2013) Robust Decision Making Lempert & Collins (2007) 10.1111/j.1539-6924.2007.00940.x Hamarat et al (2013) 10.1016/j.techfore.2012.10.004 Scenario Discovery Bryant and Lempert (2010) 10.1016/j.techfore.2009.08.002 Kwakkel et al (2013) 10.1016/j.techfore.2012.09.012 knowledge about likelihoods knowledge about outcomes problematicunproblematic problematic unproblematic RISKAMBIGUITY UNCERTAINTYIGNORANCE

5. Taxonomy of robustness frameworks Herman et al (2015) How should robustness be defined for water systems planning under change? Journal of Water Resources Planning & Management

6. What is robustness?

8. How do you develop scenarios?

9. Top down versus bottom up scenario methods Scenario Logic Identify uncertain factors Cluster factors into megatrends Assess that megatrends are independent Score megatrends on degree of uncertainty and impact High impact high uncertain megatrends form the scenario logic Each quadrant forms a scenario Scenario discovery Identify uncertain factors Design experiments covering the uncertainty space Conduct experiment using a computational model Use statistical machine learning techniques to identify hypercubes in the uncertainty space Communicate hypercubes as scenarios

10. Exploratory modeling approach Agnostic about modeling paradigm SD, DEVS, ABM, spatially explicit hydrological modes, climate models, etc. EMA is not merely a post-processing step after a model has already been build in conceptualization, identify key uncertainties in specification, design model to explore over uncertainties in verification and validation assess comprehensiveness of exploratory character etc. The iterative analysis of results is the most time consuming phase Risk of information overload

11. Fast and Simple models The more detailed the model, the more precise its results, the more reliable its results

12. Fast and Simple models The more detailed the model, the more precise its results, the more reliable its results

13. Fast and Simple models The more detailed the model, the more precise its results, the more reliable its results Consequence Ever increasing size of models Ever increasing runtimes (despite technological development) Ever more difficult to assess whether the model produces the right outputs for the right reason

14. Fast and Simple models The more detailed the model, the more precise its results, the more reliable its results Consequence Ever increasing size of models Ever increasing runtimes (despite technological development) Ever more difficult to assess whether the model produces the right outputs for the right reason It is better to be roughly right, then precisely wrong

15. Open exploration Design of experiments Factorial methods Monte Carlo sampling Latin Hypercube sampling Used for Identification of bandwidth outcomes Identification of types of behavior Subsequent analysis Factor identification Factor prioritization

16. Exploration

17. Choosing a distribution

19. Analysis of Results Factor identification Patient Rule Induction Method CART Dimensional stacking Factor Prioritization Sobol, Moris Feature selection techniques

20. Directed search Optimization as search technique Mulit-objective optimization Robust optimization Used for Worst case discovery Identification of boundaries where behavior switches Policy design Subsequent analyses Trade-offs Tipping points

21. Support for exploratory modeling Python Exploratory modeling workbench https://github.com/quaquel/EMAworkbench http://emaworkbench.readthedocs.org/en/latest/ Dependencies: Python distribution (anaconda 2.7) deap (via pip) jpype (depends on OS) R Scenario discovery toolkit Open MORDM Hadka et al (2015) An open source framework for many-objective robust decision making, Environmental Modelling & Software

22. EXAMPLES

23. ATES smart grids

24. Can reduce energy use/GHG emissions by 50%+ for large buildings 3,000 systems in function, 20,000 expected by 2025 – largest user of groundwater by 2020 How to manage this technology at a larger scale? Bonte, 2014 Aquifer Thermal Energy Storage (ATES) in the Netherlands

25. Hybrid simulation architecture EMA Workbench suite for exploratory modelling and scenario discovery: http://www.simulation.tbm.tudelft.nl

26. Sample output

28. Scenario Discovery MSc. Thesis of Sebas Greeven

29. RDM for Waas Case Haasnoot et al (2012) https://dx.doi.org/10.1007/s10584-012-0444-2https://dx.doi.org/10.1007/s10584-012-0444-2 Full analysis: http://tinyurl.com/qyt7ozhhttp://tinyurl.com/qyt7ozh

30. Waas Case Policy options: Strengthening and heightening of dikes Room for the river Multi level safety River basin management Uncertainties River runoff Land use Relationship between water levels and failure of dikes Relationship between flood levels and economic damages Efficacy of actions Outcomes: Casualties Economic damage Costs

31. First iteration 5000 experiments Strong correlation between casualties and damages

32. First iteration 5000 experiments Strong correlation between casualties and damages Scenario discovery results 972 cases of interest Combination of W+ and any urbanization scenario Clear need for action

33. Second iteration 5 policy options, out of which dike heightening appears to be the most promising Less correlation between damages and casualties

34. Second iteration 5 policy options, out of which dike heightening appears to be the most promising Less correlation between damages and casualties Scenario discovery Casualties due strong urbanization Damages more difficult to explain Need to expand or complement policy

35. Third iteration 3 potential solutions with different trade offs Dike heightening + evacuation Dike heightening + climate dikes Substantial dike heightening Scenario discovery inconclusive

36. Resulting policy Three possible options Dike 1:500 + 0.5 + early evacuation cheap, reasonable effective mainly for reducing casualties Dike 1:500 + 0.5 + climate dikes very expensive, quite effective for reducing both casualties and damages Dike 1:1000 modestly cheap, reasonable effective for reducing both casualties and damages Preferences on outcomes will determine the final decision

37. Dynamic Adaptive Policy Pathways Haasnoot et al (2013) https://dx.doi.org/10.1016/j.gloenvcha.2012.12.006

38. Adaptation pathways Design for adaptation over time Keep multiple options open for the future Stress testing of policies and plans before implementation Let policies fail ‘in silico’

39. Waas Case Policy options: Strengthening and heightening of dikes Room for the river Multi level safety River basin management Uncertainties River runoff Land use Relationship between water levels and failure of dikes Relationship between flood levels and economic damages Efficacy of actions Outcomes: Casualties Economic damage Costs Haasnoot et al (2012) https://dx.doi.org/10.1007/s10584-012-0444-2https://dx.doi.org/10.1007/s10584-012-0444-2

40. Designing adaptation pathways Curse of dimensionality Multiple outcomes  multi objective Many scenarios  robust Many possible sequences of actions  optimization Solver: BORG Many different understandings of robustness Kwakkel, J.H., et al. (2014) https://dx.doi.org/10.1007/s10584-014-1210-4https://dx.doi.org/10.1007/s10584-014-1210-4

41. DAPP pathway map

42. Understanding the trade offs