RIVM/MNP Guidance for Uncertainty Assessment and communication Jeroen van der Sluijs, James Risbey, Penny Kloprogge (Copernicus Institute, Utrecht) Jerry Ravetz (RMC, London) Silvio Funtowicz, Serafin Corral, Ângela Pereira (JRC, Ispra) Bruna de Marchi Rob Hoppe, Simone Huijs (Fac. Public Policy, Twente Univ.) Marjolein van Asselt (ICIS, Maastricht) Peter Jansen, Arthur Petersen, Anton van der Giessen (RIVM)

RIVM / De Kwaadsteniet (1999) “RIVM over-exact prognoses based on virtual reality of computer models” Newspaper headlines: Environmental institute lies and deceits Fuss in parliament after criticism on environmental numbers The bankruptcy of the environmental numbers Society has a right on fair information, RIVM does not provide it

RIVM learning process <1999 Innovative methodological R&D&D on uncertainty assessment and management (e.g TARGETS) 1999 De Kwaadsteniet affair Fact sheets 1999-2000 National review 2000 International audit >2000 Multi-disciplinary project 2001–2002 Development of Guidance

Insights on uncertainty Uncertainty is partly socially constructed and its assessment always involves subjective judgement Omitting uncertainty management can lead to scandals, crisis and misunderstandings More research does not necessarily reduce uncertainty High quality  low uncertainty Shift in focus needed from reducing uncertainty towards a systematic management of uncertainty Uncertainty is a multi-dimensional concept and can manifest itself at different locations

Goals Structured and transparent approach that facilitates awareness, identification, and incorporation of uncertainty May not reduce uncertainties, but provides a means to assess their potential consequences and avoid pitfalls associated with ignoring or ignorance of uncertainties Guidance for use and help against misuse of uncertainty tools Provide useful uncertainty assessments (robust knowledge) Facilitate effective communication on uncertainties in terms of robustness of knowledge Fit RIVM's specific role in the decision analytic cycle

Project phases: 25 October 2001 Expert workshop 22 November 2001 User workshop Expert review (2002) User review (2002) Implementation (2003)

Guidance components

Mini check Problem framing Involvement of stakeholders Selection of indicators Appraisal of knowledge base Mapping key uncertainties Reporting

Main steps Minicheck/Quick scan Problem framing Communication & dialogue Process/context assessment (history, stakeholders, values) (Assess limitations of) Environmental assessment methods Uncertainty identification and prioritization Uncertainty analysis Review, evaluation, interpretation Reporting

Uncertainty typology Level of uncertainty Nature of uncertainty Statistical uncertainty (range / pdf) Scenario uncertainty (what-if) Recognized ignorance Nature of uncertainty Knowledge related Variability related Qualification of knowledge base (NUSAP/Pedigree) valueladenness of choices and assumptions

Locations of uncertainty Context Expert judgment Model structure technical (computer code & algorithms) parameters inputs Data  Outputs

Outputs Quickscan (1) Description of the problem Gauge of how well assessment tools address the problem List of which uncertainties are salient on the basis of problem structure Indication whether to involve stakeholders Indication where in policy life cycle the problem is List of stakeholders Identification of areas of agreement/disagreement on value dimensions

Outputs Quickscan (2) Prioritized list of salient uncertainties Communication plan: when and how to involve what stakeholders First indication of appropriate tools to address uncertainties identified Assessment of attainable robustness of results + indication what it might take to increase robustness Assessment of the relevance of results to the problem Pitfalls and hints to facilitate effective communication of results

Toolbox uncertainty assessment Sensitivity Analysis (screening, local global) Error propagation equation (TIER 1) Monte Carlo (TIER 2) NUSAP Expert Elicitation Scenario analysis Extended Quality Assurance (pedigree scheme) Perspective based approaches (cultural theory) Checklist model quality assistance (see www.nusap.net) ....

Toolbox For each tool: Main purpose and use Sorts and locations of uncertainty addressed Required resources Strengths and limitations Guidance on application Hints on complementarity with other tools Typical pitfalls Key references (handbooks, user-guides, web resources, example studies, experts)

Review, synthesis and evaluation Synthesise quantitative & qualitative results Revisit problem and assessment steps Frame findings in terms of robustness of the environmental assessment concerned Relevance of results to the problem Discuss implications of findings for different settings of burden of proof

Reporting Context of communication Who are target audiences Language Method and style Content

More information: www.nusap.net