1. Copernicus Institute Interfaces between Science & Society, Milano, 27-28 November 2003 Quicksandy Knowledge Bases The need for guidance for dealing with uncertainty, assumptions and value commitments in environmental assessment Jeroen van der Sluijs (UU Copernicus Institute) j.p.vandersluijs@chem.uu.nl

2. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Quicksand Quicksand is only dangerous if you misunderstand its nature and treat it as if it is not different from the rest of the beach Survival handbook: It is basically just sand mixed with water, which makes it behave different from sand “The worst thing to do is to thrash around and move your arms and legs through the mixture, because you'll only force yourself further into it” Quicksandy knowledge bases are basically a mixture of knowledge and ignorance and you need to understand its nature and need some guidance to handle it

3. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Complex environmental problems Typical characteristics (Funtowicz & Ravetz): Decisions will need to be made before conclusive scientific evidence is available; Decision stakes are high: potential error costs of wrong decisions can be huge Values are in dispute Knowledge base is mixture of knowledge and ignorance:  large (partly irreducible) uncertainties, knowledge gaps, and imperfect understanding; Assessment dominated by models, scenarios, and assumptions Many (hidden) value loadings in problem frames, indicators, assumptions Uncertainty management is essential

4. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Omitting uncertainty management can lead to crisis and loss of trust Keepin & Wynne, 1984 –IIASA global Energy Scenarios are assumption laden, highly unstable, and based on informal guesswork De Kwaadsteniet, 1999 –RIVM over-exact prognoses based on virtual reality of computer models BSE in UK –if Government’s position on BSE had reflected more accurately scientific uncertainty, both the public and the authorities would have been better prepared for the shift in scientific consensus which prompted the notorious change of policy ( House of Lords, 2000 )

5. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Once environmental numbers are thrown over the disciplinary fence, important caveats tend to be ignored, uncertainties compressed and numbers used at face value e.g. Climate Sensitivity, see Van der Sluijs, Wynne, Shackley, 1998: 1.5-4.5 °C ? ! Crossing the disciplinary boundaries Resulting misconception: Worst case = 4.5°C

6. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net The uncertainty problem Now widely held that uncertainty assessment is essential But in practice… Little appreciation for multi dimensional nature of uncertainty Lack of understanding of characteristics and relative importance State of the Art Methodology not widely disseminated Need for guidance and training

7. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net RIVM Uncertainty Guidance Structure uncertainty management in environmental assessment Promote awareness & critical (self)reflection May not reduce uncertainties, but provides means to assess their potential consequences and avoid pitfalls associated with ignoring or ignorance of uncertainties Assist in documenting uncertainty, assumptions, and value-loadings Guides the use and helps against misuse of uncertainty tools Promote the adoption of uncertainty methods in common practice at RIVM/MNP Facilitate design of effective strategies for communicating uncertainty

8. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Project phases: 25 October 2001 Multidisciplinary Expert workshop 22 November 2001 User workshop Expert review User review Development of a portal: Quick Scan & Mini Checklist Testing phase 2002 Implementation (2002/2003; MV6)

9. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net RIVM/MNP Guidance for Uncertainty Assessment and Communication All five documents can be downloaded from www.nusap.net I. Mini-Checklist III. Tool Catalogue Uncertainty Assessment II. Quickscan Questionnaire III. Detailed Guidance (incl. Glossary) II. Quickscan Hints & Actions List

10. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Sections of the checklist Problem framing Involvement of stakeholders Indicators Appraisal of knowledge base Mapping, assessment and prioritization of relevant uncertainties Reporting of uncertainty information

11. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Outputs Quickscan Rival problem framings, awareness of what is left out List of salient uncertainties based on problem structure Scoring of maturity of problem in the policy process & role of assessment in this process List of relevant stakeholders and indication of whether, when and how to involve them –Problem co-framer, knowledge co-producer, extended peer review Identification of areas of agreement and disagreement among stakeholders on value dimensions of the problem Gauge of how well assessment tools and indicators address the problem Prioritized list of salient uncertainties (using typology) Selection of tools for addressing each uncertainty type Assessment of (likely) robustness of results

12. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net

13. Tool catalogue Quantitative tools (Sensitivity Analysis, Monte Carlo) NUSAP Expert Elicitation Scenario analysis PRIMA Extended Quality Assurance (pedigree scheme) Checklist model quality assistance Assumption analysis Actor analysis …...

14. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Tool catalogue For each tool: What sorts and locations of uncertainty does this tool address? What resources are required to use it? Strengths and limitations guidance on application & complementarity Typical pitfalls of each tool References to handbooks, example case studies, web-sites, experts etc.

15. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Triggers that increase policy relevance of uncertainty Close to a norm or target Near a threshold of severe impact Possibility of serious irreversible effects (precautionary principle) On steep part of cost-curve or impact curve Low pedigree High (potential) value-ladenness ‘temperatute’ of scientific or societal controversies

16. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Reporting Make uncertainties, key assumptions and value loadings explicit Assess robustness of results Discuss implications of uncertainty for different settings of burden of proof Relevance of results to the problem Progressive disclosure of information -> traceability and backing

17. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Conclusions The guidance: Structures the tasks of uncertainty management Can be used flexibly –Quick&dirty, Quick-scan, full-mode –Before/during/after Promotes reflection and forces deliberate choice on how uncertainties are handled Helps to avoid pitfalls Its development and introduction at RIVM constitutes an institutional innovation

18. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Conclusions (2)  Similar to a patient information leaflet alerting the patient to risks and unsuitable uses of a medicine, the guidance enables the delivery of policy-relevant quantitative information together with the essential warnings on its nature, limitations and pitfalls. It thereby promotes responsible and effective use of knowledge and ignorance in policy processes.

19. Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net Text contributions by: Jeroen van der Sluijs James Risbey Arthur Petersen Peter Jansen Bruna de Marchi Serafin Corral Silvio Funtowicz Rob Hoppe Simone Huijs Penny Kloprogge Angela Pereira Jerry Ravetz Willemijn Tuinstra Marjolein van Asselt Further Workshop participants and or reviewers Poul Harremoes Peter Heuberger Andrea Saltelli Stefano Tarantola Wim Turkenburg John van Aardenne Anton van der Giessen Mark van Oorschot, Ad Seebregts Hans Visser Warren Walker >40 RIVM employees ……. Acknowledgements