1. RIVM/MNP Guidance for Uncertainty Assessment and communication
Jeroen van der Sluijs, James Risbey, Penny Kloprogge
(Copernicus Institute, Utrecht)
Jerry Ravetz (RMC, London)
Silvio Funtowicz, Serafin Corral, Ângela Pereira (JRC, Ispra)
Bruna de Marchi
Rob Hoppe, Simone Huijs (Fac. Public Policy, Twente Univ.)
Marjolein van Asselt (ICIS, Maastricht)
Peter Jansen, Arthur Petersen, Anton van der Giessen (RIVM)

2. RIVM / De Kwaadsteniet (1999)
“RIVM over-exact prognoses based on virtual reality of computer models”
Newspaper headlines:
Environmental institute lies and deceits
Fuss in parliament after criticism on environmental numbers
The bankruptcy of the environmental numbers
Society has a right on fair information, RIVM does not provide it

3. RIVM learning process
<1999 Innovative methodological R&D&D on uncertainty assessment and management (e.g TARGETS)
1999 De Kwaadsteniet affair
Fact sheets
National review
2000 International audit
>2000 Multi-disciplinary project
2001–2002 Development of Guidance

4. Insights on uncertainty
Uncertainty is partly socially constructed and its assessment always involves subjective judgement
Omitting uncertainty management can lead to scandals, crisis and misunderstandings
More research does not necessarily reduce uncertainty
High quality  low uncertainty
Shift in focus needed from reducing uncertainty towards a systematic management of uncertainty
Uncertainty is a multi-dimensional concept and can manifest itself at different locations

5. Goals
Structured and transparent approach that facilitates awareness, identification, and incorporation of uncertainty
May not reduce uncertainties, but provides a means to assess their potential consequences and avoid pitfalls associated with ignoring or ignorance of uncertainties
Guidance for use and help against misuse of uncertainty tools
Provide useful uncertainty assessments (robust knowledge)
Facilitate effective communication on uncertainties in terms of robustness of knowledge
Fit RIVM's specific role in the decision analytic cycle

6. Project phases: 25 October 2001 Expert workshop
22 November 2001 User workshop
Expert review (2002)
User review (2002)
Implementation (2003)

7. Guidance components

8. Mini check Problem framing Involvement of stakeholders
Selection of indicators
Appraisal of knowledge base
Mapping key uncertainties
Reporting

9. Main steps Minicheck/Quick scan Problem framing
Communication & dialogue
Process/context assessment (history, stakeholders, values)
(Assess limitations of) Environmental assessment methods
Uncertainty identification and prioritization
Uncertainty analysis
Review, evaluation, interpretation
Reporting

10. Uncertainty typology Level of uncertainty Nature of uncertainty
Statistical uncertainty (range / pdf)
Scenario uncertainty (what-if)
Recognized ignorance
Nature of uncertainty
Knowledge related
Variability related
Qualification of knowledge base (NUSAP/Pedigree)
valueladenness of choices and assumptions

11. Locations of uncertainty
Context
Expert judgment
Model
structure
technical (computer code & algorithms)
parameters
inputs
Data
 Outputs

13. Outputs Quickscan (1) Description of the problem
Gauge of how well assessment tools address the problem
List of which uncertainties are salient on the basis of problem structure
Indication whether to involve stakeholders
Indication where in policy life cycle the problem is
List of stakeholders
Identification of areas of agreement/disagreement on value dimensions

14. Outputs Quickscan (2) Prioritized list of salient uncertainties
Communication plan: when and how to involve what stakeholders
First indication of appropriate tools to address uncertainties identified
Assessment of attainable robustness of results + indication what it might take to increase robustness
Assessment of the relevance of results to the problem
Pitfalls and hints to facilitate effective communication of results

15. Toolbox uncertainty assessment
Sensitivity Analysis (screening, local global)
Error propagation equation (TIER 1)
Monte Carlo (TIER 2)
NUSAP
Expert Elicitation
Scenario analysis
Extended Quality Assurance (pedigree scheme)
Perspective based approaches (cultural theory)
Checklist model quality assistance
(see
....

16. Toolbox For each tool: Main purpose and use
Sorts and locations of uncertainty addressed
Required resources
Strengths and limitations
Guidance on application
Hints on complementarity with other tools
Typical pitfalls
Key references (handbooks, user-guides, web resources, example studies, experts)

17. Review, synthesis and evaluation
Synthesise quantitative & qualitative results
Revisit problem and assessment steps
Frame findings in terms of robustness of the environmental assessment concerned
Relevance of results to the problem
Discuss implications of findings for different settings of burden of proof

18. Reporting Context of communication Who are target audiences Language
Method and style
Content

19. More information: