Uncertainty assessment of the IMAGE/TIMER B1 CO 2 emissions scenario using the NUSAP method Jeroen P. van der Sluijs 1 Jose Potting 1 James Risbey 1 Detlef.

Slides:



Advertisements
Similar presentations
Framework for Operations and Implementation Research in Health
Advertisements

Cities and Green Growth OECD Green Cities Programme
World Water Scenarios 2012 – 2035 William J Cosgrove & Gilberto Gallopin Chicago 17 July 2009.
Climate changes in Southern Africa; downscaling future (IPCC) projections Olivier Crespo Thanks to M. Tadross Climate Systems Analysis Group University.
Innovation systems and technological transitions – towards a common framework Jochen Markard Innovation Research in Utility Sectors (CIRUS) Workshop Zurich,
Enhancing Data Quality of Distributive Trade Statistics Workshop for African countries on the Implementation of International Recommendations for Distributive.
1/18 Long-term Scenarios for Climate Change-Implications for Energy, GHG Emissions and Air Quality Shilpa Rao, International Institute of Applied Systems.
GreenSTEP Statewide Transportation Greenhouse Gas Model Cutting Carbs Conference December 3, 2008 Brian Gregor ODOT Transportation Planning Analysis Unit.
RegIS2: Regional Climate Change Impact & Response Studies RegIS2: Regional Climate Change Impact & Response Studies
Copenhagen 29 June Energy and climate outlook: Renewables in a world and European perspective Peter Russ.
WMO UNEP INTERGOVERNMENTAL PANEL ON CLIMATE CHANGE NATIONAL GREENHOUSE GAS INVENTORIES PROGRAMME WMO UNEP IPCC Good Practice Guidance Simon Eggleston Technical.
6.1 Module 6 Reporting of Mitigation Assessments in National Communications Ms. Emily Ojoo-Massawa CGE Chair.
Australian Perspective: EGSS Developments and Application Issues Peter Williams Assistant Director, Centre of Environment Statistics (P)+61 (0)
Factors affecting contractors’ risk attitudes in construction projects: Case study from China 박병권.
16469 Low Energy Building Design Sustainability – an overview
WORKSHOP ON TECHNOLOGY PATHWAYS FORWARD FOR CARBON CAPTURE & STORAGE ON NATURAL GAS POWER SYSTEMS April 22, 2014 Revis W. James Director, Generation R&D.
“An efficient transformation to a lower carbon economy “
Multi-criteria comparison of fuel policies: Renewable fuel mandate, emission standards, and GHG tax Deepak Rajagopal (UCLA), Gal Hochman (Rutgers), David.
‘Projections in Hindsight’ – An assessment of past emission projections reported by Member States under EU air pollution and greenhouse gas legislation.
Copernicus Institute Knowledge Quality Assessment rethinking uncertainty challenges for science and society Jeroen van der Sluijs Copernicus Institute.
Fuel Poverty. Structure of the Presentation Background: What is fuel poverty? Issues to consider when measuring fuel poverty. Ways to measure fuel poverty.
VTT-STUK assessment method for safety evaluation of safety-critical computer based systems - application in BE-SECBS project.
Opting for “Long Term Operations” Technical, economic and regulatory considerations MARC Conference June 8, 2010 Sean Bushart, EPRI Sr. Program Manager.
Classroom Assessment A Practical Guide for Educators by Craig A
Michalis Adamantiadis Transport Policy Adviser, SSATP SSATP Capacity Development Strategy Annual Meeting, December 2012.
Chapter 6 : Software Metrics
The current position: a policy stocktake and reflection on the role of economic modelling to date Sarah Hendry, Department for Environment, Food & Rural.
Creating a Shared Vision Model. What is a Shared Vision Model? A “Shared Vision” model is a collective view of a water resources system developed by managers.
Crossing Methodological Borders to Develop and Implement an Approach for Determining the Value of Energy Efficiency R&D Programs Presented at the American.
PM2.5 Model Performance Evaluation- Purpose and Goals PM Model Evaluation Workshop February 10, 2004 Chapel Hill, NC Brian Timin EPA/OAQPS.
Measuring Complex Achievement
Overview of Economic Methods to Simulate Land Competition Forestry and Agriculture Greenhouse Gas Modeling Forum National Conservation Training Center.
Low carbon scenarios for the UK Energy White Paper Peter G Taylor Presented at “Energy, greenhouse gas emissions and climate change scenarios” June.
Basics of GHG inventory preparation and Introduction to the IPCC Guidelines and Good Practice Guidelines UNFCCC Workshop on the use of the guidelines.
The Evaluation of Publicly Funded Research Berlin, 26/27 September 2005 Evaluation for a changing research base Paul Hubbard Head of Research Policy, HEFCE,
Ucertainty estimates as part of the inventory process Kristin Rypdal, CICERO.
URBDP 591 I Lecture 3: Research Process Objectives What are the major steps in the research process? What is an operational definition of variables? What.
1.How much adaptation do we need within the period to ~2030 to cope with ‘inevitable’ climate change? (lines up with 2030 RCP) 1.What climate changes (global.
Copernicus Institute Universiteit Utrecht Taking uncertainty on board in decision making The example of adaptation to climate change.
Global Warming - 1 An Assessment The balance of the evidence... PowerPoint 97 PowerPoint 97 To download: Shift LeftClick Please respect copyright on this.
Bangalore, India,17-18 December 2012 METHODOLOGIES TO ASSESS THE ENVIRONMENTAL IMPACTS OF ICT KEVIN J HOUSTON, CEO CARBON MASTERS Member of Independent.
Johnthescone The IPCC Special Report on Renewable Energy Sources and Climate Change Mitigation UN Climate Change Conference June 2011 Bonn, Germany, 7.
© OECD/IEA INTERNATIONAL ENERGY AGENCY Worldwide Trends in Energy Use and Efficiency Key Insights from IEA Indicator Analysis ENERGY INDICATORS.
Role of Integrated Assessment Modelling (IAM) in climate change policy analysis The Global Integrated Assessment Model (GIAM) An ABARE-CSIRO joint initiative.
Chapter 6: Integrating Knowledge and Action Scott Kaminski ME / 9 / 2005.
U.S. Energy Information Administration Independent Statistics & Analysis Thoughts on Energy Projections Used in Climate Change Analysis for.
U.S. National Communication: Projections and Effects of Policies and Measures United States Delegation UNFCCC Workshop on National Communications from.
UNFCCC Workshop on the preparation of national communications from non-Annex I Parties General description of steps taken or envisaged by non-Annex I.
Steven Rose (EPRI) April 9, th Workshop of the Forestry and Agriculture Greenhouse Gas Modeling Forum, Shepherdstown, WV, April 7-9, 2009 Agriculture.
An Expanded Model of Evidence-based Practice in Special Education Randy Keyworth Jack States Ronnie Detrich Wing Institute.
Intergovernmental Panel on Climate Change (IPCC) The IPCC is the leading international body for the assessment of climate change. It was established by.
Warwick Business School The drivers of low carbon business strategies Andrew Sentance, Warwick Business School Warwick University Climate Policy Workshop.
Copernicus Institute Interfaces between Science & Society, Milano, November 2003 Quicksandy Knowledge Bases The need for guidance for dealing with.
UNDP Guidance for National Communication Project Proposals UNFCCC Workshop on the Preparation of National Communications from non-Annex I Parties Manila,
Guidance for Uncertainty Scanning and Assessment at RIVM Jeroen van der Sluijs, James Risbey, Penny Kloprogge (Copernicus Institute, Utrecht) Jerry Ravetz.
Integrated Energy-Environmental Modeling for Regional Scenario Analysis Timothy Johnson U.S. EPA Office of Research and Development Research Triangle Park,
Copernicus Institute Interfaces between Science & Society, Milano, more info: Break-out session Uncertainty, assumptions and value.
Copernicus Institute FP6 MUST workshop, 11 November 2002, Brussels Managing Uncertainty in science for suSTainability future research challenges for Europe.
Improving performance, reducing risk Dr Apostolos Noulis, Lead Assessor, Business Development Mgr Thessaloniki, 02 June 2014 ISO Energy Management.
| Paul Lucas 1 Future energy system challenges for Africa: Insights from Integrated Assessment Models 1 st Africa Sustainable Development Forum.
Thoughts on IT Enterprise Architecture Maturity Models for the
The reflexive management of uncertainty and the use of the NUSAP scheme ____________________________________________________________ M. CRAYE Brussels,
Model Summary Fred Lauer
1. Introduction Over the past decade, the tertiary industry (Service sector) has been the largest and the fastest growing sector in China The tertiary.
RIVM/MNP Guidance for Uncertainty Assessment and communication
Review of the RAINS Integrated Assessment Model
USAEE Conference 2016, October 26, Tulsa, Oklahoma, USA
Measuring Data Quality and Compilation of Metadata
Towards a Water Scarcity & Drought Indicator System (WSDiS)
Presentation transcript:

Uncertainty assessment of the IMAGE/TIMER B1 CO 2 emissions scenario using the NUSAP method Jeroen P. van der Sluijs 1 Jose Potting 1 James Risbey 1 Detlef van Vuuren 2 Bert de Vries 2 Arthur Beusen 2 Peter Heuberger 2 Serafin Corral Quintana 3 Silvio Funtowicz 3 Penny Kloprogge 1 David Nuijten 1 Arthur Petersen 5 Jerry Ravetz 4 1 UU-STS 2 RIVM 3 EC-JRC 4 RMC 5 VUA

Objective Develop a framework for uncertainty assessment and management addressing quantitative & qualitative dimensions test & demonstrate usefulness in IA models

IMAGE 2: Framework of models and Linkages World Economy (WorldScan) Population (Popher) Change in GDP, population & others (i.e. scenario assumptions) Land-use emissions Energy demand & supply (TIMER) Energy & industry emissions Land demand, use & cover Emissions & land-use changes Carbon cycle Atmospheric chemistry Concentration changes Climate (Zonal Climate Model or ECBilt) Climatic changes Natural systems Agricultural Impacts Water Impacts Land degradation Sea level rise FeedbacksFeedbacks Impacts

TIMER Model : five submodels Energy Demand (ED) Liquid Fuel supply (LF) Gaseous Fuel supply (GF) Electric Power Generation (EPG) Solid Fuel supply (SF) Population (POPHER) Inputs:Population, GDP capita -1, activity in energy sectors, assumptions regarding technological development, depletion and others. Outputs:End-use energy consumption, primary energy consumption. Fuel demand Prices Economy (WorldScan) Electricity demand

Main objectives TIMER To analyse the long-term dynamics of the energy system, and in particular changes in energy demand and the transition to non-fossil fuels within an integrated modeling framework; To construct and simulate greenhouse gas emission scenarios that are used in other submodels of IMAGE 2.2 or that are used in meta-models of IMAGE;

Key questions  What are key uncertainties in TIMER?  What is the role of model structure uncertainties in TIMER?  Uncertainty in which input variables and parameters dominate uncertainty in model outcome?  What is the strength of the sensitive parameters (pedigree)?

Location of uncertainty Input data Parameters Technical model structure Conceptual model sruct. /assumptions Indicators Problem framing System boundary Socio-political and institutional context

Sorts of uncertainty Inexactness Unreliability Value loading Ignorance

Inexactness Variability / heterogeneity Lack of knowledge Definitional vagueness Resolution error Aggregation error

Unreliability Limited internal strength in: –Use of proxies –Empirical basis –Theoretical understanding –Methodological rigour (incl. management of anomalies) –Validation Limited external strength in: –Exploration of rival problem framings –Management of dissent –Extended peer acceptance / stakeholder involvement –Transparency –Accessibility Future scope Linguistic imprecision

Value loading Bias –In knowledge production –Motivational bias (interests, incentives) –Disciplinary bias –Cultural bias –Choice of (modelling) approach (e.g. bottom up, top down) –Subjective Judgement –In knowledge utilization –Strategic/selective knowledge use Disagreement –about knowledge –about values

Ignorance System indeterminacy –open endedness –chaotic behavior Active ignorance –Model fixes for reasons understood –limited domains of applicability of functional relations –Surprise A Passive ignorance –Bugs (numerical / software / hardware error) –Model fixes for reasons not understood –Surprise B

Method  Checklist for model quality assistance  Meta-level analysis SRES scenarios to explore model structure uncertainties  Global sensitivity analysis (Morris)  NUSAP expert elicitation workshop to assess pedigree of sensitive model components  Diagnostic diagram to prioritise uncertainties by combination of criticality (Morris) and strength (pedigree)

Checklist Assist in quality control in complex models Not models are good or bad but ‘better’ and ‘worse’ forms of modelling practice Quality relates to fitness for function Help guard against poor practice Flag pittfalls

Checklist structure Screening questions Model & problem domain Internal strength Interface with users Use in policy process Overall assessment

Maria Message Aim Minicam ASF Image (Van Vuuren et al. 2000) (SRES scenarios reported to IPCC (2000) by six different modelling groups) B1-marker

Morris (1991) facilitates global sensitivity analysis in minimum number of model runs covers entire range of possible values for each variable parameters varied one step at a time in such a way that if sensitivity of one parameter is contingent on the values that other parameters may take, Morris captures such dependencies

Most sensitive model components: Population levels and economic activity Intra-sectoral structural change Progress ratios for technological improvements Size and cost supply curves of fossil fuels resources Autonomous and price-induced energy efficiency improvement Initial costs and depletion of renewables

NUS A P: Pedigree Evaluates the strength of the number by looking at: Background history by which the number was produced Underpinning and scientific status of the number

Parameter Pedigree Proxy Empirical basis Theoretical understanding Methodological rigour Validation

Proxy Sometimes it is not possible to obtain direct measurements or estimates of the parameter and so some form of proxy measure is used. Proxy refers to how good or close a measure of the quantity which we model is to the actual quantity we represent. An exact measure of the quantity would score four. If the measured quantity is not clearly related to the desired quantity the score would be zero.

Empirical basis Empirical quality typically refers to the degree to which direct observations are used to estimate the parameter. When the parameter is based upon good quality observational data, the pedigree score will be high. Sometimes directly observed data are not available and the parameter is estimated based on partial measurements or calculated from other quantities. Parameters determined by such indirect methods have a weaker empirical basis and will generally score lower than those based on direct observations.

Theoretical understanding The parameter will have some basis in theoretical understanding of the phenomenon it represents. This criterion refers to the extent en partiality of the theoretical understanding. Parameters based on well established theory will score high on this metric, while parameters whose theoretical basis has the status of crude speculation will score low.

Methodological rigour Some method will be used to collect, check, and revise the data used for making parameter estimates. Methodological quality refers to the norms for methodological rigour in this process applied by peers in the relevant disciplines. Well established and respected methods for measuring and processing the data would score high on this metric, while untested or unreliable methods would tend to score lower.

Validation This metric refers to the degree to which one has been able to cross-check the data against independent sources. When the parameter has been compared with appropriate sets of independent data to assess its reliability it will score high on this metric. In many cases, independent data for the same parameter over the same time period are not available and other datasets must be used for validation. This may require a compromise in the length or overlap of the datasets, or may require use of a related, but different, proxy variable, or perhaps use of data that has been aggregated on different scales. The more indirect or incomplete the validation, the lower it will score on this metric.

Elicitation workshop 18 experts (in 3 parallel groups of 6) discussed parameters, one by one, using information & scoring cards Individual expert judgements, informed by group discussion

Example result gas depletion multiplier Same data represented as kite diagram: Green = min. scores, Amber= max scores, Light green = min. scores if outliers omitted (Traffic light analogy) Radar diagram: Each coloured line represents scores given by one expert

Average scores (0-4) proxy 2½±½ empirical2±½ theory2±½ method 2±½ validation 1±½ valueladeness2½±1 competence 2±½

Conclusions (1)  Model quality assurance checklist proves quick scan to flag major areas of concern and associated pitfalls in the complex mass uncertainties.  Meta-level intercomparison of TIMER with the other SRES models gave us some insight in the potential roles of model structure uncertainties.

Conclusions (2)  Global sensitivity analysis supplemented with expert elicitation constitutes an efficient selection mechanism to further focus the diagnosis of key uncertainties.  Our pedigree elicitation procedure yields a differentiated insight into parameter strength.

Conclusions (3)  The diagnostic diagram puts spread and strength together to provide guidance in prioritisation of key uncertainties.

Conclusions (4) NUSAP method: can be applied to complex models in a meaningful way helps to focus research efforts on the potentially most problematic model components pinpoints specific weaknesses in these components

More information: