Training Materials for National Greenhouse Gas Inventories Consultative Group of Experts (CGE) CGE Training Materials National Greenhouse Gas Inventories Uncertainty Analysis Version 2, April 2012
Target Audience and Objective of the Training Materials These training materials are suitable for people with beginner to intermediate level knowledge of national greenhouse gas (GHG) inventory development. After having read this presentation, in combination with the related documentation, the reader should: Have an overview of how to conduct uncertainty assessment Have a general understanding of the methods and tools available, as well as of the main challenges of GHG inventory development in that particular area Be able to determine which methods suit their country’s situation best Know where to find more detailed information on the topic discussed. These training materials have been developed primarily on the basis of methodologies developed, by the IPCC; hence the reader is always encouraged to refer to the original documents to obtain further detailed information on a particular issue. 2 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Acronyms and Abbreviations FAOFood and Agriculture Organization of the United Nations GHGGreenhouse Gas IEAInternational Energy Agency OECDOrganisation for Economic Co-operation and Development QA/QCQuality Assurance/Quality Control 3 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Structure of this presentation Overview of guidance Some thoughts on this guidance in practice Overview of verification 4 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Thoughts on Uncertainty Analysis Why? Why is uncertainty analysis needed? Is it important? We need clear justification. What? What is involved. What do the results mean? We should show that uncertainty analysis is practical for all. When? Uncertainty analysis should be an integral part of inventory compilation – not an “add on” at the end! How? The method chosen should be matched to resources and expertise, while giving useful information. 5 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Benefits of Uncertainty Analysis 6 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Uncertainty Estimation - Overview Gather Information Collect uncertainty information on activity data and emission factors Decide approach to use Error Propagation Monte Carlo Perform Inventory Analysis Spreadsheet Software tool 7 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Specifying Uncertainty Uncertainty is quoted as the 2.5 and 97.5 percentile i.e. bounds around a 95% confidence interval. This can be expressed as: 234 ± 23% (- 50%, + 100%) 2000 (a factor of 2) (i.e. - 50%, + 100%) An order of magnitude (i.e. 1 to 100). 8 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Probability Density 9 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Probability Density 10 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Inventory Cycle 11 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories Report inventory Start New Estimate Building on experience of previous inventory (if any) Setup and implement quality control Identify key categories Check inventory through QA Select methods and data considering uncertainty and time-series consistency Conduct key category analysis Uncertainty: Evaluate input data and assess overall inventory Collect data and estimate emissions and removals Compile inventory Make necessary revisions (if any)
Sources of Uncertainty Assumptions and methods These method may not accurately reflect the emission. Good practice requires that biases be reduced as much as possible. Guidelines aim to be as unbiased and complete as possible. Input data Measured values have errors and emission factors may not be truly representative. Calculation errors Implement good QA/QC to stop these. 12 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Uncertainties Arise in Input Data Because of… Lack of data Use of proxies, extrapolation etc. Missing data. Data not truly representative Statistical random sampling error Measurement error Misreporting. Consideration of these during data collection phase will minimise errors. 13 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Uncertainty Information 14 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Sources of Data National statistical agencies Sectoral experts, stakeholder organizations Other national experts IPCC Emission Factor Database Other international experts International organizations publishing statistics e.g., United Nations, FAO, IEA, OECD and the IMF (which maintains international activity as well as economic data) Reference libraries (national libraries) Scientific and technical articles in environmental books, journals and reports. Universities Web search for organizations and specialists National Inventory Reports from Parties to the United Nations Framework Convention on Climate Change Ministries, local and traditional authorities. 15 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Expert Judgement Expert judgement on methodological choice and choice of input data to use is ultimately the basis of all inventory development. Experts with suitable backgrounds can be found in government, industrial trade associations, technical institutes, industry and universities. The goal of expert judgement may be choosing the: Proper methodology Parameter value and uncertainty from ranges provided Most appropriate activity data to use Most appropriate way to apply a methodology Or determining the appropriate mix of technologies in use. Expert judgement is always required since one must judge whether the data are a representative random sample and, if so, what methods to use to analyse the data. This requires both technical and statistical judgement. 16 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Working With Experts Wherever possible, expert judgement should be elicited using an appropriate protocol (e.g. Stanford/SRI protocol): Motivating: Establish a rapport with the expert, describe the context, explain the most commonly occurring biases. Structuring: Clearly define the quantities for which judgements are to be sought (e.g. resulting emissions or removals should be for typical conditions averaged over a one- year period). Conditioning: Work with the expert to identify and record all relevant data, models and theory relating to the formulation of the judgements. Encoding: Request and quantify the expert’s judgement (may differ but should include uncertainty information). Verification: Analyse and feedback conclusions regarding their judgement. Is what has been encoded really what the expert meant? Document! 17 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Remember… Producing high quality “good practice” emission and removal estimates is paramount. Efforts on uncertainty analysis should be small in comparison with the efforts on the inventory estimates themselves. Data collection activities should consider data uncertainties: This will ensure the best data is collected and ensures good practice estimates As you collect data you should assess how “good” it is. At its simplest, a well-planned uncertainty assessment should only take a few extra hours! 18 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Methods to Combine Uncertainties Error propagation Simple - standard spreadsheet can be used Guidelines give explanation and equations Difficult to deal with correlations Strictly (standard deviation/mean) < 0.3 A simple solution is provided. Monte-Carlo Simulation More complex - Use specialized software Needs shape of probability density function (pdf) Suitable where uncertainties are large, non-Gaussian, complex algorithms, correlations exist and uncertainties vary with time. 19 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Enter emissions data Enter uncertainties Data calculated using simple equations 20 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories Error Propagation - Example
Note short list of source/sinks Emission factor uncertainties based on defaults in guidelines Activity data uncertainties based on source of data
Error Propagation: example of results 22 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Error Propagation: example of results (continued) 23 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: example Key requirements Not just uncertainties but also probability density function (pdf) Mean Width Shape (e.g. normal, log-normal, Weibul, gamma, uniform, triangular, fractile Principle Select random values of input parameters from their pdf and calculate the corresponding emission. Repeat many times and the distribution of the results is the pdf of the result, from which mean and uncertainty can be estimated. 24 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: example (continued) 25 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 1 run 26 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 10 runs 27 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 20 runs 28 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 50 runs 29 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 100 runs 30 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 500 runs 31 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 1000 runs 32 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 5000 runs 33 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: 10,000 runs 34 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Monte-Carlo Method – Simplified Approach: summary results 35 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Summary Even simple uncertainty estimates give useful information - If they are performed well! Assessment of uncertainty in the input parameters should be part of the standard data collection QA/QC: Careful consideration will improve estimates as well as providing input data for uncertainty analysis. If resources are limited: amounts spent on uncertainty analysis should be small compared with total effort. For simple estimates: Uncertainty in activity data assessed as data collected Uncertainty in emission factors from guidelines if readily available Aggregate categories/gases to independent groups of sources/sinks Use Approach 1 – the spreadsheet requires little statistical knowledge. 36 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories
Thank you Diagrams © IPCC Except where noted otherwise 37 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories