1. Training Materials for National Greenhouse Gas Inventories Consultative Group of Experts (CGE) CGE Training Materials National Greenhouse Gas Inventories Uncertainty Analysis Version 2, April 2012

2. Target Audience and Objective of the Training Materials  These training materials are suitable for people with beginner to intermediate level knowledge of national greenhouse gas (GHG) inventory development.  After having read this presentation, in combination with the related documentation, the reader should:  Have an overview of how to conduct uncertainty assessment  Have a general understanding of the methods and tools available, as well as of the main challenges of GHG inventory development in that particular area  Be able to determine which methods suit their country’s situation best  Know where to find more detailed information on the topic discussed.  These training materials have been developed primarily on the basis of methodologies developed, by the IPCC; hence the reader is always encouraged to refer to the original documents to obtain further detailed information on a particular issue. 2 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

3. Acronyms and Abbreviations  FAOFood and Agriculture Organization of the United Nations  GHGGreenhouse Gas  IEAInternational Energy Agency  OECDOrganisation for Economic Co-operation and Development  QA/QCQuality Assurance/Quality Control 3 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

4. Structure of this presentation  Overview of guidance  Some thoughts on this guidance in practice  Overview of verification 4 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

5. Thoughts on Uncertainty Analysis  Why?  Why is uncertainty analysis needed? Is it important?  We need clear justification.  What?  What is involved. What do the results mean?  We should show that uncertainty analysis is practical for all.  When?  Uncertainty analysis should be an integral part of inventory compilation – not an “add on” at the end!  How?  The method chosen should be matched to resources and expertise, while giving useful information. 5 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

6. Benefits of Uncertainty Analysis 6 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

7. Uncertainty Estimation - Overview Gather Information Collect uncertainty information on activity data and emission factors Decide approach to use Error Propagation Monte Carlo Perform Inventory Analysis Spreadsheet Software tool 7 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

8. Specifying Uncertainty  Uncertainty is quoted as the 2.5 and 97.5 percentile i.e. bounds around a 95% confidence interval.  This can be expressed as:  234 ± 23%  26400 (- 50%, + 100%)  2000 (a factor of 2) (i.e. - 50%, + 100%)  An order of magnitude (i.e. 1 to 100). 8 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

9. Probability Density 9 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

10. Probability Density 10 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

11. Inventory Cycle 11 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories Report inventory Start New Estimate Building on experience of previous inventory (if any) Setup and implement quality control Identify key categories Check inventory through QA Select methods and data considering uncertainty and time-series consistency Conduct key category analysis Uncertainty: Evaluate input data and assess overall inventory Collect data and estimate emissions and removals Compile inventory Make necessary revisions (if any)

12. Sources of Uncertainty  Assumptions and methods  These method may not accurately reflect the emission. Good practice requires that biases be reduced as much as possible. Guidelines aim to be as unbiased and complete as possible.  Input data  Measured values have errors and emission factors may not be truly representative.  Calculation errors  Implement good QA/QC to stop these. 12 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

13. Uncertainties Arise in Input Data Because of…  Lack of data  Use of proxies, extrapolation etc.  Missing data.  Data not truly representative  Statistical random sampling error  Measurement error  Misreporting. Consideration of these during data collection phase will minimise errors. 13 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

14. Uncertainty Information 14 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

15. Sources of Data  National statistical agencies  Sectoral experts, stakeholder organizations  Other national experts  IPCC Emission Factor Database  Other international experts  International organizations publishing statistics e.g., United Nations, FAO, IEA, OECD and the IMF (which maintains international activity as well as economic data)  Reference libraries (national libraries)  Scientific and technical articles in environmental books, journals and reports.  Universities  Web search for organizations and specialists  National Inventory Reports from Parties to the United Nations Framework Convention on Climate Change  Ministries, local and traditional authorities. 15 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

16. Expert Judgement  Expert judgement on methodological choice and choice of input data to use is ultimately the basis of all inventory development.  Experts with suitable backgrounds can be found in government, industrial trade associations, technical institutes, industry and universities.  The goal of expert judgement may be choosing the:  Proper methodology  Parameter value and uncertainty from ranges provided  Most appropriate activity data to use  Most appropriate way to apply a methodology  Or determining the appropriate mix of technologies in use.  Expert judgement is always required since one must judge whether the data are a representative random sample and, if so, what methods to use to analyse the data.  This requires both technical and statistical judgement. 16 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

17. Working With Experts  Wherever possible, expert judgement should be elicited using an appropriate protocol (e.g. Stanford/SRI protocol):  Motivating: Establish a rapport with the expert, describe the context, explain the most commonly occurring biases.  Structuring: Clearly define the quantities for which judgements are to be sought (e.g. resulting emissions or removals should be for typical conditions averaged over a one- year period).  Conditioning: Work with the expert to identify and record all relevant data, models and theory relating to the formulation of the judgements.  Encoding: Request and quantify the expert’s judgement (may differ but should include uncertainty information).  Verification: Analyse and feedback conclusions regarding their judgement. Is what has been encoded really what the expert meant?  Document! 17 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

18. Remember…  Producing high quality “good practice” emission and removal estimates is paramount.  Efforts on uncertainty analysis should be small in comparison with the efforts on the inventory estimates themselves.  Data collection activities should consider data uncertainties:  This will ensure the best data is collected and ensures good practice estimates  As you collect data you should assess how “good” it is.  At its simplest, a well-planned uncertainty assessment should only take a few extra hours! 18 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

19. Methods to Combine Uncertainties  Error propagation  Simple - standard spreadsheet can be used  Guidelines give explanation and equations  Difficult to deal with correlations  Strictly (standard deviation/mean) < 0.3  A simple solution is provided.  Monte-Carlo Simulation  More complex - Use specialized software  Needs shape of probability density function (pdf)  Suitable where uncertainties are large, non-Gaussian, complex algorithms, correlations exist and uncertainties vary with time. 19 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

20. Enter emissions data Enter uncertainties Data calculated using simple equations 20 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories Error Propagation - Example

21. Note short list of source/sinks Emission factor uncertainties based on defaults in guidelines Activity data uncertainties based on source of data

22. Error Propagation: example of results 22 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

23. Error Propagation: example of results (continued) 23 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

24. Monte-Carlo Method – Simplified Approach: example  Key requirements  Not just uncertainties but also probability density function (pdf)  Mean  Width  Shape (e.g. normal, log-normal, Weibul, gamma, uniform, triangular, fractile  Principle  Select random values of input parameters from their pdf and calculate the corresponding emission. Repeat many times and the distribution of the results is the pdf of the result, from which mean and uncertainty can be estimated. 24 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

25. Monte-Carlo Method – Simplified Approach: example (continued) 25 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

26. Monte-Carlo Method – Simplified Approach: 1 run 26 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

27. Monte-Carlo Method – Simplified Approach: 10 runs 27 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

28. Monte-Carlo Method – Simplified Approach: 20 runs 28 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

29. Monte-Carlo Method – Simplified Approach: 50 runs 29 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

30. Monte-Carlo Method – Simplified Approach: 100 runs 30 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

31. Monte-Carlo Method – Simplified Approach: 500 runs 31 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

32. Monte-Carlo Method – Simplified Approach: 1000 runs 32 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

33. Monte-Carlo Method – Simplified Approach: 5000 runs 33 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

34. Monte-Carlo Method – Simplified Approach: 10,000 runs 34 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

35. Monte-Carlo Method – Simplified Approach: summary results 35 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

36. Summary  Even simple uncertainty estimates give useful information - If they are performed well!  Assessment of uncertainty in the input parameters should be part of the standard data collection QA/QC:  Careful consideration will improve estimates as well as providing input data for uncertainty analysis.  If resources are limited: amounts spent on uncertainty analysis should be small compared with total effort.  For simple estimates:  Uncertainty in activity data assessed as data collected  Uncertainty in emission factors from guidelines if readily available  Aggregate categories/gases to independent groups of sources/sinks  Use Approach 1 – the spreadsheet requires little statistical knowledge. 36 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories

37. Thank you Diagrams © IPCC Except where noted otherwise 37 Consultative Group of Experts (CGE) Training Materials for National Greenhouse Gas Inventories