“Analytical Tools and Data Collection” Workshop 15-16 April 2009 EuropeAid/125317/D/SER/TR Roman Ladus Session 6 Measuring Impacts 1

Stages of Impact Analysis Assess Impacts and Compare Options Gather Evidence Identify Impacts

Three Major Types of Policy Analysis Cost Benefit Analysis (CBA) = Benefits minus Costs Compares overall negative and positive impacts to ask IF options should be considered based on whether they produce more benefits than costs.. Cost Effectiveness Analysis (CEA) = Compares costs per metric of benefit for different options and asks WHAT KIND of action should be taken to maximize results per unit cost. Least Cost Analysis (LCA) = lowest cost to reach a FIXED target. Compares options that reach the same result and asks WHICH action reaches a fixed target at lowest cost. Costs Effects (lives saved, accidents reduced) See Reach proposal – a cost-effectiveness test in a benefit-cost analysis 3

CEA Example: Reduction of Injuries in Minibuses Costs (TL) No. of injuries prevented CEA Option 1 4,000,000 1,200 3,333 Option 2 5,000,000 1,800 2,778 Option 3 7,000,000 2,000 3,500

LCA Example: Reducing Emissions Fixed Target Option Cost Reduce emissions by 30 % Fixed caps for all emissions € 300 million Tradable permits € 100 million More stringent regulation on high volume emissions € 200 million

Measuring & Analysis: Three Elements of CBA Identification of potential benefits and costs Evaluation of costs and benefits Spread of costs and benefits over time

Measuring & Analysis: Identification of Costs and Benefits Set baseline Consider ‘Do nothing option’ Avoid double counting

Measuring & Analysis: Example of wrong baseline Issue: Re-equipped Minibuses causing injuries and discomfort Option: Do nothing Option: Quality requirements Benefits: None Decrease in healthcare expenses Increase in productivity of passengers Costs: High healthcare expenses Lost productivity of passengers Investment in new minibuses

Measuring & Analysis: Example Double Counting Option: Ban of smoking in public places Benefits: Health benefits Productivity benefits Costs: Decrease in sales for importers Decrease in sales for producers Decrease in sales for bars

Measuring & Analysis: Quantification and Valuation Whenever possible, quantify (measure) impacts using different metrics (kilos, metres etc) Whenever possible, value impacts in a single metric to enable adding them together and easy comparison Monetary valuation is also referred to as ‘market valuation’ or ‘economic valuation’ Use different scenarios whenever appropriate

Measuring & Analysis: Estimating Prices Market value = willingness to pay Some prices do not have market value: Health benefits Time savings Environmental benefits and costs Human life The basic approach is to estimate the ‘willingness to pay’ (or ‘willingness to accept compensation’) for a particular outcome / opportunity cost

Measuring & Analysis: Methods for Estimating Prices 1. Use market prices information Change in productivity Human capital cost valuation 2. Valuation using information on individuals’ preferences Replacement cost or preventive expenditure method Contingent valuation method Surrogate market valuation method 3. Benefit Transfer Use estimates of shadow prices from other studies

Measuring & Analysis: Spread of Costs and Benefits over Time The value of 1TL received tomorrow is less than the value of 1TL received today Individuals expect to receive a rate of interest on savings to compensate for giving up consumption today The rate of interest gives us a measure of an individual’s rate of time preference Using rate of time preference to convert future value into equivalent value today is called discounting. This enables comparison of benefits and costs that occur in different time periods

Measuring & Analysis: Calculating Net Present Value/Discounting (Benefits – Costs) * Discount Factor, for year 0 + (Benefits – Costs) * Discount Factor, for year 1 + (Benefits – Costs) * Discount Factor, for year 2 + etc Discount Factor = (use discount tables) Benefits justify the costs if the NPV is positive (NPV>0) 1 (1+ interest rate)year

Measuring & Analysis: Example of NPV Calculation Year 1 2 3 4 Benefits 1500 Costs 300 Net Benefits -1500 -300 1200 Discount Factor (for 10%) 1.000 0.909 0.826 0.751 0.683 Discounted Net Present Value -272.7 -247.8 901.2 819.6 Cumulative Net Benefits = – 1500 – 300 – 300 + 1200 + 1200 = 300 Net Present Value = – 1500 – 272.7 – 247.8 + 901.2 + 819.6 = – 299.7

Measuring & Analysis: Risk Assessment In assessment of impacts consider risks and likelihood of their occurrence Describe the consequences of risks – could they stop the regulation from meeting its objectives? Risk analysis provides an overall judgement on whether the risk-adjusted benefits exceed the risk-adjusted costs

Measuring & Analysis: Must We Quantify Everything? No, but…. Governments should not act unless the results are worth the costs Governments should find the lowest cost solution to solve problems Good qualitative comparisons may be better than very uncertain quantitative comparisons Use quantitative and qualitative metrics to compare benefits and costs The important point is that benefits and costs are sufficiently clear that the public can understand

Qualitative Weighting Where it is difficult to value impacts, describe them by looking at other impact measures You might provide information such as: how many people will be affected what type of people might be affected the nature and impact of the effects Effects may be categorized as ‘small’, ‘medium’ or ‘large’ to compare and weight various impacts 18

Qualitative Presentation: EC Sugar Quotas IA 19

Qualitative Presentation: UK RIA on Implementation of EU Directive on Maximum Levels of Organochlorine Compounds in Animal Feed

Qualitative presentation: EU Postal Services (2006) 21

Quantitative presentation: UK RIA for Directive on Minimum Rules for the Protection of Chicken Kept for Meat

Quantitative presentation: UK RIA for Directive Laying Down Minimum Rules for the Protection of Chicken Kept for Meat

Continuation of the previous example

Combining Quantitative and Qualitative Presentations Example: UK RIA on Gambling Act Option 1 Retaining the status quo Option 2 Implement the Gambling Act 2005 Total cost per annum No additional costs £1.9 – 2.6m Total benefit per annum No additional benefits Greater employment opportunities Higher revenue generation as a result of additional websites being based in the UK and non-UK residents visiting UK-based websites Improved regulation Better choice, information and safeguards for customers Other costs Voluntary contributions to the Responsibility In Gambling Trust Additional software to protect the vulnerable and prevent crime Training to identify problem gambling Source: www.evidence-based-medicine.co.uk 25

Presenting Results and Recommendation: Decision Criteria Assessment of costs and benefits informs decision and provides evidence for recommended option Do the benefits justify the costs? Is the approach the least costly option for achieving the objective?

Presenting Results and Recommendation Result of assessment should be clearly described and explained in RIA Sources of evidence should be provided to build credibility Quantify as many impacts (costs and benefits) as possible Present results in tabular form: show positive impacts (benefits) and negative impacts (costs) separately and, if possible, net benefits are calculated Present separately any significant distributional impact