1. Some comments about forecasting Based on a paper provisionally entitled “Forecasting GDP and its expenditure components by the Economist Intelligence Unit: Are Country Reports worth paying for?” Corné van Walbeek

2. Forecasting techniques Non-quantitative techniques –“I think that…” –Consensus seeking (e.g. Delphi method) –Scenario planning Quantitative techniques –Time series methods (e.g. ARIMA) –Predicting with simple (single equation) behavioural models –Multiple equation models Others –Technical analysis, especially for shares and currencies

3. Some background about macroeconomic forecasting Economists are not particularly good at forecasting –Especially not in turbulent times (Granger, 1996) –Very poor at predicting recessions (Loungani, 2001) –Forecasts tend to cluster together, often quite far from the actual value (Granger, 1996) Most studies consider the accuracy of GDP growth and inflation forecasts (Ash et al, 1998, Oller & Barot, 2000, Vogel, 2007) Strong focus on industrialised countries (US agencies, IMF, OECD) Strong focus on institutional forecasts; not much on private sector forecasts

4. Criteria for forecast accuracy Bias –Mean error Size of forecast error –Root mean square error Ability to beat naïve alternative –RMSE EIU /RMSE naive < 1 Directional accuracy –Forecasting accelerations and decelerations correctly

5. A typical forecasting process Use econometric models –Details are often published if organisation is “public” –If it is a private company, details typically not provided Model consists of –Behavioural equations –Standard macroeconomic identities (e.g. GDP = C + I + G + X - M –Global identities (e.g. ΣX = ΣM) if relevant Distinguish between exogenous and endogenous variables Manual adjustments are made to forecasts if deemed necessary Rigorous and iterative process of quality control and checking of forecasts

6. An example of the data: Austria, January 2007 Current year (t) forecast Next-year (t+1) forecast “Actual” value of last year (t-1) Against this value the forecasts for 2006 are measured

7. Magnitudes of the forecast errors

8. Some comments about the RMSEs 1.They are large –For current year forecasts: between 1.4 and 9.8 percentage points; median = 3.5 percentage points –For next-year forecasts: between 15 and 30 per cent larger than current-year forecasts 2.Large differences in RMSEs between magnitudes –RMSEs around 2 percentage points: C, G, TDD and GDP –RMSEs around 5 percentage points: I, X and M 3.Lower RMSEs for developed countries; higher RMSEs for developing countries

9. Comparing the EIU’s forecasts against naïve predictions Assumption used for this paper: –The naively predicted growth rate for this year and for next year is the “estimated” growth rate for the previous year Calculate RMSE ratio = RMSE EIU /RMSE naive If RMSE ratio < 1, then EIU forecasts are better (have smaller errors) than naïve alternative

10. Average of 0.77 Average of 0.82

11. Two recommendations 1.More modesty please! –Words like “prescient”, “decisive verdicts”, “precision”, etc. do not belong in a forecaster’s vocabulary 2.Publish confidence intervals –E.g. 67% confidence intervals (= point estimate ± RMSE) –67% (or 50%) confidence intervals 1.Are not affected by outlying forecast errors 2.Are not as large as 95% confidence intervals (see Granger, 1996) –The existing RMSEs would be a good first approximation for such intervals –What if the intervals are embarrassingly large? Be honest (“This magnitude is very difficult to forecast”) Advantages of publishing confidence intervals: 1.Emphasises the stochastic nature of forecasting to clients 2.Increases the credibility of the EIU (“Now they are always wrong. At least they will be right two thirds of the time”) 3.Allows users to do scenario planning with realistic “optimistic” and “pessimistic” scenarios