by Hélène Erkel-Rousse and Christelle Minodier (INSEE, France) Third Joint European Commission-OECD Workshop on International Development of Business and Consumer Tendency Surveys, Brussels, November 2007 Short-term forecasting of the GDP growth rate using the BTS in industry and in services: An out-of-sample analysis

Page 2 The two issues addressed in the paper: › 1) BTS are widely used for the short-term forecasting of economic activity. However, to our knowledge, there has been no recent attempt to establish the significance of their contribution to the quality of forecasts (using modern econometric techniques). › 2) The service survey contains a specific piece of information on GDP growth with respect to the industry survey (Bouton and Erkel-Rousse, 2003). However, it remains to be shown that this specific piece of information permits one to significantly improve the quality of short-term GDP forecasts with respect to models involving variables from the industry survey exclusively.

Page 3 The study consists in: › - estimating several VAR models and univariate calibration models of the GDP quarterly growth rate encompassing miscellaneous variables derived from the industry and service surveys as well as GDP lags › - estimating competing models with no service variable or, even, no BTS variable (simple AR models of GDP) on several time periods (real-time analysis - RTA) › - simulating each model up to a four-quarter forecast horizon, and deriving series of forecasting errors (RTA) › - comparing the predictive accuracy of the different models using Clark-West or Harvey, Leybourne and Newbold tests (depending on the models) › Data used: quarterly survey data and, to a lesser extent, monthly survey data.

Page 4 Structure of the Talk › Data › Methodology › Results › Conclusion

Page 5 Data (1) General characteristic features › : creation of the INSEE industry survey › : creation of the INSEE service survey (on a quarterly basis) › - June 2000: creation of the monthly service survey, adding of a few questions › - Since the 1990’s: progressive extension of the sector coverage of the service survey (in the data used: business services (2/3), household services (> 20%), real estate activities (>10%)) › - January 2004: several changes in the wording of the questionnaires of the two surveys and adding of a few questions in the service survey for European harmonisation purpose › - January 2004: the two surveys become compulsory

Page 6 Data (2) The data used in the study › Macro variable to be forecasted:  the quarterly GDP growth rate from the French Quarterly Accounts (hereafter “GDP”) › Industry survey data:  the 6 main monthly balances and 2 quarterly ones (past and expected demand)  2 static common factors in industry: a monthly one (derived from the 6 main monthly balances – published each month by INSEE) and a quarterly one (encompassing the 6 monthly balances + the 2 quarterly ones) › Service survey data:  the 3 main monthly balances and 3 main quarterly ones  the dynamic common factor in services (introduced by Cornec and Deperraz, 2007, derived from the 6 balances and published each month by INSEE)

Page 7 Methodology (1) › We define several forecasting models of GDP for each month in a given quarter, so as to be able to up-date the short-term forecasts of GDP each month on the basis of the most recent piece of information given by the BTS. Notation: Ind_m1 = variable Ind derived from the industry survey relating to month 1 Ser_m4 = variable Ser derived from the industry survey relating to month 4… with m1 (m2,m3, resp.) = 1 st (2 nd, 3 rd, resp.) month in the current quarter and m4 = 1 st month in the following quarter. Examples: - In the second quarter of year 2000, m1 = April, m2 = May, m3 = June, m4 = July, of year In the last quarter of year 2004, m1 = October, m2= November, m3 = December 2004, m4 = January 2005.

Page 8 Methodology (2): VAR models Comparison of the predictive accuracy of 3 VAR models of 2 kinds (non restricted, restricted) per couple of survey variables used: - VAR with 3 variables: GDP, Ind_mi, Ser_mi - VAR with 2 variables: GDP, Ind_mi - simple AR model of GDP (basic benchmark) with Ind_mi (Ser_mi, resp.) = a survey variable from the industry (services resp.) survey relating to month mi, i = 1 to 4. Non restricted VAR = VAR with 2 lags estimated using OLS Restricted VAR = VAR with 4 lags with exclusion restrictions, estimated using SURE Tests of equal predictive accuracy in nested models = Clark-West tests, with: 1) Scilab – Grocer (correcting autocorrelation within forecast error series using Newey-West variances, for forecast horizons = 2 to 4 quarters) 2) SAS – procedure autoreg, options nlag=4 and backstep (testing for autocorrelation up to 4 lags and correcting it when necessary using Yule-Walker estimates)

Page 9 Methodology (3): Univariate calibration models (1) › Intuition: when the length of both estimation series and forecast error series is short (months 2 and 3, especially 2 – no survey in August), univariate calibration models might be better adapted because more parsimonious than VAR models. › Kinds of models estimated: one set per month mi (i = 1 to 4). At a given month when BTS are available up to the 1st forecast horizon: Models used for the forecasting of GDP at a 1 quarter horizon: GDP = function of the current and lagged values of survey variables Models used for the forecasting of GDP at a 2 quarter horizon: GDP = function of the lagged values of survey variables Models used for the forecasting of GDP at a 3 quarter horizon: GDP = function of the lagged values of survey variables to the exclusion of the first lags

Page 10 Methodology (4): Univariate calibration models (2) Comparison of the predictive accuracies of several univariate calibration models of GDP: - some including explanatory variables from the two BTS - some including explanatory variables from the industry survey only - AR model of GDP (basic benchmark) - the “best” VAR3 models with, again, different sets of models depending on the month in (or just after) the quarter (mi, i = 1 to 4). Depending on the models whose predictive accuracies are compared, we performed: - either Clark-West tests (for the comparison of nested models) - or Harvey, Leybourne and Newbold tests (for the comparison of non-nested models) Software used: Scilab – Grocer.

Page 11 Results 1) VAR models (1) › Choice of survey variables: - Most leading balances: Industry survey: expected production (monthly) Service survey: expected operating profit (quarterly) - Most correlated common factors (with GDP) at each month mi: Industry survey: for m1 and m4: the quarterly common factor for m2 and m3: the monthly common factor Service survey: the dynamic common factor

Page 12 Results 1) VAR models (2) › Contribution of BTS to the short-term forecasting of GDP (with respect to benchmark AR models): - Very significant at the 1 and 2 quarter horizons in most cases (very small P-values) - Significant at the 5% or 10% thresholds for several models relating to quarterly months (m1 and m4) at the 3 and, even, 4 quarter horizons (even though the quality of their forecasts remains poor) - Often more significant to forecast the first release of GDP than the last available release (at the 3 and 4 quarter horizons and, to a lesser extent at the 2 quarter horizon)

Page 13 Results 1) VAR models (3) › Contribution of the service survey to the short-term forecasting of GDP (with respect to VAR models with 2 variables: GDP and Ind_mi): 1) In the case of “quarterly” months (m1 and m4), for which fairly long time series in services are available: - Significant, especially when the service variable is the dynamic common factor - Higher contribution: for the 2 and 3 quarter forecast horizons (non significant at the 4 quarter horizon) 2) In the case of “non quarterly” months (m2 and m3), for which only short time series for services are available - Less high contribution of the service survey at this stage… but some encouraging significant results for month m2 - Important remark: The methodology used creates a serious bias against the service survey (either linear interpolations are used for data before June 2000 or the last available quarterly value of a balance – that in m1- is used for months m2 and m3 while more recent monthly industry data are used)

Page 14 Results 1) VAR models (4) Example: Results from the non restricted VAR model with the quarterly common factor in industry and the dynamic common factor in services (m4)

Page 15 Results 2) Univariate calibration models (1) We are currently working on these models. Two kinds of models have been estimated, using the Scilab-Grocer software: -Models whose optimal specifications are automatically determined by the software -Models whose specifications are determined by the authors so that every explanatory variable has an impact of the expected sign on GDP. Our first preliminary results suggest that these kinds of models might enable one to lead to clearly positive results as concerns the contribution of the service survey, notably in month m3 (especially models whose specifications are automatically determined). However (at least on the basis of our preliminary results), these kinds of models do not appear to always lead to significantly better GDP forecasts than the VAR models.

Page 16 Results 2) Univariate calibration models (2) Example: Univariate models relating to a three-quarter forecast horizon

Page 17 Conclusion › 1) The study clearly confirms the predictive power of BTS for GDP growth at short-term horizons › 2) It also shows that the quarterly service survey has predictive power alongside with the manufacturing survey › 3) As far as monthly service data are concerned, it is definitely too early to have firm conclusions. Our results derive from methodologies that generate negative biases to the detriment of the service survey (use of either linear interpolated data before June 2000 or less up-to-date data than industry ones). Nonetheless, some of the results suggest that monthly service data might also permit one to improve the short-term forecasting of GDP. To be confirmed in 6 or 7 years when long enough monthly service series are available!