1. Centraal Bureau voor de Statistiek Marcel Pommée National Accounts Department Statistics Netherlands (CBS) Semi-automatic integration in Semi-automatic integration in Dutch Supply and Use Tables – with special reference to time-series

2. Centraal Bureau voor de Statistiek Outline SUTs, characteristics SUTs, characteristics Reasons for automation Reasons for automation Automation with ‘machines’ Automation with ‘machines’ Balancing machine Balancing machine Quarterly machine Quarterly machine Time-series machine Time-series machine Time-series project Time-series project Summary Summary

3. Centraal Bureau voor de Statistiek SUTs, characteristics Annually, quarterly (t+45 and t+30 ?)Annually, quarterly (t+45 and t+30 ?) Detail: industries (120), commodities (630), expenditure categories (50)Detail: industries (120), commodities (630), expenditure categories (50) Focus on year on year changes, no seasonal adjustmentFocus on year on year changes, no seasonal adjustment Simultaneous balancing of current and constant pricesSimultaneous balancing of current and constant prices Data gaps filled with assumptions and extrapolationsData gaps filled with assumptions and extrapolations Manual balancingManual balancing

4. Centraal Bureau voor de Statistiek Reasons for automation Efficiency gains: budget reductions up to 34% expected in 2018Efficiency gains: budget reductions up to 34% expected in 2018 Part of redesign of chain of economic statistics:Part of redesign of chain of economic statistics: More structured processMore structured process Top-down approachTop-down approach Focus on major problemsFocus on major problems QualityQuality Visibility various adjustment steps (transparancy)Visibility various adjustment steps (transparancy) Consistency over timeConsistency over time Reproduce results (consistency)Reproduce results (consistency) First gdp estimates quickly available (analysis)First gdp estimates quickly available (analysis)

5. Centraal Bureau voor de Statistiek Automation with ‘machines’ Quadratic optimization modelQuadratic optimization model Minimizing the adjustment needed to the growth rates of quarterly seriesMinimizing the adjustment needed to the growth rates of quarterly series T-1 and unbalanced data in current and constant prices availableT-1 and unbalanced data in current and constant prices available Only semi-automatic integrationOnly semi-automatic integration Major problems tackled manuallyMajor problems tackled manually Small problems resolved through automationSmall problems resolved through automation Machines for different purposes:Machines for different purposes: Balancing machine: balancing single SUTBalancing machine: balancing single SUT Quarterly machine: rebasing years and aligning quartersQuarterly machine: rebasing years and aligning quarters Time-series machine: rebasing time-seriesTime-series machine: rebasing time-series

6. Centraal Bureau voor de Statistiek Balancing machine Balancing single SUT: major problems solved manuallyBalancing single SUT: major problems solved manually Hard and soft constraintsHard and soft constraints Suppy is equal to use by commoditySuppy is equal to use by commodity Preserve price indices by commodityPreserve price indices by commodity Preserve i/o-ratios by branches of industryPreserve i/o-ratios by branches of industry Compute trade and transport marginsCompute trade and transport margins Compute taxes and subsidies on productsCompute taxes and subsidies on products Upper and lower bounds for individual variablesUpper and lower bounds for individual variables Fixation of variablesFixation of variables Weighting of variables based on quality of datasourceWeighting of variables based on quality of datasource Specific relations (import and re-exports, building materials and construction)Specific relations (import and re-exports, building materials and construction)

7. Centraal Bureau voor de Statistiek Quarterly machine Compilation cycle: final (F), preliminary (P), very preliminary (V)Compilation cycle: final (F), preliminary (P), very preliminary (V) Quarterly machineQuarterly machine Input: F-year and 12 quarters of previous cycleInput: F-year and 12 quarters of previous cycle Output: rebased P- and V- year and 12 aligned quartersOutput: rebased P- and V- year and 12 aligned quarters Updating of P- and V-yearUpdating of P- and V-year Selected information addedSelected information added With balancing machineWith balancing machine

8. Centraal Bureau voor de Statistiek Time-series machine Time-series (1990-2009) based on benchmark revision 2010 Time-series (1990-2009) based on benchmark revision 2010 Reconstruction of complete SU and IO tables Reconstruction of complete SU and IO tables Earlier Earlier Year by year compilation => very time-consumingYear by year compilation => very time-consuming Difficult to preserve price and volume indices of original seriesDifficult to preserve price and volume indices of original series Time-series machine Time-series machine New levels given by benchmark year and reference yearsNew levels given by benchmark year and reference years Reference years, e.g. 1987, 1995, 2001 (previous revision years)Reference years, e.g. 1987, 1995, 2001 (previous revision years) Preservation of price and volume indices of original seriesPreservation of price and volume indices of original series Iterative process, manual interventionIterative process, manual intervention Result: fully consistent time-series in current and constant pricesResult: fully consistent time-series in current and constant prices

9. Centraal Bureau voor de Statistiek Time-series project (1) Year 2010 ESA 2010 conceptual revision and benchmark (statistical) revision Year 2010 ESA 2010 conceptual revision and benchmark (statistical) revision Revised GDP 7,6% higher (concepts 3% and benchmark 4,6%) Revised GDP 7,6% higher (concepts 3% and benchmark 4,6%) Covers Fully consistent ANA, QNA, ASA, QSA, LA, Covers Fully consistent ANA, QNA, ASA, QSA, LA, SUTs, IO-tables, and regional accounts Planning Benchmark year 2010publication 6th March 2014 Planning Benchmark year 2010publication 6th March 2014 2001-2009, up to 2013publication 20th June 2014 2001-2009, up to 2013publication 20th June 2014 1995-2000publication 24th September 2014 1995-2000publication 24th September 2014 Extremely tight schedule Extremely tight schedule

10. Centraal Bureau voor de Statistiek Time-series project (2) Time-series 2001-2009: series of problems Time-series 2001-2009: series of problems Start-up problems time series machine (coding errors, retrieving data, capacity limitations, processing time)Start-up problems time series machine (coding errors, retrieving data, capacity limitations, processing time) Takes a lot of time to specify constraints (fixation of variables, notably government data, weighting of variables, notably prices)Takes a lot of time to specify constraints (fixation of variables, notably government data, weighting of variables, notably prices) Many interdependencies with other NA-modules (LA, ASA, government data and financial institutions, fisim)Many interdependencies with other NA-modules (LA, ASA, government data and financial institutions, fisim) Time-series machine had to make quite large adjustments in the original series due to substantial level shifts in revision yearTime-series machine had to make quite large adjustments in the original series due to substantial level shifts in revision year

11. Centraal Bureau voor de Statistiek Time-series project (3) Time-series 2001-2009: consequences Time-series 2001-2009: consequences Difficult to understand what the machine is doing (black box)Difficult to understand what the machine is doing (black box) Planning deadlines were not metPlanning deadlines were not met Hardly any documentationHardly any documentation Data results machine less than optimal: extensive manual interventions => publication is on provisional basisData results machine less than optimal: extensive manual interventions => publication is on provisional basis Highly motivated team but tension and frustration due to adversitiesHighly motivated team but tension and frustration due to adversities

12. Centraal Bureau voor de Statistiek Time-series project (4) Time series 1995-2000: gaining experience Time series 1995-2000: gaining experience Compilation process split into parts with : Firstly a basic run without some constraints (no commodity balancing) => less adjustments by machineFirstly a basic run without some constraints (no commodity balancing) => less adjustments by machine Secondly manual intervention to solve major problems (imbalances)Secondly manual intervention to solve major problems (imbalances) Final run to solve minor inconsistencies and restore all constraintsFinal run to solve minor inconsistencies and restore all constraints Results were much better Better understanding of the machine outputBetter understanding of the machine output Less or almost no manual intervention neededLess or almost no manual intervention needed

13. Centraal Bureau voor de Statistiek Time-series project (5) Time series 1995-2000: some lessons learned Time series 1995-2000: some lessons learned Takes a lot of time to smoothly run the time-series machineTakes a lot of time to smoothly run the time-series machine Manual adjustments are mostly complex as they often affect large parts or the whole time-seriesManual adjustments are mostly complex as they often affect large parts or the whole time-series Due to all interdependencies it is important to stick to the planningDue to all interdependencies it is important to stick to the planning Complexities warrant an experienced team of compilersComplexities warrant an experienced team of compilers

14. Centraal Bureau voor de Statistiek Summary Almost 3 years experience with ‘machines’Almost 3 years experience with ‘machines’ ProsPros Efficiency gains: in terms of less fte’sEfficiency gains: in terms of less fte’s More robust statistical processMore robust statistical process Improved qualityImproved quality ConsCons Investment to develop and to buildInvestment to develop and to build Programming errorsProgramming errors More complex: takes time to understand what the machine is doingMore complex: takes time to understand what the machine is doing