European Conference on Quality in Official Statistics Quality Assurance of German Balance of Payments Statistics - 2008 update - Sabine Guschwa* sabine.guschwa@bundesbank.de European Conference on Quality in Official Statistics Rome, 8-11 July 2008 * The opinions expressed are those of the author and do not necessarily reflect the views of the Deutsche Bundesbank.

Quality assurance of German b.o.p. statistics Outline 1. Introduction 2. Quality assessment of German b.o.p. data 2008 3. First results for German i.i.p. data 4. Major influencing factors/limitations 5. Summary outlook 22.09.2018 Quality assurance of German b.o.p. statistics

Quality assurance of German b.o.p. statistics Introduction Aim of quality reporting => Inform users through: Summary information on current work Further analysis of data Information on usage of data Communication of measures, plans etc. Complementing existing information 22.09.2018 Quality assurance of German b.o.p. statistics

Quality assurance of German b.o.p. statistics Introduction Development/Short history: IMF Data Quality Assessment Framework (DQAF) (2003) Joint ECB/Eurostat TF on Quality (TF QA) (2003 – 2005) (Follow-up) Work on quality issues (ECB, Eurostat, DE) IMF ROSC Mission in Germany (2005) Available b.o.p. quality assessments/reports: ECB “EU B.o.p./I.i.p. Statistical Methods” (B.o.p. Book) IMF SDDS Metadata IMF “Germany: ROSC Report – Data Module, Response by the Authorities, and Detailed Assessment Using the DQAF” (Jan. 2006) ECB “Euro area b.o.p. and i.i.p. statistics, Annual quality report 2008”, Guideline ECB/2004/15 of 16 July 2004, ECB/2007/3 Quality assessments of data transmitted to Eurostat. Commission/Eurostat quality assessment according to EC Regulation 184/2005 of 12 January 2005, EC 602/2006 Outline of a quality report for German b.o.p. in 2006, updated in 2007 and 2008 22.09.2018 Quality assurance of German b.o.p. statistics

Quality assurance of German b.o.p. statistics 1. Introduction Quality assessments/reports => follow internationally recommended quality dimensions defined by the IMF DQAF (ECB, Eurostat, DE): Prerequisites of quality Assurances of integrity Methodological soundness Accuracy and reliability Serviceability Accessibility Combining qualitative and quantitative information. 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Methodological soundness Chronology of changes in German b.o.p. for 1995-2008 (excerpt) Date Changes to the balance of payments Period March 2008 Methodological changes: netted disclosure of tax revenue & refunds (as negative tax revenue) transfer of settlement contributions to the EU budget from VAT-based to GNI-based resources desulphurisation of natural gas recorded as processing change in recording of warrants with automatic exercise feature under debt securities (from financial derivatives) accruals recording of transfers from EU budget (EU agr. fund) http://www.bundesbank.de/download/statistik/sdds/stat_zahlungsbilanz/sdds_zb_revisionen_2007_e.pdf 1981 to date 2007 to date 1977 to date Feb 2007 to date 2006 - 2003 March 2007 Methodological changes cover: recording of transfers from EU budget (EU agricultural fund) at the time when the claim arises (accruals basis) monthly allocation of expenditure on travel defined more precisely, further refinement of factors applied to package holidays http://www.bundesbank.de/download/statistik/sdds/stat_zahlungsbilanz/sdds_zb_revisionen_2006_e.pdf 2006 to date 2006/2002 to date 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Accuracy and reliability Quantitative indicators on revision studies (TF QA): -> 2008 update for b.o.p. data (first results in 2004) -> 2008 first results for quarterly i.i.p. stock data!! -> for b.o.p. current account gross flows, i.i.p. stock data: Mean absolute percentage error / revision (MAPE) Percentage of upward revisions (% UR) Directional reliability (Q (%)) -> for b.o.p. net flows: Mean absolute relative error / revision (MARE) Root mean square relative error / revision (RMSRE, bias, regression, unsystematic component) % UR Q (%) 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Accuracy and reliability Revision studies (size, direction, reliability of trends) Revision studies for German b.o.p. as of April 2008 B.o.p. Item Period MAPE % Upward revision Q (%)   3rd quartile 90th percentile Max Current account credits Jan04 - Dec07 0.038 0.055 0.064 0.119 93.8% 87.2% Current account debits 0.022 0.034 0.046 0.093 68.8% 89.4% Goods credits (exports) 0.005 0.006 0.011 0.014 60.4% 95.7% Goods debits (imports) 0.007 0.026 47.9% 93.6% Services credits 0.107 0.139 0.170 0.231 100% 91.5 % Services debits 0.043 0.062 0.082 0.169 83.3% 91.5% Income credits 0.236 0.408 0.483 0.707 95.8% 66.0% Income debits 0.156 0.230 0.399 0.574 50.0% Current transfers credits 0.293 0.166 0.829 3.334 37.5% 83.0% Current transfers debits 0.035 0.052 0.091 0.142 52.1% 85.1% 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Revision studies for German b.o.p. as of April 2008, contd. B.o.p. Item Period MARE RMSRE / MSRE % Upward revision Q (%)   3rd quartile 90th percentile Max Mean comp. (%) Regress comp. (%) Unsystcomp. (%) Capital acc. - net Jan04-Dec07 0.393 0.092 0.500 10.667 1.050 4.5% 18.8% 76.7% 43.8% 93.6% Direct inv. – net 0.542 0.718 1.191 2.085 0.520 12.6% 0.0% 87.4% 27.1% 89.4% Direct inv. abr. 0.552 0.656 0.941 1.866 0.506 57.0% 0.3% 42.7% 10.4% 87.2% Direct inv. in G. 0.502 0.638 1.108 2.049 0.455 27.4% 2.2% 70.5% 75.0% Portf. inv. – net 0.269 0.312 0.465 1.296 0.282 0.5% 0.7% 98.8% 54.2% Portf. inv.assets 0.192 0.277 0.354 0.644 0.190 0.4% 4.6% 95.0% 41.7% 95.7% Portf. inv. liab. 0.281 0.375 0.523 1.513 0.309 0.9% 1.4% 97.7% Other inv. – net 0.145 0.187 0.693 0.141 9.9% 89.7% 66.7% 100% Other inv. assets 0.066 0.095 0.151 0.257 0.067 13.6% 1.5% 84.9% 39.6% Other inv. liab. 0.075 0.098 0.168 0.303 0.079 40.3% 0.1% 59.6% 79.2% 97.6% 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Revision studies on current transfers credits Date Reference period MAPE % Upward revisions Q (%)   3rd quartile 90th percentile Max Apr 08 Jan04 - Dec07 0.293 0.166 0.829 3.334 37.5% 83.0% Apr 07 Jan03 - Dec06 0.165 0.125 0.205 3.363 54.2% 80.9% Apr 06 Jan02 - Dec05 0.085 0.115 0.173 0.408 39.6% 85.1% 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Main results from 2008 b.o.p. revision studies: Comparatively large MAPE values for current transfers credits influenced by methodological change (accruals for EU transfers) Comparatively large MAPE/MARE for income credits and debits / direct investment abroad and in Germany, combined with systematic pattern of revisions, (comp. lower Q(%)) -> Supported by earlier years´ results Due to inclusion of new data from direct investment survey replacing and/or correcting preliminary estimates for reinvested earnings. Applies accordingly in future. Systematic pattern for services credits and debits -> Detected earlier, refinement of estimates in 2003 and 2007/2008 Results of indicators will change only gradually 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Serviceability (internal consistency) Errors and omissions relative to b.o.p. gross flows (goods, services, di, pi) Date Period Observations reference % of positive EO Average abs (EO) RMSRE(EO)/MSRE(EO)   1st quartile 3rd quartile Bias (%) Variance (%) Apr 08 Jan76-Dec07 All obs. (384) 54.4% 0.138 0.048 0.188 0.182 1.4% 98.6% Last 36 obs. 61.1% 0.071 0.026 0.122 0.087 1.8% 98.2% Last 12 obs. 58.3% 0.076 0.032 0.116 0.089 5.7% 94.3% Obs. 361 to 372 66.7% 0.072 0.022 0.126 0.091 0.1% 99.9% Apr 07 Jan75-Dec06 54.5% 0.141 0.049 0.193 0.185 1.3% 98.7% 0.082 0.031 0.129 0.104 2.4% 97.6% 83.3% 0.067 0.092 0.080 4.8% 95.2% 0.069 0.023 0.099 0.093 1.9% 98.1% Apr 06 Jan74-Dec05 53.9% 0.147 0.054 0.218 0.196 1.1% 98.9% 55.6% 0.030 0.125 1.6% 98.4% 50.0% 0.027 0.128 0.3% 99.7% 0.094 0.047 4.3% 95.8% 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Internal consistency 22.09.2018 Quality assurance of German b.o.p. statistics

2. Quality assessment of German b.o.p. data Internal consistency Impact of cash transactions which are not included ... Shipping of (euro) banknotes Travel expenditure Net EUR 85.57 billion (2004 to 2007) ... for comparison: Cumulated errors and omissions over the same period: EUR 82.43 billion (2004 to 2007) 22.09.2018 Quality assurance of German b.o.p. statistics

3. First results for German i.i.p. data Revision studies for German i.i.p. as of June 2008 I.i.p. Item Period MAPE | MAE (EUR million) % UR Assets Liab.  Assets Liabilities I. MFIs (excl. BBK) Q1 04 – Q4 06 0.009 0.004 17,226 5,885 100.0% 50.0% Direct investment 0.071 0.118 4,662 2,679 75.0% 16.7% Portfolio investment 0.025 0.007 13,029 4,849 83.3% Loans, curr./ deposits 0.000 174 0.0% 91.7% Other assets/liabilities 0.010 0.064 21 59 II. Enterprises, Households 0.031 0.035 58,342 36,741 0.065 0.032 36,727 16,030 58.3% 0.019 0.055 18,609 22,099 25.0% 0.120 0.254 32,096 43,160 Trade credits 0.100 0.140 8,815 8,660 0.026 0.201 844 497 41.7% 22.09.2018 Quality assurance of German b.o.p. statistics

3. First results for German i.i.p. data Revision studies for German i.i.p. as of June 2008, continued I.i.p. Item Period MAPE | MAE (EUR million) % UR Assets %UR Liab.  Assets Liabilities III. General Government Q1 04 – Q4 06 0.001 0.026 27 17,542 83.3% 0.0% participating interest in int. organisations 0.000 15 16.7% Portfolio investment 0.027 17,931 Loans, curr./deposits 0.005 1 112 33.3% Other assets/liabilities 0.017 0.604 11 277 100.0% IV. Bundesbank 22.09.2018 Quality assurance of German b.o.p. statistics

4. Major influencing factors/limitations Incorporation of new information from delayed data sources Methodological changes better conform to int. recommendations (BPM5, TF RoW) Update of int. guidelines (BPM6, BMD4, SNA) Changes in data sources Economic developments (globalisation, ...) Political developments (Intrastat, EMU, SEPA, etc.) Changes in compilation systems + Difficulty of interpretation (ambiguity, trade-offs) + Lack of secondary information for control purposes 22.09.2018 Quality assurance of German b.o.p. statistics

Quality assurance of German b.o.p. statistics 5. Summary outlook Quality reporting important for assessment & usage of data Improves transparency & serviceability Qualitative and quantitative information Check self assessment against external assessment Future focus: Regularly monitor and further enhance quality Enhance documentation Monitor developments esp. in financial account Maintain contacts with users & respondents Changes in guidelines and compilation systems -> substantial & lasting impact on quality indicators 22.09.2018 Quality assurance of German b.o.p. statistics