1. European Conference on Quality in Official Statistics
Quality Assurance of German Balance of Payments Statistics update -
Sabine Guschwa*
European Conference on Quality in Official Statistics
Rome, 8-11 July 2008
* The opinions expressed are those of the author and do not necessarily reflect the views of the Deutsche Bundesbank.

2. Quality assurance of German b.o.p. statistics
Outline
1. Introduction
2. Quality assessment of German b.o.p. data 2008
3. First results for German i.i.p. data
4. Major influencing factors/limitations
5. Summary outlook
Quality assurance of German b.o.p. statistics

3. Quality assurance of German b.o.p. statistics
Introduction
Aim of quality reporting
=> Inform users through:
Summary information on current work
Further analysis of data
Information on usage of data
Communication of measures, plans etc.
Complementing existing information
Quality assurance of German b.o.p. statistics

4. Quality assurance of German b.o.p. statistics
Introduction
Development/Short history:
IMF Data Quality Assessment Framework (DQAF) (2003)
Joint ECB/Eurostat TF on Quality (TF QA) (2003 – 2005)
(Follow-up) Work on quality issues (ECB, Eurostat, DE)
IMF ROSC Mission in Germany (2005)
Available b.o.p. quality assessments/reports:
ECB “EU B.o.p./I.i.p. Statistical Methods” (B.o.p. Book)
IMF SDDS Metadata
IMF “Germany: ROSC Report – Data Module, Response by the Authorities, and Detailed Assessment Using the DQAF” (Jan. 2006)
ECB “Euro area b.o.p. and i.i.p. statistics, Annual quality report 2008”, Guideline ECB/2004/15 of 16 July 2004, ECB/2007/3
Quality assessments of data transmitted to Eurostat. Commission/Eurostat quality assessment according to EC Regulation 184/2005 of 12 January 2005, EC 602/2006
Outline of a quality report for German b.o.p. in 2006, updated in 2007 and 2008
Quality assurance of German b.o.p. statistics

5. Quality assurance of German b.o.p. statistics
1. Introduction
Quality assessments/reports
=> follow internationally recommended
quality dimensions defined by the IMF DQAF
(ECB, Eurostat, DE):
Prerequisites of quality
Assurances of integrity
Methodological soundness
Accuracy and reliability
Serviceability
Accessibility
Combining qualitative and quantitative information.
Quality assurance of German b.o.p. statistics

6. 2. Quality assessment of German b.o.p. data
Methodological soundness
Chronology of changes in German b.o.p. for (excerpt)
Date
Changes to the balance of payments
Period
March 2008
Methodological changes:
netted disclosure of tax revenue & refunds (as negative tax revenue)
transfer of settlement contributions to the EU budget from VAT-based to GNI-based resources
desulphurisation of natural gas recorded as processing
change in recording of warrants with automatic exercise feature under debt securities (from financial derivatives)
accruals recording of transfers from EU budget (EU agr. fund)
1981 to date
2007 to date
1977 to date
Feb 2007 to date
March 2007
Methodological changes cover:
recording of transfers from EU budget (EU agricultural fund) at the time when the claim arises (accruals basis)
monthly allocation of expenditure on travel defined more precisely, further refinement of factors applied to package holidays
2006 to date
2006/2002 to date
Quality assurance of German b.o.p. statistics

7. 2. Quality assessment of German b.o.p. data
Accuracy and reliability
Quantitative indicators on revision studies (TF QA):
-> 2008 update for b.o.p. data (first results in 2004)
-> 2008 first results for quarterly i.i.p. stock data!!
-> for b.o.p. current account gross flows, i.i.p. stock data:
Mean absolute percentage error / revision (MAPE)
Percentage of upward revisions (% UR)
Directional reliability (Q (%))
-> for b.o.p. net flows:
Mean absolute relative error / revision (MARE)
Root mean square relative error / revision (RMSRE, bias, regression, unsystematic component)
% UR
Q (%)
Quality assurance of German b.o.p. statistics

8. 2. Quality assessment of German b.o.p. data
Accuracy and reliability
Revision studies (size, direction, reliability of trends)
Revision studies for German b.o.p. as of April 2008
B.o.p. Item
Period
MAPE
% Upward revision
Q (%)
3rd quartile
90th percentile
Max
Current account credits
Jan04 - Dec07
0.038
0.055
0.064
0.119
93.8%
87.2%
Current account debits
0.022
0.034
0.046
0.093
68.8%
89.4%
Goods credits (exports)
0.005
0.006
0.011
0.014
60.4%
95.7%
Goods debits (imports)
0.007
0.026
47.9%
93.6%
Services credits
0.107
0.139
0.170
0.231
100%
91.5 %
Services debits
0.043
0.062
0.082
0.169
83.3%
91.5%
Income credits
0.236
0.408
0.483
0.707
95.8%
66.0%
Income debits
0.156
0.230
0.399
0.574
50.0%
Current transfers credits
0.293
0.166
0.829
3.334
37.5%
83.0%
Current transfers debits
0.035
0.052
0.091
0.142
52.1%
85.1%
Quality assurance of German b.o.p. statistics

9. 2. Quality assessment of German b.o.p. data
Revision studies for German b.o.p. as of April 2008, contd.
B.o.p. Item
Period
MARE
RMSRE / MSRE
% Upward revision
Q (%)
3rd quartile
90th percentile
Max
Mean comp. (%)
Regress comp. (%)
Unsystcomp. (%)
Capital acc. - net
Jan04-Dec07
0.393
0.092
0.500
10.667
1.050
4.5%
18.8%
76.7%
43.8%
93.6%
Direct inv. – net
0.542
0.718
1.191
2.085
0.520
12.6%
0.0%
87.4%
27.1%
89.4%
Direct inv. abr.
0.552
0.656
0.941
1.866
0.506
57.0%
0.3%
42.7%
10.4%
87.2%
Direct inv. in G.
0.502
0.638
1.108
2.049
0.455
27.4%
2.2%
70.5%
75.0%
Portf. inv. – net
0.269
0.312
0.465
1.296
0.282
0.5%
0.7%
98.8%
54.2%
Portf. inv.assets
0.192
0.277
0.354
0.644
0.190
0.4%
4.6%
95.0%
41.7%
95.7%
Portf. inv. liab.
0.281
0.375
0.523
1.513
0.309
0.9%
1.4%
97.7%
Other inv. – net
0.145
0.187
0.693
0.141
9.9%
89.7%
66.7%
100%
Other inv. assets
0.066
0.095
0.151
0.257
0.067
13.6%
1.5%
84.9%
39.6%
Other inv. liab.
0.075
0.098
0.168
0.303
0.079
40.3%
0.1%
59.6%
79.2%
97.6%
Quality assurance of German b.o.p. statistics

10. 2. Quality assessment of German b.o.p. data
Revision studies on current transfers credits
Date
Reference
period
MAPE
% Upward revisions
Q (%)
3rd quartile
90th percentile
Max
Apr 08
Jan04 - Dec07
0.293
0.166
0.829
3.334
37.5%
83.0%
Apr 07
Jan03 - Dec06
0.165
0.125
0.205
3.363
54.2%
80.9%
Apr 06
Jan02 - Dec05
0.085
0.115
0.173
0.408
39.6%
85.1%
Quality assurance of German b.o.p. statistics

11. 2. Quality assessment of German b.o.p. data
Main results from 2008 b.o.p. revision studies:
Comparatively large MAPE values for current transfers credits influenced by methodological change (accruals for EU transfers)
Comparatively large MAPE/MARE for income credits and debits / direct investment abroad and in Germany, combined with systematic pattern of revisions, (comp. lower Q(%))
-> Supported by earlier years´ results
Due to inclusion of new data from direct investment survey replacing and/or correcting preliminary estimates for reinvested earnings. Applies accordingly in future.
Systematic pattern for services credits and debits
-> Detected earlier, refinement of estimates in and 2007/2008
Results of indicators will change only gradually
Quality assurance of German b.o.p. statistics

12. 2. Quality assessment of German b.o.p. data
Serviceability (internal consistency)
Errors and omissions relative to b.o.p. gross flows (goods, services, di, pi)
Date
Period
Observations reference
% of positive EO
Average abs (EO)
RMSRE(EO)/MSRE(EO)
1st quartile
3rd quartile
Bias (%)
Variance (%)
Apr 08
Jan76-Dec07
All obs. (384)
54.4%
0.138
0.048
0.188
0.182
1.4%
98.6%
Last 36 obs.
61.1%
0.071
0.026
0.122
0.087
1.8%
98.2%
Last 12 obs.
58.3%
0.076
0.032
0.116
0.089
5.7%
94.3%
Obs. 361 to 372
66.7%
0.072
0.022
0.126
0.091
0.1%
99.9%
Apr 07
Jan75-Dec06
54.5%
0.141
0.049
0.193
0.185
1.3%
98.7%
0.082
0.031
0.129
0.104
2.4%
97.6%
83.3%
0.067
0.092
0.080
4.8%
95.2%
0.069
0.023
0.099
0.093
1.9%
98.1%
Apr 06
Jan74-Dec05
53.9%
0.147
0.054
0.218
0.196
1.1%
98.9%
55.6%
0.030
0.125
1.6%
98.4%
50.0%
0.027
0.128
0.3%
99.7%
0.094
0.047
4.3%
95.8%
Quality assurance of German b.o.p. statistics

13. 2. Quality assessment of German b.o.p. data
Internal consistency
Quality assurance of German b.o.p. statistics

14. 2. Quality assessment of German b.o.p. data
Internal consistency
Impact of cash transactions which are not included ...
Shipping of (euro) banknotes
Travel expenditure
Net EUR billion (2004 to 2007)
... for comparison:
Cumulated errors and omissions over the same period: EUR billion (2004 to 2007)
Quality assurance of German b.o.p. statistics

15. 3. First results for German i.i.p. data
Revision studies for German i.i.p. as of June 2008
I.i.p. Item
Period
MAPE | MAE (EUR million)
% UR
Assets
Liab.
 Assets
Liabilities
I. MFIs (excl. BBK)
Q1 04 – Q4 06
0.009
0.004
17,226
5,885
100.0%
50.0%
Direct investment
0.071
0.118
4,662
2,679
75.0%
16.7%
Portfolio investment
0.025
0.007
13,029
4,849
83.3%
Loans, curr./ deposits
0.000
174
0.0%
91.7%
Other assets/liabilities
0.010
0.064 21 59
II. Enterprises, Households
0.031
0.035
58,342
36,741
0.065
0.032
36,727
16,030
58.3%
0.019
0.055
18,609
22,099
25.0%
0.120
0.254
32,096
43,160
Trade credits
0.100
0.140
8,815
8,660
0.026
0.201
844
497
41.7%
Quality assurance of German b.o.p. statistics

16. 3. First results for German i.i.p. data
Revision studies for German i.i.p. as of June 2008, continued
I.i.p. Item
Period
MAPE | MAE (EUR million)
% UR
Assets
%UR
Liab.
 Assets
Liabilities
III. General Government
Q1 04 – Q4 06
0.001
0.026 27
17,542
83.3%
0.0%
participating interest in int. organisations
0.000 15
16.7%
Portfolio investment
0.027
17,931
Loans, curr./deposits
0.005 1
112
33.3%
Other assets/liabilities
0.017
0.604 11
277
100.0%
IV. Bundesbank
Quality assurance of German b.o.p. statistics

17. 4. Major influencing factors/limitations
Incorporation of new information from delayed data sources
Methodological changes
better conform to int. recommendations (BPM5, TF RoW)
Update of int. guidelines (BPM6, BMD4, SNA)
Changes in data sources
Economic developments (globalisation, ...)
Political developments (Intrastat, EMU, SEPA, etc.)
Changes in compilation systems
+ Difficulty of interpretation (ambiguity, trade-offs)
+ Lack of secondary information for control purposes
Quality assurance of German b.o.p. statistics

18. Quality assurance of German b.o.p. statistics
5. Summary outlook
Quality reporting important for assessment & usage of data
Improves transparency & serviceability
Qualitative and quantitative information
Check self assessment against external assessment
Future focus:
Regularly monitor and further enhance quality
Enhance documentation
Monitor developments esp. in financial account
Maintain contacts with users & respondents
Changes in guidelines and compilation systems
-> substantial & lasting impact on quality indicators
Quality assurance of German b.o.p. statistics