1. Dominique Ladiray Gian Luigi Mazzi Q2008, Roma, 9-11 July 2008 Assessing the Quality of the Euro-Indicator Database

2. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Outline › The Euro-Indicator Database –Domains, tables and indicators › History of the Database –A few hints on its evolution › Some Quality Indicators –Relevance, Coherence, Timeliness, Completeness, Accuracy, Transparency › Towards a Synthetic Quality Indicator –Measuring the progress –Principal Component Analysis

3. Q2008, Roma Assessing the Quality of the Euro-Indicator Database The Euro-Indicator Database › Objective: to provide a comprehensive and detailed portrait of the short-term economic situation in the UE and its main partners › Important dates –1999: Euro-SICS project –2002: complete redesign of the database –2005: redesign of the External Trade domain –Enlargements of the UE › Today –About 55,000 time series –8 domains (BP, BS, CP, ET, IS, LM, MF and NA) –47 tables (sub-domains) –44 countries and European aggregates

4. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Evolution of the database size since 2000

5. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Timeliness Indicators › T1: Timeliness (announced publication date – end of reference period ) –Target: Small, EU delay target (40 days?) –% of series with timeliness <= 40 days › T2: Database timeliness (date of the first appearance in the database – end of reference period) –Target: Small, EU delay target but Eurostat needs time to control data and compute aggregates (45 days?) –% of series with timeliness <= 45 days

6. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Relevance Indicators › R1: Rate of available statistics over series –Ratio between the number of available indicators and the maximum expected number of indicators (27 MS + European aggregates: moved through time). –Target: 100% › R2: Length of series –% of series of 15 years length or more (to get 2-3 business cycles). –Target: 100%

7. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Accuracy Indicators › A1: Number of revisions –Number of times a value is revised over the last year. › A2: Absolute size of revisions –Average computed on the year following the release. › A3: Relative size of revisions –Average computed on the year following the release. › Note: A indicators require a historical database › Note: lack of references (What is the optimum?)

8. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Accessibility and clarity indicators › AC1: Number of database accesses and downloads –Target? › AC2: Number of series with SDDS metadata files –Target: 100%

9. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Other possible indicators › Number of outliers (and missing values?) in the series –Target: less than 5% (?) › Percentage of SA series with residual seasonality –Target: 0% › Percentage of WDA series with residual TD effect –Target: 0% › Size of the irregular –Target: ?

10. Q2008, Roma Assessing the Quality of the Euro-Indicator Database How to summarize the information? › We can compute these n indicators for each occurrence of the database (about 55,000 series) or for each domain (8) and/or tables (47) › Checking the situation by computing the distance to the target for each domain and/or table –Root Mean Square Deviation –Mean Absolute Deviation –Note: the MAD is “more optimistic” –But none of the 2 indicators is “fair”

11. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Quality of the EuroInd Database

12. Q2008, Roma Assessing the Quality of the Euro-Indicator Database How to summarize the information? › In fact the quality has increased: more information with the same global quality index. –Weighting with the number of “active series”?

13. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Evolution over the last year

14. Q2008, Roma Assessing the Quality of the Euro-Indicator Database How to summarize the information? › Checking the evolution using PCA or MFA: –We can project the 8 domains and/or 47 tables and/or 44 geographical aggregates (centers of gravity) on the first factorial plan

15. Q2008, Roma Assessing the Quality of the Euro-Indicator Database Evolution over the last year BP MF NA IS BS LM CP ET