1. Data requirements and summary statistics for revisions analysis Performing revisions analysis for sub-annual economic statistics Michela Gamba, Statistics Directorate - OECD OECD Short-Term Economic Statistics Working Party ______________________________ Paris 23-24 June 2008

2. Scope of revisions analysis WHY USEFUL? –Allow users and providers to better understand quality of published sub-annual statistics and to study magnitude of revisions WHAT IS NEEDED? –A Real-time database to perform Revision analysis –A dataset containing a separate time series for the variable as appeared in each official past release 2

3. OECD input to Task Force on revisions analysis 1. DATA & METADATA KEY ELEMENTS »Guidelines for data and metadata requirements 2. INVENTORY OF DATA SOURCES »List of existing sources available publicly 3. SUMMARY STATISTICS »Review summary statistics used »Include relevant statistics- how much the “story has changed” »Selected statistics to be included in the toolkit »Provide description and purpose of statistics 3

4. DATA & METADATA KEY ELEMENTS 8 requirements to build a RT database 1 COUNTRY/REGION DESCRIPTION -> identify country/region/ zone aggregate 2 VARIABLE DESCRIPTION ->Full description of variable (classification system, value/volume, ref years) with links to current revision policy associated requires well detailed METADATA 3 VARIABLE MEASURES -> gross/ SA/ trend/ provided as published (index or level) + GR 4

5. DATA & METADATA KEY ELEMENTS 8 requirements to build a RT database (cont.) 4 IDENTIFICATION OF VINTAGE -> data and metadata snapshot for each variable, for each official release -> Field providing the date of release (one new data point for each vintage, forming a triangle) 5 LENGTH OF VINTAGE -> going back as far as possible… 5

6. DATA & METADATA KEY ELEMENTS 8 requirements to build a RT database (cont.) 6 LENGTH OF TIME SERIES -> ideally the full time series available for each official release..general rule of at least one year before the starting point of vintages to allow y_o_y growth rates 7 ONGOING UPDATING -> store all new vintages with regular achieving 8 DATA ACCESSING -> different formats available but..it is important to be able to extract data in the right format ( i.e. triangle table) 6

7. Summary statistics 3 levels of analysis 1.BASIC/ CORE MEASURES Targeting users that require quick, easy to understand information 2.ADDITIONAL/ ADVANCED MEASURES Targeting users that require more in-depth analysis 3.SOPHISTICATED / SPECIAL USER MEASURES Information for detailed research purposes – references to formula only 7

8. ummary statistics (cont.) Summary statistics (cont.) Default dataset information Period of analysis reference data points of the time series being analysed (e.g. from 1994Q1 – 2007Q3) Revision Interval L_P revision interval being analysed for the vintages of the sampled time points n number of observations 8

9. ummary statistics (cont.) Drop-down boxes Summary statistics (cont.) Drop-down boxes 1.USUAL SIZE AND RANGE OF REVISIONS Basic measures Mean absolute revision Range that 90% of revisions lie within Advanced/additional measures Median absolute revision 2.ASSESSMENT OF POSSIBLE DIRECTIONAL TENDENCY IN REVISIONS Basic measures Mean Statistical significance of the mean Advanced/additional measures Median % positive/ negative/ zero revisions HAC standard deviation Adjusted t-stat of mean revision Critical value of t-stat for significance of mean 9

10. ummary statistics (cont.) Drop-down boxes Summary statistics (cont.) Drop-down boxes 3.VARIABILTY OF REVISIONS Basic measures Standard deviation Advanced/additional measures Root mean square revision Quartile deviation Min/ max/ range Sophisticated/ special user measures Skewness 4.IMPACT OF REVISIONS ON SIGN OF GROWTH RATES Basic measures % sign(later)= % sign(earlier) Advanced/additional measures Acceleration/ deceleration test 10

11. ummary statistics (cont.) Drop-down boxes Summary statistics (cont.) Drop-down boxes 5.EFFICIENCY ASSESSMENT Advanced/additional measures Correlation between revision and earlier/ later estimate Test if revisions are noise/ news Sophisticated/ special user measures Mean squared revision Mean squared revision Decomposition UM Mean squared revision Decomposition UR Mean squared revision Decomposition UD 11

12. The tool kit…