Income distribution flash estimates- progress report

Flash estimate is a rapid estimate: Project Objective Produce flash estimates (FE) for key income-related indicators, as a component of the ESS strategy for providing more timely data on income Eurostat definition: Rapid estimate, also called early estimate, refers to a timely numerical evaluation (i.e. an estimate) of an economic variable of interest, using the real-time data ‡flow, in such a way that an early picture of the present, the recent past or the near future can be formed. Flash estimate is a rapid estimate: produced by the statistical institution in charge of the regular estimate based on an incomplete set of information (hard and soft data) Use of several statistical techniques (including non-traditional methods such as modelling)) Specificities Estimate the income distribution Statistical uncertainty of the target indicators (sample) Stability

 2015 results are purely illustrative, NOT published Cycle 2016 First cycle FE 2015 at Eurostat level for the 28 countries Develop the FE methodology: microsimulation and macro-level models Quality Assessment framework (QAF) Communication strategy Produce 2015 FE  2015 results are purely illustrative, NOT published Dedicated TF on FE: 2 meetings (30th Jun and 25th Oct) Meeting with main users National "work in progress" Grants on flash and current income Publication FE FR and UK: national FE based on microsimulation Euromod: nowcasts for 2015 Old round: SK, LT, SI, El ongoing new round NL, EL, SE, UK, LU from grants a new round of grants on timeliness including FE started Nowcasts are published by a different statistical organization

Alternative to microsim/ macro level models Methodological approaches Microsimulation Alternative to microsim/ macro level models Basic models (as benchmarks) Current Income – TBD National estimates No one single method(approach) that works for all countries There is no single way of producing the flash estimates, best model depends on country specificities, related to a variety of factors like demographics, labor market structure, the consistency of Euromod with SILC and its timeliness, availability of data For AROP: MS: AT, BE, BG, DK, EL, ES, FI, FR*, HR, HU, IT, LV, MT, NL, PL, PT, SE, UK* Alternative to microsimulation/macro level models: CY, CZ, DE, EE, IE, LT, LU, RO, SI, SK, The decision on what selection algorithm to use is taken on a country-by-country basis MS: 18 Alternative: 10

Quality Assurance: input sources, estimation process QAF The general principles presented and revised with stakeholders and WG SILC in line with the five dimensions of the code of practice full transparence- a complete methodological report Two pillars Quality Assurance: input sources, estimation process Quality Assessment: ! Accuracy (empirical performance) but also relevance for users, accessibility and clarity =>" a credibility toolbox" Aimed at supporting decision regarding Consideration/selection of models developed inside and outside Eurostat Publication of FE The framework has been established so that methodologies and estimates developed either within ESS (by Eurostat or NSIs) or outside ESS (e.g., Euromod by ISER), can be assessed based on common quality criteria. Quality Assurance: focus on input and process Quality Assessment: focus on output Credibility toolbox: other more qualitative criteria need to be considered, expert judgment of the models, impact of policies in microsimulation , PL said in the survey that different models are different philosophies and we need to go beyond empirical performance only In the methodology for rapid estimates they mention; model should be interpretable (explicable and understandable from the economic point of view), specified correctly, be stable and give good forecasts

Communication: full disclosure

2) Magnitude-Direction (MD) scale 3)Significance-Direction (SD) scale Communication: only change classes 2) Magnitude-Direction (MD) scale [+++] major increase [+] moderate increase [Ø] (quasi) stable / minor changes [-] moderate decrease [---] major decrease 3)Significance-Direction (SD) scale Significant increase [+] Non-significant change [Ø] Significant decrease [-] I will first present the SD and MD scales, which are the basis for the basic, categorial performance metrics; the general idea of the two scales is to assign the YoY change in the indicator to a broad but informative class. MD scale is a more detailed version of SD, where significant changes are split into magnitude classes. *** Absolute, pre-defined, general thresholds (i.e., not expressed as multiples of the country-specific standard deviation) could be used instead for defining the change classes.

Communication using MD scale Colour coding the magnitude of the YoY change (expressed as multiples of country specific standard deviatio-SD) Not available due to missing input data or low quality As mentioned before, our choice for delivering the results are the SD / MD scales. These are two examples, presented in a previous meeting, of presenting the results using the MD scale. In the table on the right, the MD scale is supplemented with the probabilities of each change class. Given the delivery format (qualitative change classes, instead of quantitative PE), we shall assess the output of our work based on the appropriate metrics, i.e., proportion of classification errors.

Communication using MD scale- across years

Further results (1)

Further results (2) Poland: there appears to be some dynamic in the distribution (the inter-quintile range widening), but it is not captured by either AROP or QSR, who register no significant YoY variation Czech Rep: not much happens with the deciles, but there is movement in AROP Romania: much of the dynamics appears similar to Poland, but variation in AROP is more pronounced QSR appears as basically inert. It only detects a significant change in RO in 2013-14, but 2014 doesn't look particularly special when looking at the deciles: the same widening gap due to poorest people falling behind. These 3 countries illustrate the fact that focusing on very synthetic indicators like AROP & QSR, and especially on their YoY variation, without a longer-term overview, misses a lot of the dynamics going on "under the surface", which in our case is better captured by the evolution of the quintiles. The solution might not lay in developing new, improved indicators, but visual tools (illustrated here) for insight generation and facilitation of policy- and decision-making. 

% countries in each SD class (2014): OBS vs. EST Performance assessment using SD scale (1) % countries in each SD class (2014): OBS vs. EST How do the results appear when we look at them through the SD lens? This is not a full ex-post assessment, just an overview of the results, focused on classification errors.

Continue development of the FE methodology Cycle 2017 Continue development of the FE methodology Consolidate the QAF: establish criteria for quality and publication Validation process with TF and ILC WG Enlarged TF March 2017: experts from university and other international institutions Consultation users Publish 2016 FE as "Experimental Data"  provided that the quality assessment is favorable Deadline: September 2017 April-May 2017: LFS and Quarterly National Accounts June 2017: Preliminary Euromod 2015 files available =>August 2017: Final Euromod release Summer 2017: SILC 2016 available for most countries ***

Securing access to data Role of the ILC WG (1) Securing access to data Eurostat for national data when used for Euromod ISER for the UDB 2015 Validation methodology Early in the process: ongoing survey with ILC WG Best practices with the TF Participatory approach and bilateral contacts for improving estimation National specific solutions possible To be discussed: ILC WG June 2017

Aug-Sep 2017: Consultation of the WG on the final figures Role of the ILC WG (2) Validation results FE 2016 Aug-Sep 2017: Consultation of the WG on the final figures Full information will be available for Member States Overall methodological report with FE 2016 including Quality assessment of the methodologies proposed Communication plan To be disseminated: September 2017  provided that the quality assessment is favorable

Timeline 2017

take note of the progress of the project on flash estimates, Questions to the DSS take note of the progress of the project on flash estimates, provide views on the methodology for developing flash estimates on income inequalities and poverty, and on how the results should be published, express their opinion in particular on: the two-step validation approach with the Member States put in place for the cycle 2017; the publication of the FE 2016 as "experimental data", provided that the quality is considered acceptable.