1. Giuliano Amerini Eurostat
ESSPROS current status and results of the user questionnaire (session 1)
Giuliano Amerini
Eurostat
19/11/2014
ESSPROS Conference

2. Introduction Overview of ESSPROS data collections
User questionnaire: current use of ESSPROS
User questionnaire: future ESSPROS developments
19/11/2014
ESSPROS Conference

3. 1. Overview of ESSPROS data collections
19/11/2014
ESSPROS Conference

4. 1. ESSPROS collections: Scope
European System of Social PROtection Statistics:
Protection of households and individuals against risks or needs:
Sickness/Health care
Disability
Old age
Survivors
Family/Children
Unemployment
Housing
Social exclusion
Social - excludes private insurance
Individual services: excludes collective services
19/11/2014
ESSPROS Conference

5. 1. ESSPROS collections: Data & info
Quantitative
Expenditure of social protection schemes : - social benefits ("gross")
Receipts of social protection schemes: - social contributions, - general government contributions
Number of beneficiaries: pensioners
Net social benefits: adjusted to account for the effects of the fiscal system
Qualitative
Qualitative information: detailed description of schemes and benefits
Quality reports: information on the quality of the data collected
19/11/2014
ESSPROS Conference

6. 1. ESSPROS collections: Data
Expenditure of social protection schemes
by function: sickness/health care, disability, old age, etc.
detailed benefits by function (family/children: birth grants)
by type: social benefits, administration costs, etc.
benefits: cash vs. in kind
benefits: means-tested vs. non means-tested
detailed data by scheme: for 27 countries (out of 33 participating)
Partial
Split residents vs. non-residents - not published
Expenditure by "age" (ex. Survivors) - not published
by scheme: missing 6 countries (EL, ES, PT, SI; CH; TR)
Missing
by sector/ type of schemes
19/11/2014
ESSPROS Conference

7. 1. ESSPROS collections: Data
Receipts of social protection schemes
by type: social contributions (employees, employers, self-employees, etc.), government contributions, etc.
by sector of origin (Corporations, government, households, etc.)
detailed data by scheme: for 27 countries (out of 33 participating)
Partial
by scheme: missing 6 countries (EL, ES, PT, SI; CH; TR)
Missing
by function
19/11/2014
ESSPROS Conference

8. 1. ESSPROS collections: Data
Number of beneficiaries
for pensioners
detailed benefits (14 categories of pensions)
totals without double counting for some main aggregates
by sex
end-year stocks
Partial
Non-resident beneficiaries - not published
by "age" (only survivors pensions) - not published
Missing
Other beneficiaries beyond pensioners: disability, unemployment, social exclusion, etc.
Flows/ average stocks / Full Time Equivalent (FTE)
19/11/2014
Beyond pensioners ESSPROS Conference

9. 1. ESSPROS collections: Data
Net social benefits
Taxes and social contributions paid by recipients on (cash) social benefits ("Net social benefits – restricted approach")
Missing:
"social"- related:
Tax breaks on non-benefit income
Effects of fiscal system on non-beneficary income
Indirect taxation
("Net social benefits – enlarged approach")
19/11/2014
Beyond pensioners ESSPROS Conference

10. 1. ESSPROS collections: legal basis
European Parliament & Council Regulation 458/2007
Commission Regulations: 1322/2007, 10/2008, 110/2011 and 263/2011
Methodological reference:
ESSPROS Manual
19/11/2014
ESSPROS Conference

11. 1. ESSPROS collections: partners
Data providers:
National Statistical Institutes / Ministries of Social Affairs 33 participating countries: - EU-28 - EFTA: IS, NO, CH - missing: LI - CC+PCC: RS, TR - missing: AL, BA, ME, MK, KS
Main users
EC DGs (EMPL, SANCO, ECFIN, TAXUD)
international organisations (OECD, ILO, UNICEF)
Other EU (SPC-ISG, SPC-FEE, SPC-AGE, EPC-AWG)
19/11/2014
ESSPROS Conference

12. 1. ESSPROS collections: Dissemination
Statistical database (Eurobase): quantitative data
Metadata system (ESMS): metadata
Dedicated section: Qualitative information (by country, by scheme), Quality reports (including national quality reports, by scheme), publications, manuals, etc.
Statistics explained website: statistical articles
Notes links to pages are provided within the slide.
19/11/2014
ESSPROS Conference

13. 2. User questionnaire: current use of ESSPROS
19/11/2014
ESSPROS Conference

14. 2. User questionnaire: Overview
Questionnaire issued to better understand:
Why/how data is being used
How it meets user needs
Users’ assessment of the quality
What users would like to see improved
Available online June-September 2014
Disseminated via Eurostat website (main page, dedicated ESSPROS page, data tree),
61 responses - reasonable coverage of key users
 document A (full report) + Annex (questionnaire)
19/11/2014
ESSPROS Conference

15. 2. User questionnaire: Awareness
Figure 1 - Awareness of ESSPROS datasets (% respondents)
Key observations:
Awareness of quantitative datasets (100%) higher than the qualitative products (77%).
Data on social protection expenditure is most well-known (98%) while less than two thirds of users are aware of quality reports (64%).
All observations:
social protection expenditure - 98%
pension beneficiaries - 85%
social protection receipts - 84%
net social protection expenditure 66% - low awareness likely linked to data only being available via Eurobase from 2014 Q2 onwards
qualitative information - 77 %
quality reports - 64 % - low awareness likely linked to national reports not being disseminated prior to May 2013
Only 56% aware of all of the different ESSPROS products.
19/11/2014
ESSPROS Conference

16. 2. User questionnaire: Use
Figure 2 - Frequency of use (% users)
Key observation:
- Respondents were mostly heavy users with more than half (53%) reporting using ESSPROS data more than once a month.
Note:
The relatively high proportion of frequent users (at least once a week) probably reflects:
the targeting of the questionnaire
the greater incentive for such users to show their interest in the future development of ESSPROS
19/11/2014
ESSPROS Conference

17. 2. User questionnaire: Use
Figure 3 - Products used to access ESSPROS data and associated information
(% users)
Key observations:
Data most often accessed via Eurobase (over 90% of users).
Only a third of users have used the accompanying metadata (ESMS) which provides important information on the data disseminated.
All observations:
Eurostat’s statistical database – 93%
ESMS metadata system – 36%
Publications - 64%
Statistics explained - 48%
20% of users have used all the Eurostat products mentioned.
Notes:
- Eurostat also releases some ESSPROS data via CIRCABC but this tends to have much less visibility as the primary intention of this system is not public dissemination.
- ESSPROS data is also shared with other organisations which disseminate the data through their own statistical databases (e.g. OECD) and users are free to use this data in their own publications. 21% of users had used other (non-Eurostat) products to access ESSPROS data and associated information.
19/11/2014
ESSPROS Conference

18. 2. User questionnaire: Use
Figure 4 - Use and awareness of ESSPROS datasets (% users, % respondents)
Key observations:
Most users reported using quantitative datasets (98%), fewer use the qualitative products (64%)
Social protection expenditure most widely used (91%)
Use generally reflected levels of awareness
All observations (use only):
quantitative datasets - 98%
social protection expenditure - 91%
pension beneficiaries - 73%
social protection receipts - 71%
net social protection expenditure - 39%
qualitative information - 63%
quality reports - 38%
30% use all the different sets of qualitative and quantitative data associated with ESSPROS.
19/11/2014
ESSPROS Conference

19. 2. User questionnaire: Use
Main conclusions:
Low use of METADATA !!
Low use of qualitative information and quality reports
19/11/2014
ESSPROS Conference

20. 2. User questionnaire: Importance and purpose of use
Figure 5 - Importance of ESSPROS data for user’s work (% users)
Key observation:
- More than three quarters of respondents indicated that ESSPROS data are very important or important for their work.
19/11/2014
ESSPROS Conference

21. 2. User questionnaire: Importance and purpose of use
Figure 6 - Use of ESSPROS data (% users)
Key observations:
Data most often used for quantitative analysis (93%) and in particular for time-series analysis (66%), cross-sectional analysis (59%) and comparison with other sources of data (59%).  importance of ensuring a consistent application of concepts between countries and over time.
Data widely used for qualitative analysis (80% of users) often for background information or policy analysis (66% and 63% respectively).
Other:
- Data widely used in research and reports for public or internal use (73% of users)  underlining need for highest quality data possible.
19/11/2014
ESSPROS Conference

22. 2. User questionnaire: Quality
Figure 7 - Average rating of how ESSPROS fulfils ESS criteria
Key observations
- 90% of users consider the fulfilment of each of the ESS criteria as adequate, good or very good.
- However, timeliness/punctuality and to a lesser extent comparability received lower than average ratings suggesting that these are areas where users feel there is the most room for improvement.
Note: A rating of 1 to 5 is allocated to responses reporting that the fulfilment of a given criterion was very poor, poor, adequate, good, very good respectively. The average rating is an average of these across all respondents which gave an opinion. The colour coding gives an indication of the contributions of different responses by evaluation to the total score.
Relevance refers to the extent to which the statistical data satisfy the needs of the users.
Accessibility and Clarity: Accessibility refers to the physical conditions under which users can obtain the statistical data. Clarity refers to the availability of appropriate documentation linked to the statistical data and to the additional assistance which producers supply to users.
Timeliness and Punctuality: Timeliness of statistical data is the length of time between their availability and the moment at which the phenomena they describe occurred. Punctuality refers to any time lag between the release and the target date by which the data should have been delivered.
Coherence aims to measure the reliability of the statistical data if combined with other statistics in different ways and for other uses.
Comparability tries to measure the effect of the differences in the applied statistical concepts and measurement procedures when the statistical data are compared between geographic areas, over time or between different domains.
Accuracy refers to the closeness of the statistical data to the unknown true or exact value of the measured phenomena.
19/11/2014
ESSPROS Conference

23. 2. User questionnaire: Quality
Main conclusions (room for improvements):
Timeliness (Oct N+2 too late; main aggregates earlier?)
Comparability: more detailed qualitative information
Coherence: to improve links with other statistics
19/11/2014
ESSPROS Conference

24. 3. User questionnaire: future ESSPROS developments
19/11/2014
ESSPROS Conference

25. 3. User questionnaire: Predefined developments
Respondents interested
Specific uses identified by respondents
Net benefits following the enlarged approach
56%
Improved view of social protection
Improved cross-country comparison
Source of data on fiscal benefits
Dissemination of data by scheme
59%
More detailed picture of social protection systems
Analysis of individual schemes
Analysis of groups of schemes
Dissemination of provisional estimates to improve timeliness
For publications and reports
Note: most interest in having provisional aggregates for social protection expenditure.
19/11/2014
ESSPROS Conference

26. 3. User questionnaire: Other additional data needs
Other data thought to be useful include:
Additional breakdowns of existing data:
Expenditure and receipts data broken down by sector
Data for survivors’ benefits broken down according to the retirement age
New data:
More detailed qualitative information
Data on expenditure on long term care
Information on links with other sources of data (i.e. National accounts, COFOG, SHA, LMP)
Data on social services ("collective services")
Data on beneficiaries of non-pension benefits, in particular unemployment benefits (both full and partial) and social assistance/minimum income benefits.
Note: Among the responses there were a few cases where the data identified:
would become available through one of the predefined developments mentioned previously:
i.e. data on the financing of benefits by function, further detailed data for housing benefits
are, to some extent, already available:
i.e. data on average pension expenditure, detailed information about the benefits included in the data
19/11/2014
ESSPROS Conference

27. 3. User questionnaire: Use of other sources
77% use other sources to complement, or as an alternative to, ESSPROS data:
National data are most often used
International data most commonly used are those with the most overlap in scope with ESSPROS:
DG EMPL’s Mutual Information System on Social Protection (MISSOC) – 41%
OECD’s Social expenditure database (SOCX) – 38%
National accounts – 34%
Main reasons for use:
breakdowns available
level of detail (qualitative information)
scope of the data
Underlines potential gains of clarifying the links with other sources
19/11/2014
ESSPROS Conference

28. 3. User questionnaire: Potential developments (summary)
See as a reference document B: "list of possible future ESSPROS developments" How to read the following tables: XX= developments mentioned in the questionnaire (bias) X= developments mentioned by respondents X = Inferred developments
19/11/2014
ESSPROS Conference

29. 3. Mentioned potential developments
Identified in UQ 1
Breakdown of data on pensions by age x 5
Detailed qualitative information 7
Net enlarged module xx 8
Other beneficiaries 9
Collective services 12
Improve the use of existing content 13
Improve clarity of dissemination framework 14
Improve dissemination of data by scheme 15
Dissemination of data by groups of schemes x  16
Improve timeliness 18
Breakdowns by sector 20
Links between ESSPROS and other sources - National accounts (COFOG) - SHA including Long term care (LTC) - LMP - MISSOC - SOCX
x x x x x x
Crosses in RED: Desired development inferred from use of data (i.e. development would facilitate the analysis using ESSPROS data presented).
Crosses in BLACK: Desired development specifically mentioned in presentation.
EMPL DG Employment, Social Affairs & Inclusion presentation
ECFIN DG Economic and Financial Affairs presentation
SPC Social Protection Committee presentation
UQ User questionnaire results
19/11/2014
ESSPROS Conference

30. 3. Potential developments non-mentioned
Identified in UQ 2
Breakdown of data by country of residence 3
New aggregates of pension beneficiaries 4
Other observations of pension beneficiaries 6
Geographical coverage 10
Private insurance (third pillar) benefits 11
Sub-annual data on beneficiaries 17
Classification of pension schemes 19
Clarification of borderlines
Crosses in RED: Desired development inferred from use of data (i.e. development would facilitate the analysis using ESSPROS data presented).
Crosses in BLACK: Desired development specifically mentioned in presentation.
EMPL DG Employment, Social Affairs & Inclusion presentation
ECFIN DG Economic and Financial Affairs presentation
SPC Social Protection Committee presentation
UQ User questionnaire results
19/11/2014
ESSPROS Conference

31. Thank you Comments are welcome
19/11/2014
ESSPROS Conference

32. 3. Potential developments (summary)
Identified in UQ 1
Breakdown of data on pensions by age x 2
Breakdown of data by country of residence 3
New aggregates of pension beneficiaries 4
Other observations of pension beneficiaries 5
Detailed qualitative information 6
Geographical coverage 7
Net enlarged module xx 8
Other beneficiaries 9
Collective services 10
Private insurance (third pillar) benefits 11
Sub-annual data on beneficiaries 12
Improve the use of existing content 13
Improve clarity of dissemination framework 14
Improve dissemination of data by scheme 15
Dissemination of data by groups of schemes x  16
Improve timeliness 17
Classification of pension schemes 18
Breakdowns by sector 19
Clarification of borderlines 20
Links between ESSPROS and other sources - National accounts (COFOG) - SHA including Long term care (LTC) - LMP - MISSOC - SOCX
x x x x x x
Crosses in RED: Desired development inferred from use of data (i.e. development would facilitate the analysis using ESSPROS data presented).
Crosses in BLACK: Desired development specifically mentioned in presentation.
EMPL DG Employment, Social Affairs & Inclusion presentation
ECFIN DG Economic and Financial Affairs presentation
SPC Social Protection Committee presentation
UQ User questionnaire results
19/11/2014
ESSPROS Conference