1. Survey on representative sampling
ESF Support Centre
Alphametrics Ltd. & Applica Sprl.
Brussels, 13 November 2015

2. Background Scope Objectives 2 common output indicators
ESF longer-term result indicators
YEI longer-term result indicators
Objectives
To raise awareness on the requirements
To assess the state of preparedness
To identify potential issues

3. The survey Launching: beg-July, ongoing ( end-Nov)
Combination of open/multiple choice questions
54 MAs contacted, 24 replies received, 21 respondents
9 final
12 clarification pending
3 will not participate

4. Use and coverage of surveys
Use of surveys to collect data
In the majority of cases (16), data based on surveys for some indicators (especially CRI7 on improved labour market situation)  Other indicators: either based on registers or data collected on entry (homeless/from rural areas)
In 3 cases surveys planned for all indicators
In 1 case, no surveys planned (all data from administrative registers)
1 MA still to be decided
Coverage of surveys
Surveys based only on representative samples, in half of the cases (10)
Surveys based on all participants, in a few cases (4)
Surveys based on both, in a few cases (5)

5. Data collection approach for each indicator
Survey - full population
Survey - samples
Survey
(mix)
Registers
On entry
To be decided
No reply -
N/a
Total respondent MAs
Common output indicators
COI18 2 1 3 9 5 21
COI19 7
Common longer-term result indicators
CRI6 4 11 -
CRI7 6 12
CRI8
CRI9
YEI common longer-term result indicators
YEI RI10 8
YEI RI11 10
YEI RI12

6. Survey and sampling design
Authority responsible for drawing the samples
Mainly independent expert institution
Also IB and NSI
Sampling method: very little information at this stage
Margin of error
In 2 cases <=2 pp, in 2 cases 2-5 pp
Still to be decided in many cases
Coverage errors: very little information at this stage
(…)

7. Survey and sampling design
(…)
Non-sampling errors
Methods used to minimise
Ensuring clear formulation of questions
Questionnaires adapted to target group
Preparation of interviewers
Testing of questionnaires
On spot verifications
Definitions and translations of indicators available for participants
In many cases, determined by the external contractor/institution drawing the samples

8. Data collection Authority responsible for data collection
External body in many cases
Also NSI, MA, IB, and beneficiaries
Method for data collection
Phone (1), (1), online forms (3)
Combination of methods (4)
Still to be decided in many cases
(…)

9. Data collection (…) Data collection schedule Minimum requirements
Minimum requirements
Annual
More frequent
On entry
Other
To be decided
No reply
N/A
Total respondent MAs
COI18 1 2 8 6 3 21
COI19 4
CRI 5 -
YEI RI 9

10. Data storage Storage of LTRI data
In the original database with all the micro-data in many cases (9)
In some cases in a separate database (3)
MA only store aggregated data (4)
Depending on the data (2)
Still to be decided (3)
Ensuring the correct reference population (including exit dates of participants) and data breakdown at the level of IP
Requirements for external organisations laid down in the specifications
Full set of micro-data transferred to organisation doing the sampling
Linking databases by using personal IDs

11. Main difficulties
Low response-rate, particularly for hard to reach participants
Cost of surveys
Precision requirements published in June 2015 not applied in surveys for YEI which had already been launched
Timeliness of surveys, possible outdated contact details
Problems to identify and contact homeless participants
Rural areas: inaccurate or incomplete postcodes, no correspondence between DEGURBA and national register for postcodes

12. Budget
Very little information about the planned budget (only 4 replies)
Evaluation
Surveys will be used to collect information for (counterfactual) evaluation purposes by 7 MAs. Not used by 3 MAs.

13. Conclusions No comprehensive picture
Few responses
Outsourcing of tasks and arrangements still ongoing in many cases
No misinterpretation of requirements or deviation from guidance was identified (3 cases pending clarification)
A mix of approaches, no predominant patterns (except CRI07)
 Participation is encouraged!

14. ESF data support centre
Advice/guidance available only until December 2015!
Methodological support on monitoring information systems
Clarification regarding data collection, recording, validation and storage
Guidance regarding transmission of structured data to the EC using SFC2014 (technical queries > SFC support team)
9.00 a.m. – 5.00 p.m. Monday-Friday
Phone: