Representative sampling Overview of the questions received by the ESF Data Support Centre Alphametrics Ltd. & Applica Sprl. Brussels, 13 March 2015.

Slides:



Advertisements
Similar presentations
Monitoring and Indicators in WORKSHOP 30 APRIL 2013 Ines Hartwig Impact Assessment & Evaluation Unit DG EMPL 1.
Advertisements

1 W ORKSHOP ON S TRATEGIC P ROGRAMMING, M ONITORING AND EVALUATION F OCUSING ON P ERFORMANCE AND RE SULTS Madrid, 22 February 2013 Ines Hartwig DG Employment,
Estimating net impacts of the European Social Fund in England Paul Ainsworth Department for Work and Pensions July 2011
The Youth Employment Initiative
Guidance on Evaluation of Youth Employment Initiative
Draft model for the Annual and Final implementation report under the Investment for Growth and Jobs goal Marko Prijatelj Directorate General for Regional.
ESF - Monitoring: Q&A Andy Fuller and Nirina Rabemiafara
Salford Futures 2013/14 Evaluation John Reehill Dave Timperley.
YEI in the PA and OP ESF TWG 4 March 2014, Brussels EMPL E1.
Common ESF Indicators in the Current Programming Period.
1 Joint meeting of ESF Evaluation Partnership and DG REGIO Evaluation Network in Gdańsk (Poland) on 8 July 2011 The Use of Counterfactual Impact Evaluation.
Regional Policy Guidance on monitoring TÓTH Gábor DG EMPL – Impact Assessment, Evaluation Unit ESF Evaluation Partnership meeting, Rome, 26 November 2014.
Implementation monitoring of the youth guarantee in Latvia Riga, April 2016.
Youth Guarantee. Why Youth Guarantee?  While young people often experience difficulty in transitioning from school to work, the recent crisis has hit.
Public Employment Service Austria
Overview of higher education statistics
Classification of the Working Age Population
National Skills Conference Presentation March 2017 Pathways of TVET College learners through TVET Colleges HSRC/DHET LMIP 5 Prof J Papier, S Needham.
Workshop on Strategic Programming, Monitoring and evaluation Focusing on Performance and REsults Madrid, 22 February 2013 Ines Hartwig DG Employment,
Public Employment Service Austria
Skills Escalator Pilot and ESF proposal Thursday 23 April 2015.
ESF Data Support Centre Data collection and validation support for the management of the ESF Andy Fuller ESF Support Centre Alphametrics Ltd.
Activation of young people in the Podkarpacie region
Presentation ESF performance report AIR 2016 ESF Technical Working Group 9 February 2018 Brussels Costanza Pagnini.
ESF - Monitoring: Q&A Andy Fuller and Nirina Rabemiafara
Specific objectives in
Overview performance report AIR2016
Workshop on Measuring the Transition from School to Labour Market Item 3 – Conceptual framework in the EU for the transition of youth from education.
Impact evaluation of actions for jobseekers under the current OP ESF- Flemish Community : beyond classical parameters for success Expert Hearing.
methodology Stratified random sample of PLOs drawn from 341; PULS databases from 69 PLOs (59 of them were complete and operable); data on
ESF EVALUATION PARTNERSHIP MEETING Bernhard Boockmann / Helmut Apel
Household situation indicators – french approach (DGEFP)
Youth Employment Initiative
Data reported and data quality in AIR 2017
The Social Investment Package (SIP) -20 February 2013
framework agreement on Inclusive Labour Markets
The Use of Counterfactual Impact Evaluation Methods in Cohesion Policy
Update of Practical guidance (Annex D) & FAQ Guide
ESF Support Centre Alphametrics Ltd. & Applica Sprl.
ESF Evaluation Partnership Meeting
Skills Escalator Pilot and ESF proposal Thursday 23 April 2015.
Amending the Performance Framework
Learning Seminar - Targeting employment policies
Challenges and lessons learnt
Update on the MIS risk assessment notes
State of play of OP negotiations
Common ESF Output Indicators
Survey on representative sampling
Guidance on Evaluation of Youth Employment Initiative
Monitoring & evaluation in
Terms of Reference provide the Commission with a tested template based on: a typology of Policy Areas a typology of Interventions and Common Indicators.
Kamil Valica Unit A.3 Impact Assessment and Evaluation
Use of administrative registers – the Swedish example
Annual Implementation Reports 2018
Costas VOYIATZIS DG EMPL - Evaluation and Impact Assessment Unit
Future Monitoring and Evaluation: Focus on results Antonella Schulte-Braucks Ines Hartwig ESF Evaluation Partnership Brussels 17 November 2011.
Common ESF Indicators in the Current Programming Period
Evaluating the effects of ESF programmes
The process of collecting data for monitoring the progress of SOPHRD related to data protection Management Authority for the Sectorial Operational Programme.
Debrief of Learning Seminar on Youth Employment Initiative
Tour de table ESF Evaluation Partnership Brussels, 7 December 2017
Counterfactual Impact Analysis applied in the ESF-Evaluation in Austria (period ) Contribution to the Expert-Hearing: Member States Experiences.
Youth Guarantee and Youth Employment Initiative – state of play
YEI reporting: state of play
Findings of the MIS survey and work programme for 2015
ESF monitoring and evaluation in Draft guidance
Web-supported Guidance
ESF evaluation partnership
Estimating net impacts of the European Social Fund in England
Evaluation of Youth Employment Initiative
Presentation transcript:

Representative sampling Overview of the questions received by the ESF Data Support Centre Alphametrics Ltd. & Applica Sprl. Brussels, 13 March 2015

Questions on dimensions Representative samples have to provide estimations per IP, category of region (except YEI) and gender. Sampling is not required at the level of the specific objective. Representativeness in terms of characteristics of participants: gender, age, employment status, education level and household situation. IP 1 Less developed Transition More developed Men Women It is good practice to ensure representativeness at the sub-regional level within each category of region (to ensure for instance that the sample is not biased towards participants from the capital in comparison to those from other areas within the same region). Regional representativeness could be achieved by establishing the sample one NUTS level lower than the level of the programme area (e.g. for an OP at NUTS 2 level, the sample should have the same distribution by NUTS 3 level as observed amongst all participants).

Questions on methods Stratified sampling Completeness requirement No obligation to use a stratified sampling method. But if this is not used, it will be necessary to have separate samples per gender and category of region. Completeness requirement In order to be considered in the reference population, records have to be complete for all non-sensitive personal data (excl. homeless/rural areas).

Questions on reference population Indicator Reference population Representativeness in terms of: Participants in employment 6m after leaving Unemployed Inactive - Unemployed/Inactive - Age - ISCED - Household situation Homeless All participants (up to end-2016) - Employment status - Age - ISCED YEI longer-term indicators All participants (each year) - Unemployed/Inactive not in education or training - Age if 25-29  Not possible to exclude specific groups from the reference population (e.g. students/children). Fields recording irrelevant data can be completed with 0. Each indicator requires a representative sample at the level of the IP reflecting the relevant population of participants. See Annex B of the EC guidance. For instance for ‘Participants with an improved labour market situation 6 months after leaving’ which refers only to participants who were employed on entry, fields can be completed with 0 for participants who were unemployed/inactive on entry.

Questions in relation to YEI 7 Longer-term result indicators (6m after leaving) Representative samples - Participants in employment 1 sample (unemployed, inactive not in education/training, ISCED, HH, age if 25-29) - Disadvantaged participants in employment 1 sample (unemployed, inactive not in education/training, ISCED, age if 25-29) - Participants with an improved labour market situation - Participants >54 in employment Not applicable (0) - Participants in continued education, training programme leading to a qualification, an apprenticeship/ traineeship - Participants in employment - Participants in self-employment 1 single sample (unemployed, inactive not in education/training, ISCED, HH, age if 25-29)  YEI indicators to be reported annually: 1 sample for each year. Annex I Annex II YEI targeted at participants <25 who are unemployed or inactive not in education/training. All the 7 longer-term result indicators should be reported. 0 values will be reported for the 2 common longer-term result indicators ‘Participants with an improved labour market situation’ and ‘Participants above 54 in employment’ as the reference population does not match the YEI target groups (young aged under 25 either unemployed or inactive not in education/training). One sample for ‘Participants in employment 6m after leaving’. One sample for ‘Disadvantaged participants in employment 6m after leaving’. One single sample for all the 3 YEI longer-term result indicators. For common longer-term result indicators (Annex I): sample should be representative in terms of the number of participants who are unemployed and those who are inactive not in education/training. For YEI longer-term result indicators (Annex II): sample should be representative in terms of the number of participants who are unemployed and those who are inactive not in education/training. YEI longer-term result indicators have to be reported annually. The sample should include participants that left operations 6 months before the end of the reported years (e.g. for AIR 2016, the sample should include participants leaving between mid-2015 and mid-2016).

Questions on the schedule Annex I common longer-term result indicators to be reported only twice (incl. for YEI) AIR 2018 (exits up to mid-2018) Final report (exits between mid-2018 and end-2023). Full sampling not possible until end-2018/mid-2024 (6m after last exit)  recommended to collect data more frequently (e.g. a sample per year) to ensure samples are not skewed towards a particular year and to help increasing response rates. Annex II YEI longer-term result indicators to be reported annually starting in April 2015. Homeless/rural areas to be reported only once in AIR 2016. Longer-term result indicators: there should be two waves of reporting and distinct samples with non-overlapping participants should be drawn: 1st wave (AIR 2018) covering participants leaving operations up to mid-2018; 2nd wave (Final report 2025) covering participants leaving between mid-2018 and end-2023. For the 1st wave, full sampling might not be possible until end-2018 (i.e. 6m after the last exits in mid-2018), for the 2nd wave until mid-2024 (i.e. 6m after the last exits in end-2023). The EC guidance however recommends that data are collected more frequently (for instance a sample for each year) to ensure that samples are not skewed towards a particular reporting year. This would also possibly help increasing the response rate as participants will be contacted shortly after leaving the operations.

ESF data support centre Advice and guidance on: Methodological support on monitoring information systems Clarification regarding data collection, recording, validation and storage Guidance regarding transmission of structured data to the EC using SFC2014 (technical queries > SFC support team) 9.00 a.m. – 5.00 p.m. Monday-Friday E-mail: support@esfsupport.eu Phone: +44 1480 702003