Findings of the MIS survey and work programme for 2015 ESF Support Centre Alphametrics Ltd. & Applica Sprl. 13 March 2015

Basis of review MIS fiches pre-filled where possible with replies to 1st survey Validation of replies Editing, translation Checks on completeness, clarity, coherence Risk assessment Several rounds of consultation (mid-Sept 2014 – end-Feb 2015) 41 replies received out of 47 National / regional level Mostly MAs, but also national coordinating bodies Risk assessment notes prepared on the basis of the information provided in the MIS fiches.

MIS structure Potential risk (EE, EL, ES, LT, UK-England: not determined yet) Potential risk In terms of data quality where the number of beneficiaries per MA is high. Beneficiaries per MA Countries <100 BE-Actiris, DE-Hamburg, LU, MT, NL, UK-Northern Ireland, UK-Scotland and UK-Wales 100-499 BE-Agence FSE, DK, DE-Berlin, DE-Saarland, DE-Schleswig-Holstein, PT-Madeira 500-999 LV 1,000-1,999 DE-Bremen, IT-Liguria, SI, SK, FI and SE 2,000-10,000 BG, CZ, DE-Baden-Württemberg, DE-Brandenburg, HR, IT-Emilia-Romagna, CY, HU, PT (National) and RO >10,000 DE-Thüringen, FR, IT (National), AT and PL

Planned developments MIS 2014-2020 Main changes Existing system to be updated in most cases New system in 10 MAs Main changes New module on micro-data Links with administrative registers Online submission of participant data New validation rules Most systems are still under development Planned for 2015 In a few cases, data input is already possible but validation rules not ready yet Potential risk In terms of schedule: MIS not fully operational for reporting on YEI (April 2015) in many countries.

Type of support offered Guidance material Specific dedicated document (sometimes available online) Integrated in the template of the questionnaires Integrated in the MIS itself FAQ on monitoring and evaluation Obligation to follow the guidance stated in the contracts in some cases Training/workshop/information sessions Helpdesk/contact points offering personalised assistance Potential risk In terms of data quality and comparability: specific guidelines are necessary to ensure that definitions are clear/consistent.

Indicators based on national definitions Foreign background/minorities, participants with disabilities and other disadvantaged: national definitions. In many MS, national definitions exist and in some MS the EC recommended definitions will be used. Potential risks Specific groups may not be included (e.g. unlawful to ask questions about minorities). Double counting: ‘Other disadvantaged’ includes groups already covered. Internal comparability: open definition of ‘Other disadvantaged’.

Consultation of data protection authorities Compliance with legislation on data protection in many cases Specific arrangements concluded for 2014-2020 Existing legislation already allows collection of personal data Administrative registers already comply with data protection laws. Consultation process still in progress in a number of cases Potential risk Delay in collecting participant data if no decision is taken relatively quickly.

Refused consent Procedures to handle refused consent Consent forms Option ‘does not wish to reply’ for questions on sensitive data Importance of complete data records highlighted in forms/guidance Individuals refusing to provide basic (non-sensitive) data are not eligible to benefit from ESF support in a number of cases. Clarification Eligibility <> Monitoring

Administrative registers Registers will be used in 19 MAs (PES, Social security, Population registers…) Use of registers not planned in 12 MAs To be decided in the remaining 10 MAs Data not covered will be collected directly from participants Potential risks Data extracted from registers are not synchronised with ESF observation points. Administrative data obtained with a time lag, which represents a risk of delay in the reporting.

Data validation Consistency and completeness of data Built-in checks in most systems (+ sometimes on-the-spot checks). Potential risk: In terms of data quality if the validation procedures are not automated. Treatment of inconsistent/incomplete records Flagging of inconsistent/incomplete records Warning messages, beneficiaries asked to correct/complete records Not possible to save incomplete records Incomplete records saved in a separate database Level of validation More than 1 level in most countries (+ sometimes external bodies). Potential risk: In terms of data quality if data is not validated at different levels (beneficiaries/IB and MA).

Data storage Level of storage of micro-data MAs in 9 cases Beneficiaries in 3 cases ΙΒ/Statistical Offices in 3 cases Multiple levels in most cases (beneficiaries, MA, IB, other). Practices in storage of data Centralised shared system (with different permission levels) Micro-data are provided to the MA on request Potential risk Single data set necessary for drawing representative sample.

Data aggregation Automated aggregation by IP/category of region in all cases except one. Potential risk: In terms of data quality if calculations are done manually. Different methods planned to ensure that incomplete/ inconsistent records are not included. Alternative aggregations in response to ad hoc requests planned in most countries but not all. Potential risk: Specific information stored but not available. Retroactive changes in aggregates is planned in most cases.

Representative sampling Responsible organisation MAs in charge in 5 cases. Outsourced in 16 cases. Data collected for all participants for all indicators in 6 cases. Data collected for all participants for some indicators in 2 cases. Potential risks Samples based on participants with incomplete records (instead of complete records only). Samples drawn at the beneficiary level (instead of the IP).

Risk assessment notes Identification of risks in terms of data quality and compliance with monitoring requirements. Suggested actions to remedy the risk(s). Further information (clarification and/or completion) needed in many cases. Follow-up will be carried out by geo-desks (ESF centre in copy). End-2015: final summary on issues which have been addressed (and how) and what is outstanding.

Next steps Follow up on the findings of the MIS survey Ongoing methodological support Update of the Practical guidance document (Annex D) and FAQ guide Developing and testing the SFC reporting module.