1. Update on the MIS risk assessment notes
ESF Support Centre
Alphametrics Ltd. & Applica Sprl.
Brussels, 10 June 2015

2. Risk assessment notes
Based on the MIS fiches received by end-February 2015, risk assessment notes were produced to identify any potential risks to data quality that arise from the MIS developed by each MA.
March 2015: the risk assessment notes were sent by the EC country desks to the MAs.
Up to now, feedback was received from 9 out of 41 MAs.

3. Main issues (re-cap) MIS structure and development
Not ready for YEI reporting
Need of testing MIS (recommended to provide support)
Definitions of indicators
Inconsistent application
Not in line with EC guidance (“migrants, foreign background, minorities”, “other disadvantaged”, and use programme-specific indicator instead)
Data collection
Arrangements with data protection authorities not in place
Participants refusing to provide information will not be counted
Use of administrative registers: delays, entry/exit dates
[…]
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

4. Main issues (re-cap) Data validation Data aggregation […]
Only at one level (e.g. only at MA level, only at beneficiary level)
No automated validation checks
Data aggregation
No automatic aggregation by IP and region
Lack of (automated) measures to ensure reporting includes retrospective changes
Delays in alternative aggregation or not possible at all
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

5. Main issues (re-cap) Data storage Representative sample […]
Full micro-data set not available
Representative sample
Sample include incomplete records, non-supported participants
Samples drawn at lower level (e.g. beneficiary)
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

6. Risks addressed MIS development
Readiness of MIS for YEI reporting
Regional MIS already operational (even if not finalised yet)
Until MIS ready, data collected/stored in PES system
YEI implementation has not started yet (system ready by June)
Development and testing phase
Working groups set up for stages of development of IT system
New MIS is being tested
IT user manual + trainings
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

7. Risks addressed Definitions – not in line with ESF guidance
Other disadvantaged: homeless not included
Will not be included to avoid duplication – refer to updated guidance
Migrants, foreign background, minorities: groups not covered recommended definition not applied
Amended definition
Definitions – inconsistent application
Other disadvantaged: open definition
The definition, although with guidance provided will remain open (i.e. risk only partially addressed)
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

8. Risks addressed Data collection
Discussions still ongoing with data protection authorities
Permission should be granted by the time MIS is ready.
Data protection authority has checked the data collection forms, the guidance note and the legislation.
Refusal to provide non-sensitive data leading to non-eligibility
Individuals can be supported (if they fulfil the eligibility criteria).
Recommendation: MIS should be able to record incomplete records (grand-total of supported participants).
Recommendation will be followed.
[…]
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

9. Risks addressed Data collection
[…]
Administrative data not reflecting situation on entry/exit, or obtained with a delay
Pilot data collection: project start date = entry date for all participants but adjustments might be made following evaluation of the pilot
Registers not updated on a daily basis but provide high quality data
Collection of sensitive data based on anonymous forms with no participant ID
Personal codes generated by the MIS might be included in the anonymous form (pending data protection agency authorisation)
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

10. Risks addressed Data aggregation
Alternative aggregations
In-built filters cover a wide range of variables
Data for ad hoc requests provided by Statistical Office
Obtained with some delay as some data are stored in the NSI
Automated mechanism to identify data that have been retrospectively modified
Dates recorded in the system and identification of updated data
34 YEI OPs in 20 MS: BE(2), BG(1), CZ(1), IE(1), EL(1), ES(1), HR(1), HU(1), CY(1), IT(1), LT(1), LV(1), FR(13), RO(1), PT(1), PL(1), SE(1), SI(1), SK(1) and UK(2).
MIS not fully operational for the YEI report  potential risk of errors in the treatment/reporting of participation records (e.g. if validation/ aggregation procedures are not in place yet).
It is necessary to test the new monitoring system (e.g. by using pilot projects) before launching it. Moreover, it would be useful to provide support (e.g. IT user manual, FAQ, IT support desk etc.) to all organisations using the system for the first time. 

11. Feedback is precious!
In addition to the risks identified in the RA notes, many issues are still pending or waiting for further clarification…
 Please send your replies/comments/updates:
1st round consultation: those who have not reacted to the RA note yet
2nd round consultation: those who already reacted to the RA note and for whom an updated RA note was sent
No feedback received on important risks identified, particularly automated data validation

12. ESF data support centre
Advice and guidance on:
Methodological support on monitoring information systems
Clarification regarding data collection, recording, validation and storage
Guidance regarding transmission of structured data to the EC using SFC2014 (technical queries > SFC support team)
9.00 a.m. – 5.00 p.m. Monday-Friday
Phone: