1. Annual activity report and residual risk
DG Employment, Social Affairs and Inclusion
DG EMPL

2. Outline Assurance process ACR 2014 AAR 2014 – assurance / reservation
Error rate and cumulative risk

3. Assurance process

4. Assurance process

5. ACR 2014 Global overview All ACRs received except 2
Commission analysis:
Increase of flat rates applied
Recalculated error rates:
17 recalculated ER with a relatively low impact of +/-0,2%
National Audit Error rates
2014
2013
Reliable, unchanged 73
61.8%
88.1% 65
55.6%
90.6%
Reliable, recalculated 31
26.3% 41
35.0%
Not reported or unreliable error rate 14
11.9% 11
9.4%
118
100.0%
117

6. ACR 2014 Global overview Fact Finding missions:
22 FF missions (ACR 2015 – 27 ff missions)
covering 37 operational programmes (ACR 2015 – 42 O.P.s)
representing 55.9% of 2014 payments (ACR 2015 – 54,7% of payments)
In addition, taking into account some meetings with AAs, 1 audit mission end 2014 and 1 FF by DG REGIO:
44 OPs covered
representing 75.5% of 2014 payments

7. ACR 2014
Global overview
ACR 2014 average error rate (on 2014 payments): 2.8 %
An increase of 0.9 % compared to 1.9 % that would have resulted by using the error rates as reported by the AAs, without any revision by DG EMPL.
Applied to 2013 interim payments, the average error rate is 3.1 %
In the range disclosed in the 2013 AAR (2.6 to 3.5 %) and consistent with the error rate of 3.1 % reported by the ECA.

8. Best estimate Error rates
ACR 2014
Error rates (after Commission analysis)
Best estimate Error rates 
Number of programmes
2014 interim payments
<2% 68
57.6%
55.0%
2-5% 23
19.5%
19.8%
5-10% 16
13.6%
22.9%
21.8%
25.2%
>10% 11
9.3%
3.4%
118
100.0%

9. Average error rate on ESF payments
ACR 2014
Comparison versus previous ACRs
2014
2013
2012
Error rate range
0 to 32,9 %
0 to 37,8 %
0 to 25,8 %
ESF payments (M€)
9.926
12.645
10.711
Average error rate on ESF payments
2,8%
3,1%
3,0%

10. ACR 2014

11. AAR 2014 - reservations Same methodology as previous years
Error rate <2% - no reservation
Error rate >=5% – reservation
Error rate 2-5%:
cumulative approach of risk exposure;
if cumulative error rate >2%  reservation
Reputational reservation: No payments 2014 but overall assessment of systems in categories 3 or 4
Partial reservation: categories 3 or 4 for a part(s) of the programme (specific IBs)

12. AAR 2014

13. AAR 2014 Implemented Financial Corrections on a cumulative basis:
in M€
Total end 2013
cumul end 2013
2014
cumul end 2014
Total end 2014 AT
1,6
1,5
0,0
3,4
5,0 BE
23,4
12,3
8,0
2,7
10,7
3,1
13,1
16,1
39,1 BG
2,6
0,3
2,9 CY CZ
41,1
9,2
50,3 DE
37,6
1,9
23,6
2,5
26,1
12,1
40,1 DK EE
0,8 ES
1064,3
180,2
734,4
99,0
833,4
149,7
1,7
151,3
1164,9 FI
0,1 FR
284,5
45,6
219,5
221,2
19,3
21,1
40,3
307,2 GR
67,3
18,8
20,3
48,5
0,5
49,0
69,2 HU
34,4
8,2
26,2
15,8
41,9
50,2 IE
34,2
28,3
2,4
18,9
21,3
53,1 IT
497,7
117,0
375,9
20,5
396,4
4,8
518,2 LT LU
6,0
4,1
1,8 LV
7,6
3,2
4,5 MT NL
203,4
159,7
43,8 PL
169,2
51,2
118,0
32,6
150,6
201,8 PT
6,5
6,3
0,2 RO
299,1
13,0
312,1 SE
11,9
11,4
11,5
0,4
12,0 SI
2,0 SK
12,9
1,0
11,0
5,9
16,8
19,7 UK
184,0
8,8
163,2
0,7
163,9
24,0
36,0
208,7
2992,1
559,7
1677,4
133,1
1810,5
755,0
155,9
910,9
3281,1

14. Error rate calculation
- National audit strategy describes sampling methodology;
- Sampling methodology = statistical (except for small populations, min 10%)
- Methodologies commonly used:
Conservative MUS;
Standard MUS;
Difference estimation/Ratio estimation;
Simple Random Sampling;
- Guidance and training provided to Audit Authorities

15. Error rate calculation
- Assessment carried out by EMPL Audit;
- Calculation of annual risk rate based on EMPL payments;
- Verification of table on "withdrawals and recoveries" (Art /2006);
- Verification of withdrawals & recoveries carried out in calendar year;
- Calculation of residual risk for programming period.

16. Error rate calculation - reliability
2015 figures provisional

17. Experience sharing by the Commission
Residual Risk : implication for closure
Experience sharing by the Commission

18. Definition of Residual Risk Rate (RRR)
Residual Risk Amount
Expenditure declared at closure

19. Residual Risk Amount Amount at risk Corrections applied = = TPERx
exp. declared in ACR
Corrections applied =
W&R reported by CA
and verified by AA

20. Financial corrections
Year
Exp. in the ACR
(A)
TPER
(D)
Risk
(E)
Financial corrections
(F)
Residual risk amount (year)
(G=E-F)
Residual Risk, (cumulative)
2010
150 3
4,5
1,5
2011
200
2,5 5 7 -2
2012
180 2
3,6
1,6 …
300
4,1
Exp. in final claim
730
Residual Risk Rate = (residual risk)/(expenditure in final claim)
0,56 %
≠ ( … + 300)
Homologues Group meeting

21. The Cumulative Residual Risk (CRR) estimated by the Commission
Experience sharing:
The Cumulative Residual Risk (CRR)
estimated by the Commission

22. Main concepts
The CRR is an estimation (likely subject to adjustments); although based on (for a large part) statistically calculated error rates, the CRR itself is not calculated with statistical parameters
It is an indicator of the corrective capacity of the management and control system as a whole throughout the lifetime of the programme (cumulative state of play while progressing towards closure)
It is used as a criterion for additional reservations in the context of the DGs' Annual Activity Reports

23. Differences with the RRR
Timeframe AAR / reporting on withdrawals and recoveries + assurance on accuracy of data
Yearly estimation, based on latest information available
Risk of misstatement for last year by using the last available error rate (error rate of year N-1 to estimate the risk in year N)
Risk estimated on the payments executed by EC, instead of expenditure declared
Statistical limitations in some cases

24. AA opinion in the closure declaration
The Residual Risk Rate is a key element of the audit opinion to be provided by AA at closure According to the closure guidelines (sections 5.3 and 5.4), in order to obtain a non-qualified opinion corrective measures will have to ensure that the residual risk rate is below material level (2%)
What if the residual risk rate at closure is above 2%?

25. AA opinion in the closure declaration
If the RRR at closure is above 2%, the AA can conclude that the management and control system failed to provide adequate assurance that the expenditure declared to the Commission is legal and regular. Therefore, in this situation, the Commission will apply a financial correction based on this residual error rate in accordance with the Commission decision C(2011) 7321

26. To summarise
Remember that the RRR is only an estimation, to be used as an indicator of the corrective capacity of the management and control system
Additional work should be envisaged at closure on the two elements of the RRR:
Amount at risk: need to confirm representativeness of error rates reported + cleaning up of information on expenditure declared in the ACRs
Corrections: need to ensure quality and reliability of data reported by CA

27. To summarise
The AA is strongly recommended to consult the Commission in due time before the closure, in order to address all potential issues related to the error rates and the corrections to be taken into account
For ERDF and CF, we start by providing you with an excel file with the:
Error rates communicated in the ACRs, as validated by REGIO (flat rates in some cases)
Expenditure declared, as from ACRs
Withdrawals and recoveries reported by CA
Proposed formulas
N.B: this table should be seen as a tool to help AAs to estimate the RRR, not as a starting point for its calculation

28. Cumulative risk calculation

29. Cumulative risk calculation
Annual Activity Report 2014
Cumulative risk exposure

30. Questions?

31. Closure – DG EMPLaudit strategy proposal

32. Guidance documents Guidance issued to Member States
Presentations offered
Closure discussed at "Homologues meeting"
General presentation ESF TWG Luxemburg – December 2015

33. DG EMPL Risk Assessment
Criteria:
Reliability of the Audit Authority;
Absorbtion rate of the O.P.;
Retrospection;
Financial Instruments;
Eligibility instructions to Beneficiaries (31/12/2015 deadline for payments)
Unique sample and coverage;
Others

34. Planning
Enquiry Planning Memorandum "Preparation for closure" in draft stage;
Audit planning Q3 and Q4
(Thematic) Reporting Q1 2017

35. Communication to AA Audit plan in draft
Presentation EMPL Board of Directors mid March
Information to AA during Annual Coordination Meetings (April – June)
Announcement of detailed audit planning in the framework of the audit missions.

36. Closure – ECA planning

37. Ongoing performance audit of closure 2007-13
Two phase audit
The 1st phase examines the design of closure
The 2nd phase will focus on effective implementation of the closure
Main audit question
Is the closure of the Cohesion and Rural Development programmes designed to achieve its effective implementation?
Areas the Court will be looking at:
Regulatory framework as a basis for an effective closure
Commission support to the Member States including feedback on Member States’ preparedness
Procedures and resources at Commission level
Resource requirements at Commission and Member State level
Not yet
planned

38. Ongoing performance audit of closure 2007-13
Audit methodology and approach
Review of regulatory framework, guidance, and procedures at the Commission and Member State level
Assessment of Commission and Member States readiness to conduct closure
Survey to Member States to receive feedback on Commission assistance including:
Audit Authorities and Managing Authorities
Paying Agencies and Certification Bodies
Interviews / visits to selected Member States

39. Ongoing performance audit of closure 2007-13
Timetable
Audit work starts in September 2015
Survey will be carried out in October and November 2015
Interviews will be undertaken in the same period and shortly after
Report should be published in second half of 2016

40. Ongoing performance audit of closure 2007-13
Potential risks:
The documents which should be provided by the Member State for the closure are not reliable and / or timely provided. This includes:
The inherent conflict between complying with rules and absorbing EU funds might motivate Member State to submit unreliable closure documents to the Commission
A risk that national checks which are the basis for the Audit Authority closure declaration might not have sufficient coverage and / or not be effective
The closure might not be implemented by Member States in a way to ensure an efficient and timely closure process

41. Ongoing performance audit of closure 2007-13
Potential risks (continued):
The Commission internal procedures might not be implemented in a way to ensure an efficient and timely closure process
The Commission guidance to the Member States may not be timely available or incomplete
The Commission is not verifying that the assurance framework is reliable, thus accepting irregular expenditure at closure. This includes:
Commission’s verifications and audit work not covering all remaining risks at closure
Commission’s checks on how anomalous errors are determined and treated by the Member States

42. Ongoing performance audit of closure 2007-13
Potential risks (continued):
The treatment of specific issues / situations occurring during the closure is inappropriate
The flexibility to allow for maximum absorption of EU funds within the closure exercise, including:
Transfer of funds between axes / priorities
Final date of eligibility of expenditure
Increases of co-financing rate

43. Thank you for your attention
Questions?