New Provider presentations Provider Risk and Assurance

Testing eligibility of apprenticeship funding This presentation will cover the Department’s approach to testing eligibility of apprenticeship funding.

Overview of our approach Testing is overseen by the Provider Risk and Assurance (PRA) function which sits within the Department’s funding agency We test on a sample basis at both provider level and apprentice level We advise providers if they are selected for testing a few weeks prior to a visit and issue our apprentice level sample a few days prior to the actual visit Our visits typically last between 3-5 days on site and we will work with you following our visits to correct any data errors and resolve any impact on funding Our testing is complementary to other assurance work undertaken including market entry checks, data analytics and focused thematic reviews Our normal arrangements are that some audits are completed by Provider Risk and Assurance (PRA) staff directly. Other testing is contracted out to external audit firms. Regardless of who is completing the testing, we have a single standard approach. All testing is overseen by PRA. We test on a sample basis – both at provider level and at apprentice level. If you are selected for audit, we will advise you a few weeks before our intended visit. In 2016/17, we plan to undertake levy testing between June and September 2017. In the next year we are likely to commence our testing earlier in the year. If you are selected for an audit, we will advise you in advance of our visit, and we will send our apprentice level samples a few days before our visit. This will allow you a short period of time to collate the necessary evidence. The number of apprentices selected for testing follows a sliding scale based on the number of apprentices on your programmes. We aim to complete our on-site within 3 to 5 working days. We will spend some up- front time arranging the visit, and also time following our visit to support you in correcting any issues identified, and to resolve any errors that may impact on the level of eligible funding. You should have a single dedicated audit lead throughout this process who will be able to handle any queries. Its important to say that funding audits complement a series other assurance work. You have already been subject to our market entry checks. We also perform sector-level data-analytics to identify specific concerns with providers’ data. Another strand of our work is to undertake focused thematic reviews, for example where a particular risk is identified. There may be additional data requests to inform our thematic reviews however, these will be planned and communicated to the sector in good time. We will audit the new apprenticeship funding as well as any other funding that you receive from ESFA. In the first year we plan to carry out some level of focussed audit work to test the new apprenticeship funding

The funding audit Subject matter Evidence Criteria It is the responsibility of the Department to ensure that public money is being used for the purposes intended and in accordance with the funding rules, and it is this assurance that the funding audit provides. The funding audit is made up of broadly three components. The subject matter is what we are seeking assurance over. This is the data you submit to the Department within the Individualised Record, or ILR. This data supports your claim for funding. The second component is the criteria, that is the rules that dictate what can be funded and so what data can be entered into the ILR. We have published a dedicated set of apprenticeship funding rules for providers on gov.uk. You’ll need to be very familiar with this document. The third component is the evidence that you will need to collect and retain, supporting that the data you have submitted is accurate, with funding claimed in line with the criteria. All three components are updated each year. We’ll talk about each of these three components in more detail, including giving some examples of the evidence we would expect to see.

Subject matter: Individualised Record The data you have submitted within the Individualised Record (ILR) must be correct The specification of the ILR is available online You must have controls in place to ensure the accuracy of data entered See https://www.gov.uk/governme nt/publications/ ilr- specification-validation-rules- and-appendices-2017-to-2018 It is the responsibility of all providers to ensure that the data is accurate. We will check the data submitted within the Individualised Record, or ILR. The specification for the ILR is available on gov.uk. The ILR itself is an XML file created from your underlying records. It is important that the data in the ILR is accurate throughout the year. You should consider the controls in place for entering the data, which should include competent and thorough review of the output report. You must not report inaccurate information even where you perceive that this would result in a more equitable claim for funding or accurate record of performance. Where your data and the evidence you hold does not support the funding you have claimed, we will take action to get the data corrected and recover funds. If you are selected for audit, we will advise which version of your ILR we will be selecting our sample from. We will also select our apprentice level sample following analysis of the ILR. (presenters NOTE: the ILR specs on gov. uk still state ‘SFA’)

Criteria: Apprenticeship funding rules for providers We will test eligibility against the funding rules published by SFA We convert the funding rules into a series of audit tests set out within our published working papers We will publish our working papers later in the year so that you can see specifically what we will be looking at See https://www.gov.uk/government/publications/appr enticeship-funding-and-performance- management-rules-2017-to-2018 We will test the eligibility of funding for each apprentice within our apprentice-level sample. We will also check to confirm if you have claimed the correct amount of funding. We will also review your ILR data and identify errors. The eligibility requirements are set out within the apprenticeship funding guidance. We covert the funding rules into a series of specific tests, against which we will seek evidence of compliance. We are developing our working papers, and will publish these later in the year, so that you are clear on what we are looking for. (Broadly, our testing will cover the following: Does evidence exist to confirm that the apprentice is eligible for ESFA funding; eg. do they meet the residency criteria, and is there evidence of a genuine job? Is the programme as designed eligible for funding and is the correct funding being claimed? Does the apprenticeship documentation meet the minimum requirements outlined in the funding rules? For example - Will the apprentice spend at lest 50% of their working hours in England? Is there evidence that the apprentice will spend at least 20% of their paid hours on off the job training? The apprenticeship must satisfy the minimum duration requirements – has that been met? Is the apprentice’s programme and the apprentice's participation as recorded on the ILR consistent with the underlying records? Are eligible costs included in the price? Has the co-investment (where applicable) been paid by the employer, and is the value correctly recorded on the ILR? Where the apprentice completes or leaves the programme, does the learning actual end date recorded on the ILR agree with underlying records? Does evidence exist to confirm eligibility for additional payments to be made (where applicable)? Do sub-contracting arrangements (where applicable) meet the requirements of the guidance?)

Evidence: Expectations set out within the funding rules You must collate and retain an evidence pack for each apprentice Evidence requirements are set out in the funding rules Government contributions (including additional payments) and payments from the digital account are driven by the underlying ILR data which must agree with the information and evidence you hold on record See https://www.gov.uk/government/publications/ sfa-financial-assurance-funding-assurance- review-programmes You are expected to hold sufficient and appropriate records to evidence compliance with the funding rules and support the data entries on the ILR. The funding rules set out what we expect so see within the evidence pack for each apprentice. Evidence comes from several different sources and you will be responsible for requesting and collating this evidence and storing it securely. Where evidence isn’t available, or cannot be provided to a reasonable timescale, we will report an error and recover funding and also request that the data be corrected. Over the next few slides we’ll look at some more examples of what specific evidence we require.

Examples of evidence we would look at Eligibility of the apprentice Each apprentice must meet the rules on who can be funded. The funding criteria include (but not limited to): the age of the apprentice Prior learning absence of any other apprenticeship enrolments Your evidence pack should include evidence of the apprentice’s eligibility and a record of what evidence the apprentice has provided. A genuine job: Your evidence pack should contain information about the apprentice’s job role and details of the contracted employment. There must be a genuine job, and they should be employed by that company and on a genuine PAYE. They must be paid a legal wage and receive support from the organisation to carry out their role. i) One of the typical evidence we will look at, is evidence to confirm the eligibility of the apprentice • Each apprentice must meet the rules on who can be funded. The funding criteria include, (but not limited to): • the age of the apprentice • prior learning the absence of any other apprenticeship enrolment Your evidence pack should include documents to confirm apprentice’s eligibility and a records of what evidence the apprentice has provided themselves. ii) We will check to confirm that you have evidence of ‘A genuine job’: • You have already heard earlier about the requirements of a ‘genuine job’. Your evidence pack should contain information about the apprentice’s job role, and details of the contracted employment.

Examples of evidence we would look at (continued) Commitment statement For each apprenticeship there should be a commitment statement signed and dated. The commitment statement must include the relevant information as set out in the funding rules. Your evidence pack should include a copy of the commitment statement for each apprentice. Price and eligible costs You can claim the cost of activity directly related to the apprenticeship. You must only claim eligible costs. You must evidence how the actual cost are calculated. i)The ‘Commitment statement’ is an important source of evidence: For each apprenticeship there should be a commitment statement signed and dated by you, the employer and the apprentice. The commitment statement must include the relevant information as set out in the funding rules. This can be signed electronically. ii) Price and eligible cost of training and assessment You have already heard what can and cant be claimed within the negotiated price. You can claim the eligible cost of activity directly related to the apprenticeship. You must only claim eligible costs, as explained in the previous slides including end point assessments where relevant We could ask for necessary financial records relating to your costs.

Examples of evidence we would look at (continued) Off-the-job training The apprentice must spend at least 20% of their time on off- the-job training, as defined in the funding rules. Your evidence pack should include details of how the 20% off-the-job training will be quantified and delivered. An absence of this evidence will result in a significant funding error. Employer payments You must retain financial evidence to show that the co-investment has been paid by the employer where relevant. Any part of the price over and above the cap must also be evidenced as paid by the employer. If you are claiming funds from the digital account, there must be appropriate evidence that the relevant employer is or is linked to the levy account holder (if different). Another key aspect we will check is to confirm that the apprentice will receive at least 20% off-the-job training. your evidence pack should include details of how the 20% off-the-job training will be quantified and delivered. You have already heard from Sarah Stear about the requirements for 20% off the job training. An absence of this evidence will result in a significant funding error. (this is based on paid hours of work; annual leave is included in paid hours). English and maths delivery, upto level 2 is not included in the 20% off the job training (this element is paid separately by ESFA; it is a separate element, as explained in the previous slide by Sara Stear)   We will check any direct payments you receive from the employer, where applicable: You must retain financial evidence to show that the co-investment has been paid by the employer. Any part of the price over and above the cap, must also be evidenced as paid by the employer. You must retain evidence of this within your financial records. Please note, if you are claiming funds from the digital account, there must be appropriate evidence that the relevant employer is or is linked to the levy account holder (if different).

Examples of evidence we would look at (continued) At least 50% working hours spent in England Apprentices must spend at least 50% of their working hours in England over the duration of the Apprenticeship Your evidence pack must include confirmation of this. Qualifying days Apprentices must be in learning for a minimum of 42 days between the learning start date and learning planned end-date before they qualify for funding. Your evidence pack must include confirmation that learning has taken place and that relevant records are available to confirm that. i) At least 50% working hours spent in England Apprentices must spend at least 50% of their working hours in England over the duration of the Apprenticeship Your evidence pack must include confirmation of this. ii) Qualifying days Apprentices must be in learning for a minimum of 42 days between the learning start date and learning planned end-date before they qualify for funding. (presenters to note: some exceptions apply as per the funding rules) Your evidence pack must include confirmation that learning has taken place and that relevant records are available to confirm that.

Examples of evidence we would look at (continued) Start of learning Funding can be claimed only from the date on which learning activity directly related to the apprenticeship, or learning aim (for English/maths), begins and can be evidenced. Subcontractor arrangements You may subcontract the delivery of part of an apprenticeship to an approved external training provider, subject to specific criteria within the funding rules. Your evidence pack should include details of any subcontractor, clearly identifying who they are. This must match the information reported to us in the ILR. i) Another important point we will check relates to: ‘Start of learning’ • Funding can be claimed only from the date on which learning activity directly related to the apprenticeship, or learning aim (for English/maths), begins. • Your evidence pack must contain evidence of the date from which learning activity directly related to the apprenticeship or the learning aim actually started. ii) We will check to confirm whether the sub-contracting arrangements you have put in place, if any, are in line with the guidance. • Your evidence pack should include details of any subcontractor, clearly identifying who they are. This must match the information reported to us.

Data quality Good understanding of the subject matter, criteria and evidence needed Maintain quality data throughout the year Adequate understanding of the data requirements Day-to-day management of data and data quality Regular review of funding monitoring reports and timely action to correct any issues Challenge by management and sound data governance With sufficient preparation throughout the year, our audit visits should not be onerous nor result in any significant follow up work. We would advise that you hold sufficient knowledge within your organisation of the subject matter, criteria and evidence needed for a funding audit. You should ensure that relevant staff are familiar with the published documents and alert to any updates. We also advise you to proactively manage the quality of your data. You should have clear systems and processes in place to share information on your apprenticeships quickly and efficiently. You should also regularly review your data quality, at an appropriate level of your organisation, and take steps to investigate and correct any issues as they arise.

Quality evidence Maintain quality audit evidence throughout the year Adequate understanding of the evidence requirements Obtaining sufficient appropriate evidence in a timely manner Retaining evidence securely Thank you for listening And finally, we would advise you to have a robust process in place for collecting and retaining evidence in support of the data submitted in the ILR. You should be familiar with our expectations as set out within the funding rules.