1. Funding Audit Masterclass
7th December 2010

2. Funding Audit 2009/10 Update
Karl Bentley
RSM Tenon

3. What we’ll be covering
Background
What we covered
What we found
Things to consider
Rumour control
The Four C’s

4. Background to the audit
Some 85 providers have been audited
Selection for the majority was on a sample basis, others based on risk or results of previous audits
We did 19 College audits and mentored PFA for a further six
Two qualified audits, one as the data did not reflect the funding, other was a technical qualification – no impact on funding but data had lots of weaknesses
Overall there were signs of improvement, especially those we have audited for the last two years

5. What we covered The usual suspects Eligibility Attendance/Existence
GLH for unlisted aims
Fee Waiver
Disadvantage Uplift
NVQ
Distance Learning
Unitisation
Partnership
Other funding streams
DSAT
EMA and ALG

6. So what did we find this time?
Can anyone guess what the issues were this time round?

7. The Results
Some marked improvements in some areas
Eligibility was less of an issue this year
Some improvements on GLH
A51a was actually being completed this time

8. The Results
GLH is still proving to be a problem
Skills for life and Supported Learning
19+ NVQ provision
Functional Skills has been a real problem this year
GLH on ILR and register did not reconcile in a lot of cases
Under and over claims
View is that some Colleges treated them as key skills, forgetting that they were unlisted aims

9. The Results continued
Start, withdrawal and planned end dates
A51a still proving to be a problem – BTEC National Awards, Certificates and Diplomas were better, but still not right
Colleges had not seen the guidance issued for Construction Diplomas though
DSATs – Plenty of issues again, Early Completers report revealed some interesting results

10. A simple request
When the sample is sent we include a document call the ‘Prepared by Provider Report’
Please read it!
It tells you exactly what you need to provide to us at the start of the audit – like on day one, not ready half way through the week
Please make sure your other funding streams reports are ready

11. Rumour Control
Not processing your withdrawals before LR01 will not help to improve your learner numbers for future years
It will catch up with you and you’ll be having to explain why the numbers changed later in year
Will probably make you far more likely to be on the risk radar for SFA and YPLA
Audit simplification will only come with a simplification to the funding – we come at the end, not the start!

12. The Four C’s
Did anyone challenge the auditors this year?
Communicate, Check, Challenge and Complain
It works
We’re aware of one case where the auditors were instructed to remove a management letter point

13. DSAT: Things to consider
Timetable your DSATs
Don’t run every report every month
Some will only need running towards the end of the year
Prioritise your reports – think about the funding first!
Personal pet hates…
Learners way past their expected end date but still active
Internally certificated qualifications should not be left as completed, awaiting result for a long time!

14. Beyond DSAT …
drakelane.co.uk
Presented by Ed Drake

15. Aims part of the validation process
To highlight how ‘cleaning’ ILR data via LIS and DSAT is only
part of the validation process
To stress the importance of setting your data standards higher
in order to approach audits with greater confidence

16. Starting point …
ILR data is “clean” according to LIS validation
… and …
ILR data is “clean” according to DSAT validation
Inaccurate data Accurate data

17. How clean is “clean”? following:
“Clean” LIS and DSAT validated data may still reflect the
following:
Clearly incorrect learning aim references
Inconsistencies between 1st and 2nd year data
Significant funding under-claims
Significant funding over-claims
- Missing activity

18. Suggested remedial / further action
Write / run additional data validation reports
Sample check to ensure that the ILR data is representative
of actual activity

19. Suggested additional validation reports
Comparison of actual (delivered) GLH against ‘listed’ (funded) GLH

20. Suggested additional validation reports
Comparison of current year outturn against previous year outturn (including a range of like-for-like analyses)

21. Suggested additional validation reports
Comparison of current year outturn against previous year outturn (including a range of like-for-like analyses)

22. Suggested additional validation reports
Comparison of current year outturn against curriculum plan
Consistency of ILR data between years for programmes which run over more than one funding year – including:
Existence
Learning aim reference
Funding stream
Start and planned end dates
Guided learning hours
Remission status

23. Suggested additional validation reports
Additional funding under/over-claim reports – including in relation to:
16-18 full time entitlement (‘enhanced’ funding)
Tuition fee remission
Disadvantage
Funding adjustments (A51a)
Improving / ‘fixing’ DSAT reports
Reconciling any 2 ILRs to understand / explain differences

24. Audit cycle When to undertake these data checks? … Yesterday!! Now!!
Ongoing
Note: The days when ILR data could be ‘tidied-up’ by the end of the year have long gone

25. SCORE ADaM Update to SCORE 2010 released yesterday
Additional 12 reports (eg. potential A51a triggers)
FREE SCORE 2009 software available for evaluation purposes
ADaM
In-year monitoring of changes commences (post-LR01)
Update due by end of January (additional nesting reports)

26. Employer Responsive Timeliness Reports
Ellie Fraizer, Data Quality & Assurance Manager
7th December 2010
Funding Audit Masterclass, Lakeside Centre - Birmingham 26

27. ER Timeliness Reports Why they are being introduced
How they are being calculated
A look at the reports
When and where they will be available 27

28. ER Timeliness Reports – Why
Skills Funding Agency Guidance Note 3
Need to set timeliness standards for efficient and transparent financial reconciliation for Employer Responsive funding
All starts must be reported within 2 reporting months
All achievements and leavers/withdrawals must be reported within 3 reporting months 28

29. ER Timeliness Reports – How
EXAMPLE
For a learner starting an aim in month X, a record for that start must be included in the return at the end of month X+1 (at the latest)
Learner starts on 1st September This must be reported for the first time in the return for September (ER02) or October 2010 (ER03). That is, at the latest, in the return that closes on 4th November 2010 (ER03) 29

30. ER Timeliness Reports – How
Reflections on methodology
Matching: No unique reference for matching aims (A05 cannot be used). Aims treated as the same if they have a common:
Provider Number (L01)
Learner Reference Number (L03)
Learning Aim Reference (A09)
Learning Aim Start Date (A27)
Consistent with methodology used for matching aims in TTG success rates 30

31. ER Timeliness Reports – How
Reflections on methodology
Instances where a single learner has more than one aim with same Learning Aim Reference (A09) and Learning Start Date (A27) are treated as a single aim
Changes in A09, A27 and A40 will not be factored into the calculation
Late accredited aims may be flagged as a late start
Methodology and coding included in zip file 31

32. ER Timeliness Reports – When & Where
Released 1st December!
Provider Gateway Document Exchange
Provider Data Quality Reports, DQ-ER folder
Report enquiries (methodology etc) and Gateway/Doc X access issues, please contact the Service Desk: 32

33. Summary and conclusions
Nick Linford

34. Avoiding the pitfalls of audit
Rule 1: Understand the types of data quality
Rule 2: Do not only prepare for audit
Rule 3: Maintain data quality all the time
Rule 4: Know what is important for you
Rule 5: Ensure SMT and Governors understand how important you are
Rule 6: Rule 5 keeps us all in a job and the college out of trouble 

35. Data quality Important to understand the types of data quality
Validity (LIS error reports)
Credibility (LIS warning reports, DSATs, Ofsted, ILR FAQs)
Completeness (Data Dashboards and Annex P thresholds)
Timeliness (ILR Hard Closes and new ER reports)
Also understand that ILR data should not always be accurate……..
e.g. The learning aim is what was planned, not what was actually achieved.

36. It’s not only the funding auditor
Skills Funding Agency Account Manager
Ofsted (Jo Parkman)
Local Authority & YPLA (soon to be EFA)
The learners (see Personal Learning Record)
And of course your college (LR01 withdrawals…)
P.S. Framework for Excellence results published publically next week

37. Maintain data quality all the time
It is no longer advisable to wait until the end of the year before cleaning the data prior to audit
~ Monthly ER timeliness report
~ Ofsted only need to give 2 weeks notice
~ Monitoring performance in-year (all the time) is a great way to drive up efficiencies
Do you need to pay for internal auditors, and where you do are you being honest (and are they any good)?

38. What’s most important to you?
Are all 66 DSAT reports critical to you?
Biggest potential pitfalls
~ Unlisted functional skills (incl. embedding)
~ Use of the A51a discount field (esp. construction)
~ Eligibility for FL2 and FL3 entitlement
~ Transfers and learning aim changes
~ Guided Learning Hours (incl. listed aims)
Note: If there has been an error it is ok to change it at any time

39. SFA Policy Statement highlights how important you are
Higher Risk Providers & Persons and Agency
Contracting (03/11/10)
“a small number of providers have tarnished this otherwise positive relationship, and have achieved financial gain at the expense of both learners and the public purse, whether through ignorance or design.”
Additional criteria for rejection of providers
“previous activities that have resulted in significant repayment of funding (£100k or more)”
“two or more instances where the Agency’s staff have audited the provision of the bidder and identified significant concerns resulting in a qualified opinion”
ACTOR helps implement this

40. Nationally produced ILR audit reports
~ LIS errors ~ LIS warnings ~ OLDC validation (ULN and Employer Identifier) ~ DSAT reports ~ Data Credibility and Completeness (Dashboards, Annex P thresholds) ~ ER Timeliness

41. Other ILR audit tools ~ SCORE and ADaM
~ Your own system validations and reports
School 6th forms don’t know how lucky they are!
School 6th forms don’t know how lucky they are!

42. School Success Rates
YPLA have recently published school success rates following an FoI request from a journalist
23% unknown outcomes for Academies
16% unknown outcomes for School 6th Forms
In total 160,000 exams taken where the outcomes were unknown
Always nice to end a seminar on a high (AKA feeling smug!)

43. Any questions?
Sign up for free updates at