1. ©Velg Training 2013 AIS NSW VET Network Day 15 th July 2013 Tamara Simon Velg Training

2. Overview Regulation 1 Key areas of non-compliances 2 Role of the RTO 3 Continuous Improvement 4

3. Overview Training and Assessment Strategy (TAS) 5 Industry Consultation 6 Record Keeping 7 Assessment 8

4. Regulation

5. What’s your biggest challenge in delivering VET in your school?

6. RTO Compliance - VQF VET Quality Framework Standards Fit and Proper Person Financial Viability Data Provision AQF

7. ASQA exercises its responsibilities based on: a robust framework of legislation and standards a risk-assessment focus active engagement with industry a rigorous audit methodology fairness and transparency promoting informed consumer choice accountability accessibility ASQA’s approach to regulation

8. Key areas of non-compliance

9. Generic strategies for training and assessment Lack of industry consultation to inform strategies Poor assessment tools and processes Lack of effective validation of assessment Currency of staff competencies Lack of systematic processes – data collective and analysis Lack of data collection and analysis on client services Evidence of meaningful improvements ‘Torpid’ policies and procedures Common issues at audit

10. School non-compliances Qualification has been issued and school has phoned Board to issue different (incorrect) credentials TAS outlines onsite and offsite access to facilities and resources but incorrect or don’t have them Old units of competency listed on marketing material issued to students and parents Changing teachers to deliver VET qualification and not notifying AIS NSW – teacher doesn’t meet SNR 15.4 AIS NSW Findings

11. School non-compliances School issuing their own qualifications/units of competency to students Assessment instruments not valid Teachers not following policies and procedures No evidence of validation and moderation AIS NSW Findings

12. Role of the RTO AIS NSW

13. AIS NSW They are the RTO – not each school Risk is all with themPartnering agreementsNotify of changes and gain approval1 school can de-register the RTO

14. Documented agreement Register of partnering arrangements Evidence of monitoring Partnering arrangements

15. Continuous Improvement

16. Training and Assessment Client Services Management of the RTO Continuous Improvement

17. Is there a difference?

18. Follow Up Regular Internal Reviews/Audits Examples Evident and Documented Part of the culture Continuous Improvement

19. Training and Assessment Strategy (TAS)

20. Need to be documented – may be made up of a number of documents Need to reflect what actually happens Need to reflect current training package Include evidence of industry consultation Need to include details of monitoring/improvements Training and assessment strategies

21. Training and Assessment Strategy inclusions Qualification and core and elective units Clients/learner needs Prerequisites/co-requisitesLicensing/regulatory requirements Entry requirements/course durationMode of delivery/delivery strategies Assessment strategies/evidence gathering techniques (including RPL & reasonable adjustment) Delivery and assessment staff requirements Resources/infrastructure requirements Assessment validation process PathwaysIndustry consultation

22. Industry Consultation

23. SNR 15.2 Strategies for training and assessment meet the requirements of the relevant Training Package or VET accredited course and have been developed through effective consultation with industry. What the Standards say?

24. Hmmm…. What does ‘effective’ industry consultation mean? How do we do this?

25. How many industry organisations? How many ‘types’ of organisations? How many people within each organisation? Whom should you consult?

26. If only there was a…..

27. 1 Gain industry support 2 Ensure industry needs are met (employability) 3 Provide updates on regulations or other industry relevant legislation 4 Develop knowledge of broad range of industry practices 5 Equipment and machinery standards 6 Assist with placing students in the workplace Industry consultation is needed:

28. Industry…… Employs the graduates See the results in the workplace (ROI) See what’s working and what’s not working Can make suggestions for improvement Can provide invaluable feedback

29. Industry Skills Council Industry Association Industry Networks Industry in your local area Employers Who should you consult?

30. What’s next… document it! Telephone\Face to Face conversations Emails Questionnaire Organisation Name Position What was discussed? What was confirmed? What was suggested? What was amended resulting from feedback?

31. Training program is 12 months long 3 week intensive every 3 months Training topics will be… Test their skills Knowledge test Practice in the workplace Workplace supervisor signs off to say ready Workplace Assessment Skills/Tasks they will be able to do are - ???? - ???? - ???? Do you think that sounds like the skills you need in your workplace? It’s about how we consult Assessment Duration Content Tasks Skills Knowledge

32. Industry Consultation Consultation with industry is about seeking advice, not direction. The RTO is answerable only to the candidate, the Training Package, the AQF and the VQF.

33. Remember…

34. Record Keeping

35. Relevant Standards SNR 23.3 The NVR registered training organisation must retain client records of attainment of units of competency and qualifications for a period of 30 years. Records Management

36. Retention requirements for completed student assessment items An RTO is required to securely retain, and be able to produce in full at audit, if requested to do so, all completed student assessment items for each student, as per the definition, for a period of six months from the date on which the judgement of competence for the student was made. ASQA General Direction funding or Board of Studies may incur further requirements over and above this UPDATED 20 February 2013 UPDATED 20 February 2013

37. AIS NSW RTO Documents Compliance Documents VET Offerings VET Course Documents VQF Publications Overall Management of Documents

38. Assessment

39. Relevant Standards SNR 15.5 Assessment including Recognition of Prior Learning (RPL): a)meets the requirements of the relevant Training Package or VET accredited course; and b)is conducted in accordance with the principles of assessment and the rules of evidence; and c)meets workplace and, where relevant, regulatory requirements; and d)is systematically validated. Assessment

40. Meets the requirements of the Training Package or accredited course Is consistent with the training and assessment strategy Is valid, reliable, flexible and fair (principles of assessment) Focuses on the application of knowledge and skill to the standard of performance required in the workplace Involves the collection of sufficient, valid, authentic and current (rules of evidence) to enable a judgement to be made about whether competency has been attained Confirms that workplace and regulatory requirements are met. Assessment is validated. Guide to compliance - Assessment

41. OR NOT TO That is the question! Assessment

42. Validation Moderation

43. 10 Things to look for - Assessment! 1.Assessment of the whole unit 2.Mapping of the assessment tools 3.Assessing at the correct AQF (& ACSF) level 4.Evidence of validation 5.Assessment tools with clear instructions 6.Assessment tools with clear marking guides 7.Appropriate observation tools 8.Assessors using the guides 9.Assessment tools that cater for learners with special learning needs 10.Evidence of transitioning 1.Assessment of the whole unit 2.Mapping of the assessment tools 3.Assessing at the correct AQF (& ACSF) level 4.Evidence of validation 5.Assessment tools with clear instructions 6.Assessment tools with clear marking guides 7.Appropriate observation tools 8.Assessors using the guides 9.Assessment tools that cater for learners with special learning needs 10.Evidence of transitioning

44. 1. Assessment of the whole unit Does the assessment evidence address all of the unit requirements? The performance criteria This is a test of the (content) validity of the assessment tools and the assessment process. The required skills and knowledge The critical aspects for assessment

45. Are actual questions mapped? Are specified sections of tasks mapped? Are observable behaviours mapped? 2. Mapping of the assessment tools Are components of the each of the assessment tools mapped to relevant unit components? This tests the (content) validity of the assessment tools For example:

46. 2. Mapping to the PCs

47. 2. Mapping to the knowledge, skills and critical aspects

48. 3. Assessing at the correct AQF level

49. Basic factual and procedural knowledge. 3. Assessing at the correct AQF level e.g. AQF Level 2 NOT Write a 1000 word essay. Conduct research on this topic Design next week’s menu for this restaurant

50. NOT: analyse distinguish examine compare contrast investigate categorise identify NOT: analyse distinguish examine compare contrast investigate categorise identify 3. AQF Level 2 Significance of Verbs – AQF Levels 1 and 2 This tests the (construct) validity of the assessment tools VERBS such as: tell list describe relate locate write find name VERBS such as: tell list describe relate locate write find name

51. 3. Consider LL & N needs What language, literacy and numeracy activities would be required in the workplace? oral or written reports forms, instructions, labels, reports, etc. measuring, scheduling, etc. etc., etc., etc. oral or written reports forms, instructions, labels, reports, etc. measuring, scheduling, etc. etc., etc., etc. Have we checked the LL&N skills: required in the workplace? of our students? of our trainers / assessors? Have we checked the LL&N skills: required in the workplace? of our students? of our trainers / assessors? Have we taken account of these in: the teaching materials? the assessment activities? Have we taken account of these in: the teaching materials? the assessment activities? This tests the (face and content) validity and the fairness of the assessment tools. It also alerts us to the significance of the Australian Core Skills Framework (ACSF). This tests the (face and content) validity and the fairness of the assessment tools. It also alerts us to the significance of the Australian Core Skills Framework (ACSF).

52. 4. Evidence of validation Assessment tools must be validated with industry Do they make sense to people engaged in the industry? Assessment tools must be validated with industry Do they make sense to people engaged in the industry? Evidence such as: Minutes of meetings Diary records of discussions Email confirming consultation Etc., etc., etc : Evidence such as: Minutes of meetings Diary records of discussions Email confirming consultation Etc., etc., etc : There should also be evidence that the assessment tools have been validated internally. This tests the (face) validity of the assessment tools

53. 5. Assessment tools with clear instructions There should be instructions for the student. WHAT has to be done WHERE WHY and HOW There should be instructions for the student. WHAT has to be done WHERE WHY and HOW There should also be instructions for the trainer. Under what conditions – duration; environment etc? What resources? What support is allowable? There should also be instructions for the trainer. Under what conditions – duration; environment etc? What resources? What support is allowable? This tests the reliability of the assessment tools

54. 6. Assessment tools with clear marking guides What are the “expected” responses? What constitutes a “satisfactory” response? Are “product” requirements spelled out? What are the “expected” responses? What constitutes a “satisfactory” response? Are “product” requirements spelled out? This assists the reliability of the assessment tools and their fairness.

55. Is each item directly observable? Is each item focussed on one behaviour? Are there descriptors of what constitutes satisfactory behaviour? Is there advice for the assessor about what to do if all items are not observed? 7. Appropriate observation tools Do observation tools turn unit requirements into observable behaviours – not just a cut and paste from the PCs? This tests the validity and reliability of the observation assessment.

56. 8. Assessors using the marking guides Is there evidence of consistency among assessors? Is there evidence of rigour in the assessment process? Is there evidence of assessors making judgements and signing off on these? Is there evidence of the provision of feedback to students that takes account of the marking guides? Is there evidence of consistency among assessors? Is there evidence of rigour in the assessment process? Is there evidence of assessors making judgements and signing off on these? Is there evidence of the provision of feedback to students that takes account of the marking guides? This tests the reliability and fairness of the assessment process.

57. 9. Assessment tools that cater for learners with special learning needs Do the tools allow for some learners to be assessed differently from others in the group? E.g. Allowing a learner to respond orally rather than in writing Allowing a learner more time or access to a particular resource. E.g. Allowing a learner to respond orally rather than in writing Allowing a learner more time or access to a particular resource. Do the assessment tools define “reasonable adjustment”? This demonstrates the flexibility and fairness of the assessment tools and processes.

58. 10. Evidence of transitioning When the Training Package has been updated or replaced, is there evidence that required transition processes are in place? E.g. Different Training and Assessment Strategies for New and Continuing cohorts Awareness of new ASQA “teach out” requirements E.g. Different Training and Assessment Strategies for New and Continuing cohorts Awareness of new ASQA “teach out” requirements

59. Summarising – 10 Things to look for! 1.Assessment of the whole unit 2.Mapping of the assessment tools 3.Assessing at the correct AQF (& ACSF) level 4.Evidence of validation 5.Assessment tools with clear instructions 6.Assessment tools with clear marking guides 7.Appropriate observation tools 8.Assessors using the guides 9.Assessment tools that cater for learners with special learning needs 10.Evidence of transitioning 1.Assessment of the whole unit 2.Mapping of the assessment tools 3.Assessing at the correct AQF (& ACSF) level 4.Evidence of validation 5.Assessment tools with clear instructions 6.Assessment tools with clear marking guides 7.Appropriate observation tools 8.Assessors using the guides 9.Assessment tools that cater for learners with special learning needs 10.Evidence of transitioning

60. How to avoid non-compliances

61. Strong Industry Relationship Compliance Quality Outcomes It isn’t only about an audit…

62. Any questions?