Consistency of Assessment (Validation) Webinar – Part 1 Renae Guthridge WA Training Institute (WATI)

What is your understanding of validation?

Describe an assessment validation process in which you have been involved. – What was the purpose? – Who was involved? – What was validated? – Were any problems identified? – How were the findings reported and followed-up?

What is assessment validation all about? What are the purposes of assessment validation?

Contribute to RTO quality assurance Address any identified area of risk in assessment practice and quality Demonstrate compliance against Standards for RTOs PURPOSES Ensure assessments meet the evidence requirements of the competency standards or other assessment benchmarks Quality review of the assessment process Improve assessment practices Evaluate the quality of assessment tools Provide professional development Increase assessor confidence Determine whether different assessors using the same tools collect the same types and levels of evidence Determine whether different assessors interpret the same evidence similarly Determine whether assessment decisions reflect the principles of assessment and rules of evidence. To ensure our assessment judgements are consistent

What is assessment validation all about? What kinds, types and methods of assessment documentation do we need to validate?

Learning based assessment Direct RPL KINDS, TYPES & METHODS OF ASSESSMENT DOCS Supplementary Indirect Instruments Assessment Strategies Marking Guide Instructions for Candidate Observation; Q&A; Case Studies; Portfolios; Projects… Instructions for Assessor Mapping Documents Assessment Plans

What is assessment validation all about? Who wants assurance of quality?

Learners / Candidates WHO WANTS QUALITY? Assessors / Trainers Technical / Subject Matter Experts Industry / Employers Registering / Course Accreditation Bodies

What is assessment validation all about? What are our benchmarks of quality?

Standards for RTOs BENCHMARKS OF QUALITY AQF Principles of Assessment TAE10 Training and Education Training Package…assessment units. Training Packages / Accredited Courses Rules of Evidence

What is assessment validation all about? What aspects of the assessment process need to be validated?

Planning assessment processes (industry consultation, links with delivery, TP/AC requirements, recognition of client needs) ASPECTS TO VALIDATE Delivery of assessment services (modes, plans, resources) Processes for the review of assessment services (validation and continuous improvement) Judgement making procedures Evidence gathering processes and tools (on- the-job, workplace, simulation) Support for learners / candidates (Information, feedback, flexibility, reasonable adjustment, appeals) Management of assessors / evidence gatherers (selection, supervision, professional development)

What is assessment validation all about? How do we assess quality and make improvements?

Assessors working in collaboration to review, compare and evaluate assessment processes and outcomes ASSESSING QUALITY It’s systematically planned and implemented Verifying consistency of evidence and judgements made against benchmark Foxes and Vixen Can occur before, during and after assessment Validating methods, tools and interpretation of evidence

Foxes and a Vixen Principles of Assessment Rules of Evidence Four Very Rampant Foxes A Satisfied Cunning Vixen

Foxes and a Vixen Principles of Assessment Rules of Evidence Flexibility Validity Reliability Fairness Authenticity Sufficiency Currency Validity

Foxes and a Vixen Flexibility

Foxes and a Vixen Validity

Foxes and a Vixen Reliability

Foxes and a Vixen Fairness

Foxes and a Vixen Authenticity

Foxes and a Vixen Sufficiency

Foxes and a Vixen Currency

Foxes and a Vixen Validity

Standards for RTOs 2015 What do you know? What is different to AQTF / VET Quality Framework?

Standards for RTOs 2015 Validation is the quality review of the assessment process. Validation involves checking that the assessment tool/s produce/s valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the training package or VET accredited courses are met. It includes reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes and acting upon such recommendations.

Standards for RTOs 2015 Standard 1 – The RTO’s training and assessment strategies and practices are responsive to industry and learner needs and meet the requirements of training packages and accredited courses comprehensive plan of systematic validation risk-based approach to developing the plan Trainers and assessors involved in training and assessment may also participate in the validation process 1.9. The RTO implements a plan for ongoing systematic validation…for each training product on the RTO’s scope of registration: a) when assessment validation will occur; b) which training products will be the focus of the validation; c) who will lead and participate in validation activities; and d) how the outcomes…will be documented and acted upon.

Standards for RTOs …each training product is validated at least once every five years, with at least 50% of products validated within the first three years of each five year cycle, taking into account… risks…on the RTO’s scope…including those risks identified by the VET Regulator …systematic validation…is undertaken by one or more persons who are not directly involved in the particular instance of delivery and assessment of the training product being validated, and who collectively have: a) vocational competencies and current industry skills relevant to the assessment being validated; b) current knowledge and skills in vocational teaching and learning; and c) the training and assessment qualification or assessor skill set referred to in Item 1 or 3 of Schedule 1. Industry experts may be involved in validation to ensure there is the combination of expertise set out in (a) to (c) above.

Standards for RTOs 2015 Independent validation of training and assessment qualifications From 1 January 2016, to deliver any AQF qualification or assessor skill set from the Training and Education Training Package (or its successor), the RTO must have undergone an independent validation… Independent validation means the validation is carried out by a validator/s who: a) are not employed or subcontracted by the RTO to provide training and assessment; and b) have no other involvement or interest in the operations of the RTO. Standard 2. The operations of the RTO are quality assured The RTO: b) systematically evaluates and uses the outcomes of the evaluations to continually improve the RTO’s training and assessment strategies and practices. Evaluation information includes but is not limited… validation outcomes...

Standards for RTOs 2015 What do you think? What impact is this going to have on consistency of assessment? What will need to change in your RTO?

RQI Write down three things you recall about today’s webinar. Write down two questions you have about today’s webinar. Write down one interesting point about today’s webinar.

Tomorrow…Part 2… Tips and techniques for: Meeting principles of assessment Ensuring rules of evidence are met Developing criteria for reviewing, comparing and evaluating assessment processes All pictures in these slides were taken from