Senior External Verifier Event August 2016

Senior External Verifier Event August 2016 Martin Chisholm Head of Service, HN & VQ

Senior External Verifier Event August 2016 Juliette McGinley Quality Assurance Logistics Team

Agenda 9.00 – 9.30 Registrations 9.30 – 9.45 Introduction and Welcome 9.45 – 10.00 Deputy Lead Verifier Role 10.00 – 11.15 Workshop 1: External Verification of HN and SVQ Qualifications 11.15 – 11.30 Tea and Coffee 11.30 - 11.45 Welcome and introduction Presentation: Digital Evidence 11.45 – 12.05 Breakout session 12.05– 12.20 Findings from breakout session 12.20 –12.30 Open Forum 12.30 – 1.30 Lunch 1.30 – 2.00 CPD 2.00 – 2.30 Operational Overview 2.30 – 2.45 Tea/Coffee 2.45 – 3.30 Workshop 2: QASP Review and Feedback - Juliette McGinley 3.30 – 3.45 QASP process Review and AOS Update 3.45 – 4.00 Summary and Close

Senior External Verifier Update Event Introducing the Deputy Lead Verifier Role Deputy Lead Verifiers will provide SEVs with advice and support – a “go to” point for all things related to Quality Assurance Criteria and External Verification All Deputy Lead Verifiers are practising SEVs/SWPs and have a proven track record of supporting their own teams *

Senior External Verifier Update Event Introducing the Deputy Lead Verifier Role Deputy Lead Verifiers will be allocated a specific group of SEVs DLVs will contact you after todays event to discuss support requirements QA Logistics Officers will continue to provide support in terms of logistics and specific arrangements for QA activity *

Senior External Verifier Event August 2016 David Pirnie Lead Verifier

Criterion 2.4 - review of resources - what systems verifiers look for/advise Initial review is part of the approval process. Centres’ systems should show all relevant resources are in place before a qualification runs. Ongoing reviews may be covered in one procedure or may come under a range of activities e.g. standardisation meetings and/or internal verification.

Criterion 2.4 – review of resources (assessment environments, assessment procedures, equipment, learning and assessment materials) We as EVs should first of all establish what a centre’s procedure is for review, and then seek evidence of implementation. Sources of evidence: approval – new Units/Awards (initial) changes to the qualifications being offered internal and/or external verification feedback following default procedures e.g. an annual review for roll- on/off SVQs, or the beginning or end of semesters for HNs in colleges. Site checklists are often a source of evidence of review of assessment environments and equipment.

Inclusion of adjectives in rationale sections of guidance Criterion 3.3 - scheduled contact that is effective. Criterion 4.2 - applying assessment and verification procedures consistently, in full and effectively.

Criterion 4.9 – feedback from Qualification Verifiers ‘Centres must provide evidence of having disseminated the feedback contained in Qualification Verifier reports in a systematic way, and wherever there are required actions………..’

Workshop 1 It is important that we centrally agree responses to the most common scenarios we as External Verifiers come across during visiting verification. To this end, please discuss the attached draft guidance, developed as a result of the feedback received from the workshop we held at the SEV meeting held in April earlier this year. We would like you to comment on the clarity of wording/phrasing. This guidance will eventually sit on the Quality Assurance Appointee Website. Time: One hour

Senior External Verifier Event August 2016 Tea Break 15 mins

Digital Assessment Evidence Senior External Verifier Event August 2016 Digital Assessment Evidence Martyn Ware Veronica Harris Assessment Development and Delivery

Purpose Update on work on digital assessment evidence Gather feedback on this work to date and proposed next steps Ensure approaches we are developing are fit for purpose for SQA and centres

Drivers Scottish Government’s commitment to digital SQA’s corporate commitment to ‘….digitally transform our organisation…’ Responding to growing practice in and demand from centres Ensuring EVs and centres have effective procedures and support in this area

Customer demand ‘It would be great if SQA was to say that from a specific date in the future your default would be digital evidence. That would help to drive a lot of the things that we as a college are trying to achieve.’ College Quality Manager, March 2014

Scotland’s Digital Future Scottish Government initiative “The Scottish Government is committed to increasing digital participation and ensuring that all of Scotland is able to enjoy the social, cultural and economic benefits of the internet. …..the people of Scotland, whatever their location, age or income, can choose digital first”. 

Digital First How do we move forward? Take stock Evaluate Plan Implement

Work to date E-moderation: Guidelines for FE staff and external moderators (2004) Guide to Visiting HN and Vocational Qualification Verification for Centres 2015 – 2018 Engagement with Colleges eAssessment Group and Colleges Quality Forum

Pilot Study 2014 – 2015 Landscape review Pilot study – 7 centres Review and 7 recommendations “The aim should be to develop a principles and guidance based structured methodology to storage and access to digital evidence that supports centres rather than being prescriptive”.

Draft Principles

Evidence Generation Strategy SQA’s principles of assessment and QA must be adhered to regardless of the format of the evidence. Systems selected for retention of learner evidence should not contain any features which might adversely affect learners Security Evidence created and stored digitally should be generated and retained under secure conditions. SQA’s principles:- Valid Reliable Practicable Equitable and fair Systems selected It is the responsibility of the centre to take appropriate steps to ensure that learners’ access to technical solutions to support their assessment for an SQA qualification does not render them potentially subject to any harm and that the systems are sufficiently secure that no unauthorised access to the learners’ personal details is permissible. Social Media example. Use example of secret or closed group. Security Evidence held securely through all stages of its journey

Security cont’d Training and Support 4. Data security policies should be in place which encompass digital evidence, including digital assessment evidence. 5. Systems and resources should be in place to allow for secure storage and management of digital evidence. Training and Support 6. Clear guidance and support should be available for learners, assessors and verifiers on the use of the system(s) including, for example, directory structures, naming conventions etc. 4. It is very important that Centres comply with Data Protection legislation and have in place documented policies on secure handling of digital materials. 5. Centres are free to select from a range of IT systems for storage and management of digital evidence. Systems should ideally be available 24/7, be mobile friendly and accessible from outside the centre. Can you recommend any case studies that could be included in the P&G? Training and Support Centres should maintain a digital record with consistent details for each of the systems in use. This could take the form of instructional videos, handbooks, leaflets etc. which can be readily accessed. There should also be named technical points of contact for any issues arising from use of the system(s), to minimise disruption.

SQA’s principles of assessment and quality assurance processes must be adhered to regardless of the format of the evidence. Verification Wherever possible, digital evidence should be reviewed for assessment and QA in situ: it should not be moved by email or other means. Evidence which is created digitally should remain digital during its journey through assessment and QA processes. Where available, templates should be used to structure and format evidence. 7. Where centres have in place controlled access arrangements, these should be used as the preferred option for allowing the inspection of evidence for assessment decisions, grading and verification and there should be no need for data to be exported from the secure storage. If possible, there should be the facility to allow external verification to take place remotely and securely. 8 In most cases there should be no need for materials to be printed out for scrutiny. There will be exceptions to this rule, for example the quality of a printed photograph when the skill of the learner to produce a high resolution graphic is being tested. 9 At the time of writing SQA is collaborating with colleges to develop templates for use in the Mahara open source e-portfolio.

2016 Survey of EVs - Results Typical response rate for online surveys 30 – 40% Our response - 56% Suggests that this is an important issue for EVs Thanks to all who participated!

Do you have experience of using digital technologies in these areas?

If you require training in using digital assessment evidence what format of training do you think would be most effective and appropriate?

Questions for Breakout Session Principles and Guidance How useful is this document as a starting point? What challenges do you see in implementing this approach? Do you think there is anything missing? What additional support and guidance would you and centres need to implement the Principles and Guidance? State of readiness What is the state of readiness in your subject area for digital verification? 0 = not ready and never will be 1 = getting there but still a long way to go 2 = half way there 3 = almost completely digital

Findings from Breakout Session

Open Forum

Conclusion Generation of digital evidence by learners will continue to grow This offers benefits for learners, centres and SQA Approaches to verification need to keep pace Dialogue with centres and EVs is key to ensuring our approaches are fit for purpose

Contact Details Martyn Ware martyn.ware@sqa.org.uk Veronica Harris veronica.harris@sqa.org.uk

Senior External Verifier Event August 2016 Lunch 1 Hour

Senior External Verifier Event August 2016 Anne Andrew Recruitment and Training Manager Appointee Management

Overview of Presentation External Verifier Key Performance Measures Continuing Professional Development

External Verifier – Key Performance Measures (KPMs) On an annual basis External Verifier’s will be measured against KPMs and informed by exception of the outcome in relation to their own performance.

External Verifier – KPMs Demonstrate an up to date and sound knowledge of the occupational standards and qualifications/courses/units and make sound judgements about the quality of assessment and the assessment process. Undertake required training to demonstrate current understanding of SQAs QA policies and processes that are relevant to the QA activity.

External Verifier –KPMs Complete scheduled allocation of QA activity within conditions and timescales for SQA. Complete QA reports as per SQA quality standards and within agreed timescales

External Verifier –KPMs Engage with all members of the team, SQA centre’s personnel and with all members of SQA staff displaying SQAs corporate values. Submit a current Personal Development Plan (PDP) and CPD record within agreed timescales where appropriate.

CPDR It’s about proving how you keep up to date …. the process of tracking and documenting the skills, knowledge and experience that you gain both formally and informally as you work, beyond any initial training. It's a record of what you experience, learn and then apply.

CPDR We require a minimum of 2 entries: How you kept your occupational expertise up-to-date How you standardised in relation to your area of expertise or on SQA policy and procedure.

Any Questions?

Senior External Verifier Event August 2016 Juliette McGinley Quality Assurance Logistics Team

Operational Aspects - Criteria Criteria 2.1 For HN and NQ –if there are no specific assessor verifier occupational/subject experience/CPD stipulated in awarding/regulatory body docs this material should be marked not verified If offered qualifications/CPD in relation to HN/NQ units this can be placed under good practice For SVQ/regulated qualifications, it is a requirement to measure at 2.1

Operational Aspects - General UHI – Verification Arrangements Visits and deadlines at end of session College Site and Timescales on allocations EV Feedback and proportionate sanctions SEV Sign off – timeframes for report sign off

Operational Aspects - General EV Induction documents on Appointee Website Behaviours Approach Awareness Working Group for IAR Reports

Status EV responsibility SEV responsibility SQA Responsibility   Visit report Created To submit Report to QV Visit report sent to QV To forward report to SEV for review Report Awaiting Senior Review Review and approve or return to verifier Report returned to Verifier Review and amend report and sent back to QV team Approved by Senior Send to centre Evidence report sent to QV

QASP – Submission Deadline & Payment Quarter Deadline Date Payment Date 1 25-Oct 15th November 2 25th January 15th February 3 21st April 15th May 4 24th July 15th August

Workshop 2 It is important that we agree the content of the QASP before its release. I would ask that you work in a group to review and comment on the QASP documentation. All comments and feedback will be collected at the end of the workshop and discussed before end of today’s session. Time: 1 Hour Feedback will be gathered by an allocated group facilitator.

SQA’s Awards Operating System What it is and why its being undertaken How it will impact on Quality Assurance How it will impact on Verifiers What the timescales are

Any Questions?