1. Senior External Verifier Event August 2016

2. Senior External Verifier Event August 2016
Martin Chisholm
Head of Service, HN & VQ

3. Senior External Verifier Event August 2016
Juliette McGinley
Quality Assurance Logistics Team

4. Agenda 9.00 – 9.30 Registrations 9.30 – 9.45 Introduction and Welcome
9.45 – Deputy Lead Verifier Role
10.00 – Workshop 1: External Verification of HN and SVQ Qualifications
11.15 – Tea and Coffee
Welcome and introduction
Presentation: Digital Evidence
11.45 – Breakout session
12.05– Findings from breakout session
12.20 – Open Forum
12.30 – Lunch
1.30 – CPD
2.00 – Operational Overview
2.30 – Tea/Coffee
2.45 – Workshop 2: QASP Review and Feedback - Juliette McGinley
3.30 – QASP process Review and AOS Update
3.45 – 4.00 Summary and Close

5. Senior External Verifier Update Event
Introducing the Deputy Lead Verifier Role
Deputy Lead Verifiers will provide SEVs with advice and support – a “go to” point for all things related to Quality Assurance Criteria and External Verification
All Deputy Lead Verifiers are practising SEVs/SWPs and have a proven track record of supporting their own teams *

6. Senior External Verifier Update Event
Introducing the Deputy Lead Verifier Role
Deputy Lead Verifiers will be allocated a specific group of SEVs
DLVs will contact you after todays event to discuss support requirements
QA Logistics Officers will continue to provide support in terms of logistics and specific arrangements for QA activity *

7. Senior External Verifier Event August 2016
David Pirnie
Lead Verifier

8. Criterion 2.4 - review of resources - what systems verifiers look for/advise
Initial review is part of the approval process. Centres’ systems should show all relevant resources are in place before a qualification runs.
Ongoing reviews may be covered in one procedure or may come under a range of activities e.g. standardisation meetings and/or internal verification.

9. Criterion 2.4 – review of resources (assessment environments, assessment procedures, equipment, learning and assessment materials)
We as EVs should first of all establish what a centre’s procedure is for review, and then seek evidence of implementation.
Sources of evidence:
approval – new Units/Awards (initial)
changes to the qualifications being offered
internal and/or external verification feedback
following default procedures e.g. an annual review for roll-
on/off SVQs, or the beginning or end of semesters for HNs in
colleges.
Site checklists are often a source of evidence of review of assessment environments and equipment.

10. Inclusion of adjectives in rationale sections of guidance
Criterion scheduled contact that is effective. Criterion applying assessment and verification procedures consistently, in full and effectively.

11. Criterion 4.9 – feedback from Qualification Verifiers
‘Centres must provide evidence of having disseminated the feedback contained in Qualification Verifier reports in a systematic way, and wherever there are required actions………..’

12. Workshop 1
It is important that we centrally agree responses to the most common scenarios we as External Verifiers come across during visiting verification. To this end, please discuss the attached draft guidance, developed as a result of the feedback received from the workshop we held at the SEV meeting held in April earlier this year. We would like you to comment on the clarity of wording/phrasing. This guidance will eventually sit on the Quality Assurance Appointee Website. Time: One hour

13. Senior External Verifier Event August 2016
Tea Break
15 mins

14. Digital Assessment Evidence
Senior External Verifier Event August 2016
Digital Assessment Evidence
Martyn Ware
Veronica Harris
Assessment Development and Delivery

15. Purpose Update on work on digital assessment evidence
Gather feedback on this work to date and
proposed next steps
Ensure approaches we are developing are fit for
purpose for SQA and centres

16. Drivers Scottish Government’s commitment to digital
SQA’s corporate commitment to ‘….digitally transform our organisation…’
Responding to growing practice in and demand from centres
Ensuring EVs and centres have effective procedures and support in this area

17. Customer demand
‘It would be great if SQA was to say that from a specific date in the future your default would be digital evidence. That would help to drive a lot of the things that we as a college are trying to achieve.’
College Quality Manager, March 2014

18. Scotland’s Digital Future
Scottish Government initiative
“The Scottish Government is committed to increasing digital participation and ensuring that all of Scotland is able to enjoy the social, cultural and economic benefits of the internet. …..the people of Scotland, whatever their location, age or income, can choose digital first”. 

19. Digital First How do we move forward? Take stock Evaluate Plan
Implement

20. Work to date
E-moderation: Guidelines for FE staff and external moderators (2004)
Guide to Visiting HN and Vocational Qualification Verification for Centres 2015 – 2018
Engagement with Colleges eAssessment Group and Colleges Quality Forum

21. Pilot Study 2014 – 2015 Landscape review Pilot study – 7 centres
Review and 7 recommendations
“The aim should be to develop a principles and guidance based structured methodology to storage and access to digital evidence that supports centres rather than being prescriptive”.

22. Draft Principles

23. Evidence Generation Strategy
SQA’s principles of assessment and QA must be adhered to regardless of the format of the evidence.
Systems selected for retention of learner evidence should not contain any features which might adversely affect learners
Security
Evidence created and stored digitally should be generated and retained under secure conditions.
SQA’s principles:-
Valid
Reliable
Practicable
Equitable and fair
Systems selected
It is the responsibility of the centre to take appropriate steps to ensure that learners’ access to technical solutions to support their assessment for an SQA qualification does not render them potentially subject to any harm and that the systems are sufficiently secure that no unauthorised access to the learners’ personal details is permissible. Social Media example.
Use example of secret or closed group.
Security
Evidence held securely through all stages of its journey

24. Security cont’d Training and Support
4. Data security policies should be in place which encompass digital evidence, including digital assessment evidence. 5. Systems and resources should be in place to allow for secure storage and management of digital evidence.
Training and Support
6. Clear guidance and support should be available for learners, assessors and verifiers on the use of the system(s) including, for example, directory structures, naming conventions etc.
4. It is very important that Centres comply with Data Protection legislation and have in place documented policies on secure handling of digital materials.
5. Centres are free to select from a range of IT systems for storage and management of digital evidence. Systems should ideally be available 24/7, be mobile friendly and accessible from outside the centre.
Can you recommend any case studies that could be included in the P&G?
Training and Support
Centres should maintain a digital record with consistent details for each of the systems in use. This could take the form of instructional videos, handbooks, leaflets etc. which can be readily accessed. There should also be named technical points of contact for any issues arising from use of the system(s), to minimise disruption.

25. SQA’s principles of assessment and quality assurance processes must be adhered to regardless of the format of the evidence.
Verification
Wherever possible, digital evidence should be reviewed for assessment and QA in situ: it should not be moved by or other means.
Evidence which is created digitally should remain digital during its journey through assessment and QA processes.
Where available, templates should be used to structure and format evidence.
7. Where centres have in place controlled access arrangements, these should be used as the preferred option for allowing the inspection of evidence for assessment decisions, grading and verification and there should be no need for data to be exported from the secure storage.
If possible, there should be the facility to allow external verification to take place remotely and securely.
8 In most cases there should be no need for materials to be printed out for scrutiny. There will be exceptions to this rule, for example the quality of a printed photograph when the skill of the learner to produce a high resolution graphic is being tested.
9 At the time of writing SQA is collaborating with colleges to develop templates for use in the Mahara open source e-portfolio.

26. 2016 Survey of EVs - Results
Typical response rate for online surveys 30 – 40%
Our response - 56%
Suggests that this is an important issue for EVs
Thanks to all who participated!

27. Do you have experience of using digital technologies in these areas?

28. If you require training in using digital assessment evidence what format of training do you think would be most effective and appropriate?

29. Questions for Breakout Session
Principles and Guidance
How useful is this document as a starting point?
What challenges do you see in implementing this approach?
Do you think there is anything missing?
What additional support and guidance would you and centres need to implement the Principles and Guidance?
State of readiness
What is the state of readiness in your subject area for digital verification?
0 = not ready and never will be
1 = getting there but still a long way to go
2 = half way there
3 = almost completely digital

30. Findings from Breakout Session

31. Open Forum

32. Conclusion
Generation of digital evidence by learners will continue to grow
This offers benefits for learners, centres and SQA
Approaches to verification need to keep pace
Dialogue with centres and EVs is key to ensuring our approaches are fit for purpose

33. Contact Details Martyn Ware martyn.ware@sqa.org.uk
Veronica Harris

34. Senior External Verifier Event August 2016
Lunch
1 Hour

35. Senior External Verifier Event August 2016
Anne Andrew
Recruitment and Training Manager
Appointee Management

36. Overview of Presentation
External Verifier Key Performance Measures
Continuing Professional Development

37. External Verifier – Key Performance Measures (KPMs)
On an annual basis External Verifier’s will be measured against KPMs and informed by exception of the outcome in relation to their own performance.

38. External Verifier – KPMs
Demonstrate an up to date and sound knowledge of the occupational standards and qualifications/courses/units and make sound judgements about the quality of assessment and the assessment process.
Undertake required training to demonstrate current understanding of SQAs QA policies and processes that are relevant to the QA activity.

39. External Verifier –KPMs
Complete scheduled allocation of QA activity within conditions and timescales for SQA.
Complete QA reports as per SQA quality standards and within agreed timescales

40. External Verifier –KPMs
Engage with all members of the team, SQA centre’s personnel and with all members of SQA staff displaying SQAs corporate values.
Submit a current Personal Development Plan (PDP) and CPD record within agreed timescales where appropriate.

41. CPDR It’s about proving how you keep up to date
…. the process of tracking and documenting the skills, knowledge and experience that you gain both formally and informally as you work, beyond any initial training. It's a record of what you experience, learn and then apply.

42. CPDR We require a minimum of 2 entries:
How you kept your occupational expertise up-to-date
How you standardised in relation to your area of expertise or on SQA policy and procedure.

43. Any Questions?

44. Senior External Verifier Event August 2016
Juliette McGinley
Quality Assurance Logistics Team

45. Operational Aspects - Criteria
Criteria 2.1 For HN and NQ –if there are no specific assessor verifier occupational/subject experience/CPD stipulated in awarding/regulatory body docs this material should be marked not verified
If offered qualifications/CPD in relation to HN/NQ units this can be placed under good practice
For SVQ/regulated qualifications, it is a requirement to measure at 2.1

46. Operational Aspects - General
UHI – Verification Arrangements
Visits and deadlines at end of session
College Site and Timescales on allocations
EV Feedback and proportionate sanctions
SEV Sign off – timeframes for report sign off

47. Operational Aspects - General
EV Induction documents on Appointee Website
Behaviours
Approach
Awareness
Working Group for IAR Reports

48. Status EV responsibility SEV responsibility SQA Responsibility
Visit report Created
To submit Report to QV
Visit report sent to QV
To forward report to SEV for review
Report Awaiting Senior Review
Review and approve or return to verifier
Report returned to Verifier
Review and amend report and sent back to QV team
Approved by Senior
Send to centre
Evidence report sent to QV

49. QASP – Submission Deadline & Payment
Quarter
Deadline Date
Payment Date 1
25-Oct
15th November 2
25th January
15th February 3
21st April
15th May 4
24th July
15th August

50. Workshop 2
It is important that we agree the content of the QASP before its release. I would ask that you work in a group to review and comment on the QASP documentation. All comments and feedback will be collected at the end of the workshop and discussed before end of today’s session. Time: 1 Hour Feedback will be gathered by an allocated group facilitator.

51. SQA’s Awards Operating System
What it is and why its being undertaken
How it will impact on Quality Assurance
How it will impact on Verifiers
What the timescales are

52. Any Questions?