1. Welcome To The HN & Vocational Senior External Verifier Event August 29th 2013

2. Agenda

3. QA Past, Present and Future….

4. And then……………

5. Your Role
‘A Senior External Verifier provides leadership and support to a team of external verifiers through the processes in accordance with SQA policy and instruction.’

6. Alastair Morton Operations Planning Manager
(HN and Vocational Qualifications)

7. Topics Review of Verification Activity 2012 – 13 College Review
Future of Reporting

8. Comments:
75 Holds reported (1st Aug 2012 – 31st July 2013)
54% of Allocated Events were Successful
38% of Events were updated as Not Running vs 36% Session
Includes visits conducted under new approach
Profile excludes China and International Centres

9. How SQA plans to tackle this
Colleges
Provided Verification Plans to all other Scotland’s Colleges
Process worked well with 10 partner colleges over last 2 sessions
Colleges highlighting Not Running – internally select for accreditation purposes – SQA will set to Not Running
First time this process has been considered
ETPs
Reviewed entry patterns for all approved SVQs over last 2 years
Allowed SQA to identify areas of candidate activity
Also allowed SQA to target individual Work Assessed units
Will select -> de-select as per College process
Longer Term
IQA project – significant expenditure in designing complete system

10. Profile of Verification Activity

11. Comment:
150 more Successful events vs session (2112 vs 1962)
Includes visits conducted under new approach
Profile excludes China and International Centres.

12. Comparing vs :
Type
Diff
SVQ
963
1049
-86
HN G/U
378
376 2 HN
358
357 1

13. Impact of Verification Profile
Resource impact on Centres
Resource impact on Appointees
Missing assessment windows – (e.g. Semester 1)
Availability of candidate evidence at Centres may drive part of this profile
Moving away from single allocation period -> quarterly allocations
React to changing developments at centre / across subject area / industry
Allocations will be issued throughout the session -> ongoing verification
Initial concentration on ETPs – assessment often throughout year

14. Scotland’s College Update

15. Scotland’s Colleges Update
Verification Planning
All Scotland’s Colleges – met / discussed finalised Verification plan
Real focus on Entry information
Discussed verification areas which have over / not verified
Plans are agreed in partnership
Swing from HN qualification to NCs
College Mergers
Number of Colleges in process of merging
Sensitive to the transitional period, particularly around availability of candidate evidence across sites
Majority of merging centres have yet to finalise single SEED number and will be operating under original numbers for next 6 – 12 months

16. Challenges of Changing Quality Assurance Approaches in Merging Colleges
Scenario 1
Scenario 2
Scenario 3
Scenario 4
New
New
New
New
Old
Old
New
New
Old
New
New
New
Old
New
Old

17. College Feedback Verifiers are not liaising with the SQA Coordinator
Some Feedback sessions could be more comprehensive
Sampling needs to be more widespread
Visit Plans are not being provided ahead of visit.
Verifiers using a College address and not a private one….
Some Verifiers are not using standard templates for contacting / planning
Visit plans are issued less than 2 weeks before visit
More Comments need to be provided against Criteria

18. HN and VQ Approach to Quality Assurance Walkthrough
Preparing
Planning Verification
Carrying Out
Request Information
Send Visit Plan Docs
Prepare
Accept
EV+
Select Sample
Prepare for Visit
Contact
Agree Agenda
Conduct Sampling
General Points
Completing Report
& Follow-Up
Communicating Decision
Judge sufficiency
Follow-up
Flowchart
Complete Report
Plan Feedback
Reporting Flowchart
Explain Decision

19. Accepting Centre Allocation on EV+
Planning
Accepting Centre Allocation on EV+
Do’s
Accept allocation within 14 days
If you decline a visit, please provide a reason
Once you have registered, please remember keep your password safe. SQA will not provide you with a password.
Verification allocations will only be provided via EV+ system
Accept
You will receive notification of allocation from EV+ via .
Decline
If you decline the allocated visit, please provide a reason
If you do not accept your allocation within 14 days, an automatic reminder will be sent by EV+
SQA may re-allocate to another Verifier if no response is received within this timescale
Please highlight any conflict of interest to QA Logistics Team
Don’ts

20. Plan to deliver balanced feedback
Communicating Decision
Plan to deliver balanced feedback
Communicate verification decisions on criteria compliance and overall risk rating.
Highlight positive aspects of visit, including good practice.
Think clearly – what really is Good Practice ?
No specific guidance on length of time required for feedback.
Do’s
Consider confidentiality and location of feedback session
Give balanced feedback
Recognise centre’s efforts where sufficient evidence provided.
Take time out between making the decision and giving feedback
Don’ts
Focus solely on Action Points

21. Criteria Reporting – The Way ahead

22. Evolution of Verification Outcome Reporting
Pre 2010
SEV Report
Count of Holds
Activity
EV8A Reports
Count of Holds
Activity
EV8A Reports
SEV Report
Senior Review
2012-Now
New Verification Reports
Activity
Senior Review
SEV Report
Detailed Analysis

23. Detailed Analysis

24. ONLINE WEB FORMATTED REPORT
The Way Forward…
Activity
Database (QAMS)
ONLINE WEB FORMATTED REPORT
Benefits from this session:
Real time information available – easy to extract and disseminate
Understand “pockets” of non-compliance quickly
Criteria compliance reporting supported by narrative
Provide information on more frequent basis
Share with Centres / SQA teams and Appointees to highlight, understand and fix
Explore use of Social Media

25. Example Activity Database Centres Implement and Communicate Fix
Information via Profiles
Centres
Appointees
SQA teams
Implement and Communicate Fix
Contains profile of criteria (R / A / G) and Good Practice
Awareness and analysis through number of views / reports
Highlight and understand
Non Compliance pattern

26. David Pirnie Lead Verifier – HN & VQ

27. Results of Survey Questionnaire June 2013
Centres
Website is easy to follow - 93%
Guidance informs what to expect from a Qualification Verification
Visit – 75%
Qualification Verification forms and templates are clear – 64%
Qualification Verifiers
Clear flowcharts and procedures - 100%
Understand the type of evidence
required – 74%
Guidance is effective - 81%
Qualification Verification forms
and templates are clear – 10%

28. Veronica Kirk Quality Enhancement Manager

29. QEM feedback
Anecdotal feedback from centres on the new systems and criteria.

30. Tea Break

31. Quality Assurance Logistics Manager
Juliette McGinley
Quality Assurance Logistics Manager

32. Overview Introduction and team update EV+ functionality
Quality Assurance Management System

33. Quality Assurance Logistics Team

34. EV+ documents and templates
Review of documents
Template index and restructure of documents
Quality Framework documents archived

35. Allocations via EV+ Process Style Allocation Accept or Decline
Style
Allocation
Accept or Decline
Visit Date
Visit Report Location
Reminder Facility
New Approach to QA
EV+
Online reporting
EV+ then QAMS
Visiting SVQ/HN
Contact QA Logistics Team
Report upload to EV+
Visiting NQ
Documentation from EV+

36. EV+ Summary Continue to accept/decline allocations
Log estimate and actual visit dates
Availability of visit plan PDF
Some reminder facilities will remain

37. Quality Assurance Management System (QAMS)

38. Benefits of the Quality Assurance Management System (QAMS)
Online Reporting for Systems and Qualification Verification
Workflow for verification activity
Information Management System
Intelligence to inform future selections for Systems and Qualification Verification
Reporting output to centres

39. Benefits of the Quality Assurance Management System (QAMS)
SQA
Information Management System
Reduction in manual monitoring, logging and tracking
Intelligence building for each centre
Centre
Improved report layout with non-compliances at glance
Improved Service Level Agreements
Evidence review only contains non-compliant criteria
Qualification and System Verifiers
Ease of use, web based system
Built in Guidance and on screen errors
Only need to view active, outstanding workload
Evidence review only contains non-compliant
criteria

40. Quality Assurance Management System (QAMS).
Online Reporting – do not use OLD excel report forms
Web link to QAMS included in guidance and EV+
Automated reminders addresses
- EV+
- QAMS

41. Welcome Back

42. David Pirnie Lead Verifier – HN & VQ

43. Supporting Qualification Verification
Update of SQA’s Quality Assurance Guidance
New system for dealing with Qualification Verification queries
Introducing the Standardisation Record

44. Quality Assurance Guidance has been updated
The purpose of SQA’s Quality Assurance
Guidance is to:
support Qualification Verifiers
promote consistency and transparency in Qualification Verification
provide a source of reference for verifiers and centres

45. Visit Planning 1. Accept allocations on EV+
2. Make initial contact with each centre and ascertain optimum time to visit (if visit is feasible)
3. *Request information from centre
4. Select sample, formulate and send Visit Plan
*Where centres have agreed to provide evidence for all
selected units, they should still let you know who the
assessors/verifiers are and list the
assessment sites being used

46. Criteria wording changes
2.8 There must be evidence of initial and on-going reviews of assessment
environment(s), assessment procedures, equipment, learning and
assessment materials
4.1 The centre’s assessment and verification procedures must be documented and implemented to meet qualification and SQA requirements  (the “review” requirement removed – now in 2.8)
3.2 Candidate’s development needs and prior achievements (where
appropriate) must be matched against the requirements of the award
(“These must be related to agreed personal action/training plans”
has been removed and placed in examples of evidence

47. QCF Assessor/Verifier Requirements
General requirements apply:
All Assessors/Internal Verifiers
In general terms, SQA requires all Assessors/Internal Verifiers to be
competent in the subject/occupational area to a level appropriate to the
qualification and competent in assessment/internal verification of the type
involved in the qualification.
QCF and Customised Awards Assessors/Internal Verifiers do not need
to show evidence of having achieved nationally recognised assessor/verifier
qualifications e.g. A1/V1/9DI as per SVQs.

48. Context section has been added
Criterion 2.8: There must be evidence of initial and on-going reviews of
assessment environment(s), assessment procedures, equipment,
learning and assessment materials.
Rationale for criterion inclusion
It is each centre’s responsibility to ensure that the assessment environment,
assessment procedures, equipment, learning and assessment materials are
up to date and provided in sufficient quantity to meet ………………………..
Context
“Evidence of planned or/and past reviews can be
Considered in judging whether sufficient evidence
exists” .

49. Context section has been added
Context - other examples:
“Collective and/or individual candidate development needs can be
considered when reviewing evidence”
4.1 “Comparing expected practice (documented procedures) with actual
practice (the implementation of those procedures) can help provide
appropriate evidence”
4.2 “Assessment methods/instruments and decisions should be sampled “

50. Other changes to Quality Assurance Guidance
Interviewing candidates is important to the verification process (all
qualifications) and should take place whenever possible
Where the qualification(s) sampled include one or more SVQ Units, the
relevant Sector’s Assessment Strategy applies to those units
New Guide to Assessment about to be published

52. Processing Qualification Verifier queries
Draft responses, dependent on what they are, may be sent to a nominated SQA Senior Manager/Department for their comment.
All queries to be submitted via the QA Appointee Website
The Quality Assurance Panel (QAP) will agree response wording and advise whether it should be included in Frequently Asked Questions (FAQs) or Answering Your Queries (AYQ) within 5 days.
FAQs accessible to all SQA centres and all SQA staff and appointees on SQA’s Q A Website.
AYQ accessible to SQA staff and appointees only on SQA’s QA Appointee website.
New items are added to either the FAQs and/or AYQ .
The FAQs and AYQ will be reviewed and the results used to update Qualification Verification procedures and guidance.

53. Introducing the Qualification Verification Standardisation Decision Record
A Standardisation Decision record is:
A single continuous and up to date record used by a Verifier Group to
record key decisions relating to the group’s subject area
Benefits
All key decisions relating to externally verifying qualification(s) in one place
New (and more timely) decisions can be added without face to face meetings
Focuses on group agreement, promoting consistency
Relevant SQA Staff can more efficiently access decisions made by each group
New group members can get up to speed more quickly

54. Questions?

55. Quality Assurance Logistics Manager
Juliette McGinley
Quality Assurance Logistics Manager

56. Standardisation Meetings and EV Updates
Standardisation meetings are booked for 4 Saturdays in October and 1 in November.
Singleton Update in November
Invites will be sent out to Seniors by 3rd September.
QAMS training sessions will run each Saturday
Disseminate information relating to Quality Assurance at standardisation meetings

57. Service Level Agreements
Qualification Approval
Accept or decline allocated visit within 3 working days
Advise of visit date within 5 working days of acceptance
Visit plan sent to centre copying in
Visit Report submitted to SQA within 5 working days of visit
Qualification Verification
Accept or decline allocated visit within 2 weeks
Contact centre to arrange visit as soon as possible
Re contact centre within 6 weeks of visit to request information
Visit plan sent to centre and SQA within at least 2 weeks before the visit

58. Housekeeping
College regionalisation – please ensure SQA has a current list of exception centres.
EV contact details – please ensure SQA holds your most current details.
Please submit IAR ASAP, deadline 30th August
New EVs – contact QA Logistics team to create an EV+ account.
Quality Assurance Appointee website – migration from SQA Academy.
Visit Plans must be sent to SQA.

59. Moving Forward
Quality Assurance Appointee Website will be updated with documentation discussed today:
Standardisation documentation
Processing qualification verification queries
EV+ guidance
SQA’s Quality Assurance presentation/videos
Qualification verification guidance
Regular e-zines to update Verifiers as documentation is published on Appointee website
All centres offering HN & VQ will move to SQA’s Quality Assurance for Higher National and Vocational Qualifications by August 2014

60. Further Information Quality Assurance Appointee Website:
QAMS online reporting:

61. Questions?

63. Senior External Verifier Event 29 August 2013

64. Appointee Management
Mark Hood Training and Development Manager Tel:

65. Senior External Verifier Event
HN/SVQ External Verifier Qualifications
Key Performance Measures (KPM)
Data Protection Act (1998)/
Protecting Information
SQA Academy Courses

66. HN External Verifier Award – 2012/2013 Academic Session
Externally Verify the Assessment Process in Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
75 HN External Verifiers enrolled on the award
39 Started in 2012/2013 academic session
36 Started in the 2011/2012 academic session
HN External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.

67. HN External Verifier Award – 2012/2013 Academic Session
Externally Verify the Assessment Process in Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
10 Candidates completed the award in the
2012/2013 academic session – started
2011/2012 academic session
11 Candidates from 2011/2012 academic session are scheduled to complete.

68. HN External Verifier Award – 2012/2013 Academic Session
Externally Verify the Assessment Process in Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
15 Candidates from the 2011/2012 academic session require centres to be allocated in the 2013/2014 academic session in order to complete award.
16 Candidates from 2012/2013 academic session are scheduled to complete prior to January 2014.
HN External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.
The Other Eleven External Verifiers Will Require One or More Running Centres in order to complete the award in the 2013/2014 academic session.

69. HN External Verifier Award – 2012/2013 Academic Session
Externally Verify the Assessment Process in Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
23 Candidates from the 2012/2013 academic session require centres to be allocated in the 2013/2014 academic session in order to complete award.
HN External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.
External Verifiers Will Require One or More Running Centres in order to complete the award in the 2013/2014 academic session.

70. SVQ External Verifier Award – 2012/2013 Academic Session
Externally Monitor and Maintain the Quality of Workplace Assessment – (FD44 04)
2012/2013 Academic Session
29 SVQ External Verifiers enrolled on the award
13 Started in 2012/2013 academic session
16 Started in the 2011/2012 academic session
SVQ External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.

71. SVQ External Verifier Award – 2012/2013 Academic Session
Externally Monitor and Maintain the Quality of Workplace Assessment – (FD44 04)
2012/2013 Academic Session
7 Candidates completed the award in the
2012/2013 academic session – started
2011/2012 academic session
1 Candidate from 2011/2012 academic session is scheduled to complete.

72. SVQ External Verifier Award – 2012/2013 Academic Session
Externally Monitor and Maintain the Quality of Workplace Assessment – (FD44 04)
2012/2013 Academic Session
8 Candidates from the 2011/2012 academic session require centres to be allocated in the 2013/2014 academic session in order to complete award.
6 Candidates from 2012/2013 academic session are scheduled to complete prior to December 2013.
SVQ External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.

73. SVQ External Verifier Award – 2012/2013 Academic Session
Externally Monitor and Maintain the Quality of Workplace Assessment – (FD44 04)
2012/2013 Academic Session
6 Candidates from the 2012/2013 academic session require centres to be allocated in the 2013/2014 academic session in order to complete award.
SVQ External Verifier Award
The award was well received by those External Verifiers who participated on the award.
In most instances the standards allowed experienced EVs to ratify that there working practices met national standards.
External Verifiers Will Require One or More Running Centres in order to complete the award in the 2013/2014 academic session.

74. HN/SVQ External Verifier Awards 2013/2014 Academic Session
HN External Verifier Award
Scheduled to commence from November 2013
SVQ External Verifier Award
Scheduled to commence from December 2013

75. Key Performance Measures (KPMs)
KPM – No. 6
submit CPD record with agreed timescales where appropriate

76. Continuing Professional Development Record (CPDR)
Submission Date: 30 June in any given year
Total Number of CPDRs expected 442
Total Number of CPDRs submitted 334
Total Number of CPDRs not submitted 98

77. Continuing Professional Development Record (CPDR)
Submission Date: 30 June in any given year
Total Number of CPDRs submitted 334
Total Number of CPDRs Compliant
Total Number of CPDRs Non-Compliant 9
Total Number of CPDRs Compliant – Late 32
Notes:
Of the 352 that were compliant – 30 of those submissions were submitted after 30 June The latest being 8 August 2012.

78. CPDR – Standardisation
Total Number of CPDRs Non Compliant
on first submission
Total Number of CPDRs Compliant on
second submission
Total Number of CPDRs Non-Compliant
submitted after – 30 June
Total Number of CPDRs Non-Compliant 9

79. Result of CPDR Non-Submission
The following may not be re-appointed for the 2013/2014 academic session as CPDR was not submitted for 2 years in a row: 15 EVs across HN and SVQ Qualifications Blocks
Not a particular pleasant slide.
SEV/EVs advised late August/early September 2012

80. Key Performance Measures (KPMs) Principal External Verifier (1 of 3)
Train, develop and support members of the EV team to meet SQA requirements <SQA>
Monitor performance of members of the EV team to meet SQA requirements <SQA>
Ensure consistency of decisions made by members of the EV team <SQA>
Ensure reports generated by members of the EV team meet quality standard <SQA>

81. Key Performance Measures (KPMs) Principal External Verifier (2 of 3)
Positive engagement with all members of the team through leadership and management of people <SQA>
Positive engagement with all members of SQA staff<SQA>
Complete Principal External Verifier annual report within conditions and timescales<SQA>

82. Key Performance Measures (KPMs) Principal External Verifier (3 of 3)
Satisfactorily fulfil the KPMs of the External Verifier role <SQA>

83. Key Performance Measures (KPMs) External Verifier (1 of 2)
Complete External Verifier training/updates <SQA/PEV>
Complete scheduled centre allocation of visits within conditions and timescales for SQA <SQA/PEV>
Complete scheduled central events within conditions and timescales for SQA where appropriate <SQA/PEV>

84. Key Performance Measures (KPMs) External Verifier (2 of 2)
Complete External Verifier reports as per SQA quality standard and within agreed timescale <PEV>
Positive engagement with all members of the team, SQA centre’s personnel and with all members of SQA staff <SQA/PEV>
Submit CPD record within agreed timescales where appropriate <SQA>

85. Data Protection Act (1998)/ Protecting Information
SQA Data Protection policy has recently been
reviewed. The updated policy aims to:
Clearly explain the implications of the Data Protection Act for SQA’s work
Set out expectations of employees, appointees, nominees, agency workers and secondees in carrying out their duties.
Explain the consequences for SQA for any policy breach
Will be on the SQA Academy. Principal External Verifiers and External Verifiers will be notified by when the course is available.

86. Data Protection Act (1988)/ Protecting Information
2013/2014 Academic Session
Protecting Information Course
Protecting and Storing Information
In the workplace
On the move
Staying safe online
Assessment
Five separate categories.
Will be on the SQA Academy. Principal External Verifiers and External Verifiers will be notified by when the course is available.

87. SQA Academy Programmes
In October 2013, I shall all SEVs/EVs and advise that the following 2 courses have been updated for this current academic session:
CPD for HN/SVQ External Verifiers
Explaining the External Verifier Awards
Explaining the External Verifier Awards – Newly appointed External Verifiers will receive this .

88. CPD for HN/SVQ External Verifiers
Changes for 2013/2014 Academic Session
One
If you are a Principal External Verifier or External Verifier with a Vocational Qualification appointment will be asked to confirm that you meet the occupational requirements of the Assessment Strategy for your area of expertise in any one academic session.

89. CPD for HN/SVQ External Verifiers
Two
If the awarding body is audited by a regulatory body (for example: SQA Accreditation or Ofqual) we may ask you to submit evidence to this effect for audit purposes.
Three
The Continuing Professional Development Record (CPDR) is now in a PDF format.

90. CPDR SQA Accreditation Unit Audit
SQA Accreditation Unit Audit – July 2013
17 Verification Groups were selected for audit

91. CPDR SQA Accreditation Unit Audit
Carpentry and Joinery (165)
Construction and Civil Engineering Services (352)
Construction Conservation (432)
Construction Engineering (352)
Construction Management (316)
Construction (Plant and Ops) (186)
Construction Technician (161)
Emergency First Aid at Work (488)
Licensed Trade Qualifications (425)
Painting and Decorating (169)
SVQ Sport and Recreation – Facilities Management/Development (202)
SVQ Sport and Recreation – Exercise and Fitness (200)
SVQ Sport and Recreation – Outdoor Education (204)
SVQ Sport and Recreation – Playwork (201)
SVQ Sport and Recreation – Spectator (204)
SVQ Sport and Recreation – Sports Coaching (199)
Wood Machining (170)