1. An Objective Measurement of Training ?
Quality Panels:
An Objective Measurement of Training ?
Jon Francis
Associate Postgraduate Dean Quality
HEE
Training The New Generation
18th January 2017

2. Session Objectives Learn about a way of assessing the quality of PGME
Understand how to run a Quality Panel
Evaluate the Merits of Quality Panel Assessment
Consider the future role of Quality Panels

3. Quality Panel: Why ?

4. Quality Panel: Why ? GMC survey imperfect tool Trainee voice is heard
Triangulation with other sources of quality data

5. Quality Panel History How to run a quality panel
HEE South West Experience
Quality Panel future

6. Who attends the panel meeting?
Training programme director or speciality tutor
Trainee
Educational supervisor
Lay representative
Quality Panel
The meeting was chaired by the TPD or speciality tutor. There was a lay rep, educational supervisors and trainees. A key point was the requirement to have more trainees than trainers on the panel.

7. What happened at the panel meeting?
Review of data from returned trainee matrices and overall gradings allocated by reps
Review of quality grading and associated information
Discussion around overall grading to be awarded
Discussion around areas of excellence and improvement
Discussion around how to take the post forward
The panels were structured in slightly different ways according to the set up of the training programme being evaluated. Generally speciality schools were structured in similar ways, GP slightly different and foundation different again due to the nature of the individual training programmes.
In brief, all trainees on a given programme completed two evaluation matrices which were based on the new GMC guidance: Promoting Excellence, standards for medical education and training. The matrices involved trainees grading their training post as excellent, good requires improvement or inadequate for various domains. (cut to next slide) with the option of comments.
The gradings and comments from these matrices were returned to the school teams and collated into an excel spreadsheet.

8. Effective Educational Environment

9. Safe Supportive Working Environment

10. Example 1

11. Example 2

12. Example 3

13. Effective Education Grade
Overall Quality Grade
Safe Working
Grade
Quality Data
Effective Education Grade

14. Quality Panel BOS experience

15. Overall Quality Grade POST Safe Supportive Working Environment
Supportive Educational Environment
External Quality Data
Overall
Foundation 1
Gastro

16. Quality Panel Review Anomaly Grade
What about stand out issues not consistent with overall grade?
Capture outlining issues requires action and recording

18. Quality Panel Review: Reports
Standardised report template for review
Recording of grading, comments, actions
Anomalies
Report written as a ‘public document’
Emphasis on actions need to improve scoring, whatever the score.
Raising the ceiling as well as the floor
SMART objectives

19. Quality Panel Review: Pros
Useful opportunity to review ‘non GMC survey’ training data
Useful opportunity to triangulate quality data
A powerful process for improving quality of training
Opportunity for greater trainee involvement

20. Quality Panel Review: Cons
a lot of work
Consistency across all schools
Survey fatigue trainees
Future conflict with HEE NETS survey ?

21. Session Objectives Learn about a way of assessing the quality of PGME
Understand how to run a Quality Panel
Evaluate the Merits of Quality Panel Assessment
Consider the future role of Quality Panels

22. Summary and conclusions
Quality Panels are an effective and helpful way of measuring the Quality of PGME
A powerful tool in driving up the quality of PGME
Enables trainee engagement in their programme
Work load, could it replace other activities ?
Future ?