1. International Consensus on Defining and Measuring Quality in Surgical Training
Pritam Singh, Rajesh Aggarwal, Boris Zevin, Teodor Grantcharov, Ara Darzi
Presented by
Pritam Singh

2. Quality of Surgical Training
Background
Quality of Surgical Training
Quality of Surgeon
Patient Outcomes
Proposed reforms to educational commissioning in the UK
Emphasis on hospitals and surgical training programs
to demonstrate evidence of high quality surgical training.
Focus has been on quantity

3. Objectives Improve Surgical Training
Define high quality surgical training
Define methods to measure quality
Measure training quality

4. Methods Thematic Analysis of 20 semi-Structured Interviews
Develop statements
Indicators of training quality
Methods to measure them
Identify international experts in Surgical Education
Invitations to online questionnaire
Rank each statement Likert scale 1-5
Consensus predefined
80% experts ranking the statement 4 or 5
Cronbach’s alpha 0.80

5. Results International Consensus
20 Semi-structured interviews
70 statements
83 invitations to experts from 13 countries
47 (57%) responses from 9 countries
International Consensus
Consensus was achieved on 33 of the 70 statements (Cronbach’s α = 0.90).

6. Consensus: Focus on Quality
Measuring the quality of surgical training will:
Have an overall positive effect on training.
Encourage competition and drive up standards of training
Create accountability for individual trainers.
How should the results of measuring surgical training quality be used?
Underperforming trusts contacted.
Firstly, focus on why measuring training quality is important – and in fact is it important?
Secondly, how should we use the results of this exercise?
In the current changing climate of educational commisioning, particularly in the UK where trusts will need to demonstrate high quality training in order ot receive trainees, if they underperform, they should be given the information needed to improve.
Having established that there is a need and a purpose for measuring the quality of surgical training – the next step is to define high quality surgical training

7. Consensus: Defining Quality
Relationship between the trainer and trainee
Good rapport is essential.
Trainer must be approachable.
Trainer must be willing to allow the trainee to perform operative cases.
Exposure to operative cases
Case mix.
Supervision versus Independence
Level of supervision
Balance between supervision and independence
Facilitates an increasing level of independence for the trainee.
Relationship was also the most common theme from the interviews, particulalry when subjects talked of their best training expereinces
Case mix is an important marker of surgical training quality. E.G. A good mix of minimally invasive or laparoscopic surgery alongside open surgery.
High quality training facilitates an increasing level of independence for the trainee. Supervision may be necessary at the start, but increasing independence can be in the form of supervision from the coffee room
Some examples about statements

8. Consensus: Defining Quality
Constructive Feedback
Trainers should offer regular, constructive feedback to the trainee.
Trainers should be given feedback about their training ability.
Strong engagement of the trainer with the training process
Structure and Organisation
Rota can determine surgical training quality.
Challenge the trainee to make progress
Personalised training
Training placements
Trainers should offer regular, constructive feedback to the trainee. This can be in a combination of informal or written feedback
Trainers should be given feedback about their training ability. – They need to be offered the opportunity to improve
Explain rota statement – for instance, shift patterns can affect the continuity between the trainee and trainer and subsequently affect the rapport
A high quality surgical training post should challenge the trainee to make progress during the post

9. Consensus: Defining Quality
Simulation and Teaching
Structured teaching programme
Teamwork
Trainee rapport with other trainees and surgeons
Stepwise approach to training
Clear objectives set jointly by the trainer and trainee
Operating Room
Lists structured to take into account time for trainees.
Involved in all operative cases even if complicated.
Repetition of similar procedures.
It is essential for training placements to be matched to the individual trainee's level.
Presence of a structured teaching programme is a marker of surgical training quality. The role of simulation within this teaching programme was less well defined.
Trainee rapport with other trainees and surgeons affects the quality of surgical training. E.g. Senior trainees teaching/mentoring junior trainees
A high quality surgical training post starts with clear objectives set jointly by the trainer and trainee
Reference the costs of OR training time which were discussed earlier today

10. Consensus: Measuring Quality
Trainee feedback
Aggregate measures
Formative evaluations of their trainers.
Multi-source feedback?
Trainee feedback.
Trainer feedback.
Trainee improvement
Evidence during each training post.
Formative evaluations of trainee from trainers
Practical technical assessment.
Include assessment of non-technical skills.
Aggregate measures from a number of trainees should be used. Avoids indivdual personality clashes
Trainees should provide formative evaluations of their trainers.
Multi-source feedback to measure surgical training quality?
Trainee feedback is an essential component.
Trainer feedback is an essential component. – they often know the limitations of their training program and would welcome the opportunity to try and improve them.
There should be actual evidence of trainee improvement during each training post.
This includes formative evaluations of trainee from trainers
Formative assessment should include practical technical assessment of trainees.
Formative assessment should include assessment of non-technical skills of trainees.
Written assessments were described as being difficult to pick up these factors

11. Conclusion International expert consensus to define Surgeons believe
Elements of high quality surgical training
Metrics for their measurement.
Surgeons believe
Comparisons of Surgical Training Quality
Create Competition
Drive up Standards of Surgical Education

12. Future Research Improve Surgical Training
Define high quality surgical training
Define methods to measure quality
Measure training quality
Develop a tool to objectively evaluate quality of surgical training programs

13. Thank You