1. Triangulation of feedback on the quality of educational supervision in the Northern Foundation School
Jeremy Twigg FPD Northern Foundation School
Anna O’Riordan FPT Newcastle University Hospitals Trust
NACT UKFPO SHARING BEST PRACTICE EVENT 10TH MARCH 2017
Anna O Riordan
Amy Girdlestone
Gemma Crackett
Angela Burton
Mike Wilson
Richard Hardern
Andy Davies
Craig Steele

2. The Need for ES Development
Revalidation
Appraisal
GMC
Educational Supervisor Recognition
All named educational supervisors must have been recognised by their postgraduate dean by 31 July 2016
Promoting Excellence standards for medical education and training
Selected
Trained
Responsible
Meets
Reviews
Brings Together relevant evidence for a summative judgement about progression
GMC Recognising and approving trainers: the implementation plan 2012
“trainers are expected to develop their skills and competence, and to consider opportunities for career progression as trainers, supported by the processes involved in trainer recognition.”

3. The Need for ES Development
UK FPO FP Reference Guide
selected and appropriately trained
committed to and engaged
LETB/deanery/foundation school must collect evidence from all prospective educational supervisors on how they meet the GMC’s guidance for recognising trainers
Only educational supervisors formally recognised by the GMC should be the named educational supervisor for a specific foundation doctor
resources and time to undertake their training role

4. The Need for ES Development
UKFPO FP Reference Guide 2016
The ES and the Trainee
Formative and summative assessments
Agreeing learning objectives
Supporting and identifying DWDN’s
Identify concerns and putting in place support addressing issues liaising with FPT/D/SD
Ensuring trainee reflects and discusses engagement
Completing EOYR confirming trainee has met requirements for satisfactory completion of FY1/2

5. Anecdotal Trainee Feedback
Previous Methodology
Anecdotal Trainee Feedback
Anecdotal Experience of Educational Supervisors at ARCP
FPT’s
FPD’s
Anecdotal Evidence of ES’s performance through DWDN within Trusts
Mixed ARCP Skills visible in portfolio’s
Differing ARCP evaluation methods across Trusts

6. Heterogeneity

7. New Methods: Triangulated Approach Old Evidence and New
Educational Supervisor Self Assessment
Educational Supervisor Trainee Assessment
Educational Supervisor ARCP Panel Assessment

8. ES Assessment in Trainee Survey ES Self Assessment ES ARCP Assessment
Methodology
ES Assessment in Trainee Survey
ES Self Assessment
ES ARCP Assessment
Anonymous
Contributes to Educational Appraisal for Consultants
Provides Feedback to ES
Provides Opportunity for Reflection for ES’s
Identifys Strengths
Identify’s areas for improvement

9. 1-Evidence from Trainee Survey
YSYS asks 4 questions of ES’s as opposed to 8 of Newcastles Questionairre

10. 1-Evidence from Trainee Survey
I met my ES at an appropriate frequency to complete my educational objectives / portfolio requirements
My ES demonstrates a clear understanding of the FP
My ES offers advice and feedback appropriate to my needs
My ES was supportive and approachable when needed
My ES reviewed my e-portfolio and made constructive comment in my meeting forms
I would recommend my ES to another trainee

11. Dynamic Range Heatmap Blobs
YSYS Scoring
Likert Scale
Weighting
Inverse Questions
Dynamic Range Heatmap Blobs
Tableau
John Duffey Project Support Officer Health Education North East
Fixed Range Heatmap Blobs
Dynamic Range Heatmap Blobs
Threshold Blobs
Dynamic Range Unsized Heatmap
Fixed Range Unsized Heatmap
Heatmap Deviation from Mean Unsized
Dynamic range heatmap Blobs
Advantages
Easy to judge relative sizes of respondent groups
Easy to differentiate relative performance
Disadvantage
No indcation of agreemnt within a respondent group (SD, CI)
No colour consistency between domains-score 60 may be pale blue on one chart and orange on another

13. 2-ES Self Assessment
I met my trainee(s) at an appropriate frequency to complete their educational objectives / portfolio requirements
I have a clear understanding of the FP
I offer advice and feedback appropriate to the needs of my FP trainee(s)
I was supportive and approachable when needed by my trainee(s)
I reviewed my trainee(s) e-portfolio and made constructive comment in the meeting forms
My trainee(s) would recommend me as an ES to others

14. 2-ES Self Assessment
If areas for development have been identified, the following resources listed below may be of help:   
Good Practice in Educational Supervision – mandatory
Attendance at FP ARCP panel – mandatory every 2 years.
Equality & Diversity Training
ePortfolio navigation
FP Curriculum outcomes
How to give careers advice
Mentoring and/or Coaching
Supporting Underperforming Trainees
Feedback for learning
Developing Effective Educational Learning Plans
Link to Northern Deanery website to access training:
National Standards for FP Educational Supervisors
Educational supervisor responsibilities include:
Meeting each rotation
agree specific learning objectives
supporting and identifying DWDN
reviewing performance at appropriate intervals
ensure doctor access to support
involve the FTPD/T and FSD as appropriate;
sharing relevant information
Trainee reflection
raise concerns with FTPD/T and FSD etc
completing an end of year assessment

15. 3–ES ARCP Assessment A good ES will…
Meet regularly with the trainee throughout the year. If these meetings are not regular the reasons will be clear from the portfolio.
Add justified, relevant and appropriate comment to the trainee’s meeting forms that help the panel reach an appropriate ARCP outcome.
Have reviewed the trainee’s summary of evidence/ ePortfolio and made justified, relevant and appropriate comment.
Have made an end of year pre-ARCP evaluation of the trainee’s likely outcome, which has been communicated to the trainee and which mirrors that of the ARCP panel.
If there are areas of improvement that the Panel consider the ES should make, please document below what these are.

16. ES reflection on feedback and points for discussion at next appraisal
3-ES ARCP Assessment
ES reflection on feedback and points for discussion at next appraisal
What have you learnt?
What will you do?
What are your initial steps?

17. 3-ES ARCP Assessment For underperforming ES’s List of Courses
HENE Website
Standards for ES’s
Responsibilities of ES to the trainee
If areas for development have been identified, the following resources listed below may be of help:   
Good Practice in Educational Supervision – mandatory
Attendance at FP ARCP panel – mandatory every 2 years.
Equality & Diversity Training
ePortfolio navigation
FP Curriculum outcomes
How to give careers advice
Mentoring and/or Coaching
Supporting Underperforming Trainees
Feedback for learning
Developing Effective Educational Learning Plans
Link to Northern Deanery website to access training:
National Standards for FP Educational Supervisors
Educational supervisor responsibilities include:
Meeting each rotation
agree specific learning objectives
supporting and identifying DWDN
reviewing performance at appropriate intervals
ensure doctor access to support
involve the FTPD/T and FSD as appropriate;
sharing relevant information
Trainee reflection
raise concerns with FTPD/T and FSD etc
completing an end of year assessment

18. 3-ES ARCP Assessment Jeremy Twigg ES Feedback 2016
Meet regularly with the trainee throughout the year. If these meetings are not regular the reasons will be clear from the portfolio.
Yes, timely reports
Yes. The panel noted that the ES had highlighting areas of the portfolio that needed work and knows the portfolio well and directed the trainee well
Add justified, relevant and appropriate comment to the trainee’s meeting forms that help the panel reach an appropriate ARCP outcome.
Yes – panel noted ES had provided useful comments on ES reports. ES came across as helpful giving useful constructive feedback to trainee.
Yes
Have reviewed the trainee’s summary of evidence/ ePortfolio and made justified, relevant and appropriate comment.
Have made an end of year pre-ARCP evaluation of the trainee’s likely outcome, which has been communicated to the trainee and which mirrors that of the ARCP panel.

19. Deanery Data Feedback to Trusts
Likert Scale
Collated for All Trusts
Converted to Numeric
Formatted to Flags
F2’s only in 2016
May roll out to include F1’s
Some trusts only keep same ES for 1 year.
ARCP washup

20. I meet my Educational Supervisor at an appropriate frequency to complete my educational objectives / portfolio requirements.
My Educational Supervisor demonstrates a clear understanding of the Foundation Programme
My Educational Supervisor offers advice and feedback appropriate to my needs
My Educational Supervisor was supportive and approachable when needed
My Educational Supervisor reviewed my e-portfolio and made constructive comments in my meeting forms
I would recommend my Educational Supervisor to another trainee

21. Free Comments
Dr XXX has been a fantastic education supervisor throughout FY1 AND fy2
always available, checks through my portfolio regularly and makes recommendations about improvemetns
My educational supervisor is excellent, he had dedicated so much of his precious time to supporting me as a trainee.
she takes it seriously and does it properly, as well as being friendly and practical.
My Ed. supervisor, Dr. XXX (consultant renal), is probably one of the nicest consultants I came across. Ever since my first meeting with him, he has treated me as a peer, has made me feel welcomed and made my work feel important and valued, he praised me for my strengths and recommended further areas of development. I am very happy and fortunate to have been assigned to him and would whole heartedly recommend him to any trainee.

22. Free Comments
My supervisor is exteremly difficult to get hold of. I have to send multiple s. She is also based in XXX- all my jobs are in UHND so I can't get time off work to meet.
I asked for a new Educational Supervisor at the end of my Foundation Year. Although I had not had a good relationship with my previous supervisor, my reasoning was purely pragmatic: he was based in XXX hospital and my placements for F2 were in XXX and XXX XXX. So I asked for a new Educational Supervisor based at XXX. I would therefore for 2 out of the 3 rotations be based in the same hospital, which I hoped, would make the supervision more practicable and straightforward. Initially, the Foundation School wanted to implement some sort of (?)reconciliation process with my supervisor, which given the demands of my busy new job was utterly impossible, and which was exactly the sort of impractical process I was trying to avoid. Eventually, after several months, I was offered a new Educational Supervisor. She was based in XXX. I attempted to gain an Educational Supervisor in my F2 Paediatric rotation and a consultant apparently agreed. Less than a week later I discovered he had taken long-term sick leave due to stress.
My educational supervisor shows very little interest in actually 'supervising' me. I dont think he even knows I am an F2 or what my needs would be. If i didnt prompt and chase him we would never ever meet.
Despite attempting to make regular contact with my education supervisor, I often do not receive a reply. I am required to repeatedly send s and also contact the education department to try and facilitate meetings.

23. END
Questions