1. Educational Supervision & Find Your Way Around in the E-portfolio Dr Jane Mamelok RCGP WPBA Clinical Lead

2. Workplace Based Assessment
“The evaluation of a doctor’s progress over time, in their performance in those areas of professional practice best tested in the workplace”
FORMATIVE
Summative judgment made by panels informed by evidence from WPBA
This is how WPBA was defined for the PMETB submission. The important point to recognise is that the training programme is mapped to the curriculum objectives and WPBA is the opportunity for the trainee to gather evidence and build their portfolios over the entire training programme. The Educational supervisor has an important role in guiding them through that process. They must understand the programme, the necessity for it to be rooted in the curriculum and the assessment schedule and requirements for each trainee. This point may be developed later in the presentation and discussed in the group.
The process is formative, gathers evidence over time that informs the panel process and the summative judgment.

3. Miller’s Pyramid Knows Knowledge AKT Shows How Knows How Performance
Competence
CSA
Does
Action
WPBA
To remind you where it sits on Millers pyramid of competence, WPBA because it looks at performance in action in the workplace tests higher up Millers and offers the opportunity if it tests across a range of complexities it can also push up Bloom's taxonomy.
AKT sits at the bottom, knows and knows how.
CSA pushing higher up
WPBA because it relates to the learners’ experience in the workplace sits at the top

4. Coverage by assessments
Curriculum
WPBA
CSA
AKT
Primary care management
Person centred care
Specific Problem-solving skills
Comprehensive approach
Community orientation
Holistic approach
Contextual Aspects
Attitudinal Aspects
Scientific Aspects
Reminder of the blueprint areas for the components
This is useful as it demonstrates that WPBA crosses nearly all the blueprint areas and also addresses the gaps not covered by other components.
Credit for this slide is Amar Rughani and the blueprint.
Psychomotor skills

5. Working with Colleagues Practising Holistically Maintaining an Ethical
RELATIONSHIP
DIAGNOSTICS
Communication &
Consulting Skills
Working with Colleagues
& in Teams
Practising Holistically
Data Gathering &
Interpretation
Clinical Management
Managing Medical
Complexity
Making a Diagnosis /
Making decisions
Professionalism
Maintaining an Ethical
Approach to Practice
Fitness to
Practise
WPBA is underpinned by a competency framework derived from the curriculum domains and mapped to good medical practice.
This slide usefully categorises them into 3 areas
Relationships
Diagnostics
Management & Professionalism
The slide should be attributed to Tim Norfolk
A useful ice breaker in a group us to delay the slide which is animated and ask them to name the 12 competency areas – most people can get about 8 or 9 with some prompting. I usually swot up beforehand so as not to embarrass myself.
Maintaining Performance,
Learning & Teaching
Primary Care
Administration
& IMT
Community
Orientation
MANAGEMENT

6. The potential of linking WPBA & the learning portfolio.
WPBA assesses performance in the workplace over the entire training programme against 12 defined competencies.
The portfolio gathers the information and offers the opportunity for reflection and formative feedback.
Links assessment to learning outcomes and continuing professional development.

7. Clinical Supervision
Clinical supervision describes the framework for regular, structured encounters reflecting on casework in the context of the post or specialty in which the health professionals are working and aims to identify areas of best practice and developmental needs.
It has an important clinical governance function.
Clinical supervision
Clinical supervision has been used in mental health services for many years, and describes the framework for regular, structured encounters reflecting on casework in the context of the post or specialty in which the health professionals are working and aims to identify areas of best practice and developmental needs. Clinical supervision has an important clinical governance role setting and improving professional standards of care. Clinical supervision can be a one to one encounter or between groups of practitioners.
In the context of specialty training, clinical supervision usually takes the form of one to one encounters between trainee (supervisee) and a more senior and experienced clinical supervisor.

8. Educational supervision
Educational supervision is organised supervision taking place in the context of a training programme, it is aimed more to act as an umbrella to guide the trainee through the training programme than assessing and discussing individual cases (which is in the domain of the clinical supervisor).
Educational supervision is organised supervision taking place in the context of a training programme, it is aimed more to act as an umbrella to guide the trainee through the training programme than assessing and discussing individual cases (which is in the domain of the clinical supervisor).
Both clinical and educational supervisor may have conflicting roles. One is developmental, allowing reflection and development, the other to provide performance assessment. The summative nature of the end point of postgraduate training and the contribution clinical and educational supervisors’ report make to that final summative decision, means there is no easy way to resolve this conflict and it must be recognised and carefully managed.

9. Gold Guide Requirements
Clinical supervision
Each trainee should have a named clinical supervisor for each placement, usually a senior doctor, who is responsible for ensuring that appropriate clinical supervision of the trainee’s day-to-day clinical performance occurs at all times, with regular feedback.
Gold Guide definitions
The Gold Guide defines and differentiates the two roles as follows:
Clinical Supervision
Each trainee should have a named clinical supervisor for each placement, usually a senior doctor, who is responsible for ensuring that appropriate clinical supervision of the trainee’s day-to-day clinical performance occurs at all times, with regular feedback.
Educational supervision
Educational supervisors are responsible for overseeing training to ensure that trainees are making the necessary clinical and educational progress. Where possible, it is desirable for trainees to have the same educational supervisor for the whole of their training programme or for stages of training (e.g. the early years or more advanced years of training).

10. Gold Guide Requirements
Educational supervision
Educational supervisors are responsible for overseeing training to ensure that trainees are making the necessary clinical and educational progress. Where possible, it is desirable for trainees to have the same educational supervisor for the whole of their training programme or for stages of training (e.g. the early years or more advanced years of training).
Gold Guide definitions
The Gold Guide defines and differentiates the two roles as follows:
Clinical Supervision
Each trainee should have a named clinical supervisor for each placement, usually a senior doctor, who is responsible for ensuring that appropriate clinical supervision of the trainee’s day-to-day clinical performance occurs at all times, with regular feedback.
Educational supervision
Educational supervisors are responsible for overseeing training to ensure that trainees are making the necessary clinical and educational progress. Where possible, it is desirable for trainees to have the same educational supervisor for the whole of their training programme or for stages of training (e.g. the early years or more advanced years of training).

11. What makes a good educational supervisor ?
Senior educator with an understanding of training programme, assessments and employment issues.
Good listener offering supportive challenge.
Can give constructive feedback
Professionalism, recognises the conflicting roles, sets boundaries etc.
What are the qualities and attributes of an educational supervisor?
The ES should be a senior person with knowledge and understanding of the educational programme, assessments and employment issues in the context of the trainees’ professional development journey.
A good listener, with counselling skills and analytical, critical thinking.
Understand feedback principles and be able to ask discriminating questions, giving constructive feedback, supportive challenge, allowing formative development.
Non-judgmental, the relationship relies on mutual trust and respect and therefore the ES should provide non judgmental reflection of observed behaviours and be able to set the necessary boundaries when assessment of performance in practice is required.
Self awareness and insight into own weaknesses and potential pitfalls. The ES should be able to recognise the problems created by transference and dependence the ES/trainee relationship can create and have strategies for preventing stress and burnout.
Professionalism, the ES must treat this relationship with professionalism, respecting confidentiality, making the process transparent.
Motivation, the ES should want to be involved in this privileged relationship however, educational organisations should value its important role in professional development and it should be appropriately rewarded.

12. The role of the ES in GP specialty training.
This can be summarised as providing an overarching umbrella of guidance helping the trainee get the best out of their programme and navigating their way through it.
However, the ES assesses the rate of progress and development of competence contributing to a summative judgement.
The primary role of the ES is to guide the trainee through the programme and get the best out of it.
The problem is that they do have a responsibility to monitor progress through the programme and assess how trainees are progressing through the programme and their progress in development of the required skills and competencies. The ES reports and their recommendations contribute to the final summative judgement.

13. Varied roles of the Educational Supervisor
Guidance and support with performance appraisal
Learning needs assessment, empowering and facilitating the learner.
Conduit and network function
Appraiser and counsellor
Coaching and career counselling
Guardian role
The key roles for an Educational Supervisor can be summarised (4).
An ES provides a guiding and supportive role; it cannot be divorced from performance appraisal and the potential conflict should be recognised and managed in such a way that does not discourage openness from the learner.
Guide learners through their programme, identifying learning and development opportunities.
Contact and conduit function, the ES can provide regular contact outside of the programme and act as a conduit for resources and liaison with programme staff.
Information and network resource, signposting the learner, “opening doors” and providing introductions appropriately.
Counsellor, it is the role of the ES to listen, provide constructive and challenging reflection and facilitate problem solving.
To nurture, identifying strengths and weaknesses, empowering and facilitating the learner on the journey from novice to expert.
Provide an appropriate role model.
Coaching, there is a role to coach learners through key stages, grooming for interview etc.
Finally there is a guardian role, the ES can pick up potential problems and provide an understanding of the support and referral framework allowing both ES and trainee to fulfil their professional obligations in Good Medical Practice (5) (e.g. encouraging a sick doctor to seek help).

14. So you are supervising a GP specialty trainee, what do you need to do?
Identify you learner, ST1 ST2 or ST3.
Arrange an early meeting to set boundaries, frequency of meetings etc.
The ES needs to be able to navigate the e-portfolio proficiently.
Review the evidence.
Give formative feedback and define PDP at the staged reviews.
Discuss the difficulty of managing more than one supervisee, most of us have 1 or 2 at each level.
Setting boundaries is important – some trainees expect the ES to read and validate all their entries.
You might want to discuss the differing roles of the clinical supervisor in conducting the assessments (or delegating them) and the ES who review them and assesses progress and feeds that back.
Being able to navigate the EP is hugely important and they MUST have access in the ES role as well as trainer to be able to access all the relevant pages and to see and write in the review page.

15. Reviewing the evidence
Check assessment schedule.
Review the self assessment ratings.
Review PDP objectives if follow on review.
Review competency and curriculum coverage
Purposeful sampling of the learning log – descriptive v reflective.
Look at CSR is there evidence to support their judgement?
Hone in on some of the COTs and CBDs

16. The Review page e-portfolio
This page demonstrates that they need to be logged on as an ES and to use the review section – red arrows

17. Check the assessment schedule
Click on all reviews and that gives the up to date summary of assessments and evidence

18. Review the PDP
This is an example of the PDP entered by the trainee

19. Review curriculum & competency coverage and self rating scales
This the trainee self assessment rating which the ES can hone in on and make comments

20. Sample the learning log
The learning log, again click on the magnifying glass allows you to hone in, entries need to be shared and then sample and read.

21. Review the Clinical Supervisors report
This is an example of a poor CSR where the trainee has been graded as competent across all the competencies and there is no free text from the CS to justify their judgements, this makes it very difficult for the ES to make a judgment on progress.

22. Questions for debate How much is enough?
How much evidence do you need to make a judgement?
What do we mean by competence?
Lesley Southgate makes the point that you only need to see poor performance repeated 2 -3 times to make a judgement that someone is not fir to progress but more is required to make the borderline/ competent decision.
Competence in this context means competent for licensing and independent practice NOT against peers or for stage in training.
This usually proves contentious and a good point to develop if you use group work.

23. How much evidence is enough?
The evidence gathered from WPBA builds up a “picture” of the competent GP.
What WPBA is doing is building up a picture based on evidence of the competent GP.
Competent for licensing, not at the stage of training or measures against their peers.
If we think of the evidence as pixels in the picture you can see that the more pixels (higher the definition) the clearer the picture becomes.
The debate rages on about how clear the picture needs to be (what is the minimum evidence required) to decide when the picture is complete and when a GP can demonstrate competence.

24. Talk through building the picture up
Talk through building the picture up. When you have the clear picture flip between the mid way and end point.You could recognise Steve at level 2 so is that as much evidence or do you need the clarity of level 3.

25. Defence of evidence Have we got this right yet?
Medical Schools using portfolios for undergraduates have a portfolio interview which allows the trainees to defend the evidence and this informs the portfolio assessment decision.
Is there a role for portfolio viva?
Is that final ARCP panels
Or RCGP oral examiners?
You might want to include this to discuss making judgements and portfolio assessment robust.
I stress that this is my personal view as lead and not RCGP policy but if we had portfolio interviews we would have a World Class Portfolio assessment in the same way as CSA is being heralded as World Class.
Whilst I can be that provocative you may chose not to.

26. Assessment decision
Portfolios are criticised because they depend on subjective judgements that are unreliable.
Schuwirth (2002) argues that they can be reliable if but the “judgements must be collected in a way that samples through possible sources of bias” – sampling professional behaviours on multiple occasions and multiple assessors

27. Key steps for development
Define a portfolio framework but no so restrictive as to trivialise the evidence .
Encourage reflection – can be learned.
Educator and assessor training are crucial.
Need to understand competency framework
Define competence
Achieve acceptable inter rater reliability
Encourage Balanced portfolios
Robust WPBA and portfolio standards for portfolio assessment – defend the JPC decision.
Developments in e-portfolio functionality must match the work on standards and competency framework.
This is a useful summary slide to inform of the changes the WPBA have planned.
The key point is that educator support and training a crucial as is the development of the competency framework that underpins WPBA.
Without an EP with a specification and functionality that delivers on the archive and ANLYTICAL role, that is intuitive and easy to use any developments in WPBA will not work.

28. Summary
The educational supervisor has a key role in directing and supporting the training programme for each learner.
There are useful analogies between the role of the appraiser and the appraisal portfolio and the similar role for educational supervisors.
Training and support for trainees and educators are key.
The e-portfolio must be user friendly and functionality appropriate to purpose.
Your feedback (to WPBA group) is important. WPBA Lead
I like the analogy that using the EP should be as easy as buying theatre tickets on line and not dependent on umpteen PDF user guides.
I make the point that I need their feedback as users for change and am happy for s at the deanery.