Workplace based assessment for the nMRCGP

nMRCGP Integrated assessment package comprising:  Applied knowledge test (AKT)  Clinical skills assessment (CSA)  Workplace based assessment (WPBA)

Workplace based assessment

Definition “The evaluation of a doctor’s progress over time in their performance in those areas of professional practice best tested in the workplace”

Some principles of assessment

Utility Validity Reliability Educational impact Acceptability Feasibility

Why workplace-based assessment? Tests something important and different from other components  “Does do versus can do” Reconnects assessment with learning Has high educational impact Valid and reliable In keeping with PMETB guidance

The WPBA framework

The components of the framework An integrated package comprising: A competency-based training record and… Two externally marked work-based tools That applies over an entire training envelope (3 years from August 2007)

The educational model of WPBA for nMRCGP

The competency-based training record

Key features Competency-based Developmental Evidential Locally assessed Triangulated

Competency-based 12 competency areas Best tested in the workplace setting Developmental progression for each competency area Competency demonstrated “when ready” Process is learner led

The 12 competency areas 1.Communication and consulting skills 2.Practising holistically 3.Data gathering and interpretation 4.Making a diagnosis/ making decisions 5.Clinical management 6.Managing complexity and promoting health 7.Primary care administration and IMT 8.Working with colleagues and in teams 9. Community orientation 10.Maintaining performance, learning and teaching 11.Maintaining an ethical approach to practice 12.Fitness to practice

Developmental progression “ a process of monitoring student’s progress through an area of learning so that decisions can be made about the best way to facilitate future learning”

Evidential Notion of multiple sampling From multiple perspectives Tool-box of “approved” methods (locally assessed and national external tools) Sufficiency of evidence defined

Locally assessed Assessed by clinical supervisor in hospital or general practice setting Regular reviews at 6 month intervals by educational supervisor  Review all the assessment information gathered  Judge progress against competency areas  Provide developmental feedback

Triangulated Internally:  Different raters  Local tools (e.g. CBD, COT, mini CEX, DOPS)  Different settings (hospital and general practice) Externally:  Multiple raters  Externally marked tools (MSF and PSQ)  Different settings (hospital and general practice)

Gathering the evidence about the learner’s developmental progress

Evidence from Locally assessed tools External tools, and… Naturally occurring information

Local tools CBD (case based discussion) COT (consultation observation tool) mini-CEX (clinical evaluation exercise) DOPS (direct observation of procedural skills)

Case based discussion Structured oral interview Designed to assess professional judgement Across a range of competency areas Starting point is the written record of cases selected by the trainee Will be used in general practice and hospital settings

COT Tool to assess consultation skills Based on MRCGP consulting skills criteria Can be assessed using video or direct observation during general practice settings

Mini CEX Used instead of COT in hospital settings

DOPS For assessing relevant technical skills during GP training:  Cervical cytology  Complex or intimate examinations (e.g. rectal, pelvic, breast)  Minor surgical skills Similar to F2 DOPS

External tools MSF (multi-source feedback) PSQ (patient satisfaction survey)

MSF Assessment of clinical ability and professional behaviour Rated by 5 clinical and 5 non clinical colleagues on 2 occasions in GP practice Simple web based tool Is able to discriminate between doctors BUT Needs skill of trainer in giving feedback

PSQ Measures consultation and relational empathy (CARE) 30 consecutive consultations Central optical scanning and generation of results Can differentiate between doctors BUT Needs skill of trainer in giving feedback

Naturally occurring evidence From direct observation during training “tagged” against appropriate competency headings Other practice-based activities

Monitoring progress and final decisions

During training Meeting at end of ST1 and ST2 A Deanery based panel will:  Review the ETR of any trainee who: “Fails “ in any of the external tools during training Where requested by the educational supervisor i.e. where there may be a problem with the trainee’s progress in performance against the 12 competency areas  Ensure the trainee is making satisfactory progress

The Final Judgement The trainer makes a recommendation as to whether the registrar has achieved competence in all 12 ETR areas at the end of training This together with achievement in the external workplace assessments will be reviewed by a Deanery panel

4 possible outcomes: Pass in ETR and external assessments – clear pass. Fail in ETR and fail in external assessments – clear fail. Referred to Deanery panel but no change in decision Pass in ETR but fail in external assessments – referred to Deanery panel for decision Fail in ETR but pass in external assessments – referred to Deanery panel for decision

Workplace-based assessment ST1 Interim review Based on evidence: *3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report Interim review Based on evidence: 3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report MSF x 2 PSQ 6M * COT if GP post ** if appropriate Deanery panel if unsatisfactory

Workplace-based assessment ST2 Interim review Based on evidence: *3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report Interim review Based on evidence: *3 x mini-CEX 3 x CBD **Clinical supervisors report **DOPS 6M * or COT if GP post ** if appropriate Deanery panel if unsatisfactory

Workplace-based assessment ST3 Interim review Based on evidence: *6 x COT 6 x CBD **DOPS **Clinical supervisors report Final review Based on evidence: 6 x COT 6 x CBD MSF x 2 PSQ **DOPS 6M * mini-CEX if hospital post ** if appropriate Deanery sign off or panel review if unsatisfactory