Promoting Excellence in Family Medicine nMRCGP Workplace-based Assessment March 2007

Promoting Excellence in Family Medicine nMRCGP Integrated assessment package comprising: Applied knowledge test (AKT) Clinical skills assessment (CSA) Workplace-based assessment (WPBA)

Promoting Excellence in Family Medicine Workplace-based assessment “The evaluation of a doctor’s progress over time in their performance in those areas of professional practice best tested in the workplace.”

Promoting Excellence in Family Medicine Some principles of assessment Validity Reliability Educational impact Acceptability Feasibility

Promoting Excellence in Family Medicine Why workplace-based assessment? Tests something important and different from other components “Does do versus can do” Reconnects assessment with learning Has high educational impact Valid and reliable In keeping with PMETB guidance

Promoting Excellence in Family Medicine The WPBA framework An integrated package comprising a competency-based training record that applies over an entire training envelope (3 years from August 2007)

Promoting Excellence in Family Medicine The educational model of WPBA for nMRCGP

Promoting Excellence in Family Medicine The competency-based training record Competency-based Developmental Evidential Locally assessed Triangulated

Promoting Excellence in Family Medicine Competency-based 12 competency areas Best tested in the workplace setting Developmental progression for each competency area Competency demonstrated “when ready” Process is learner led

Promoting Excellence in Family Medicine The 12 competency areas 1. Communication and consulting skills 2. Practising holistically 3. Data gathering and interpretation 4. Making a diagnosis/ making decisions 5. Clinical management 6. Managing complexity and promoting health 7. Primary care administration and IMT 8. Working with colleagues and in teams 9. Community orientation 10. Maintaining performance, learning and teaching 11. Maintaining an ethical approach to practice 12.Fitness to practice

Promoting Excellence in Family Medicine Developmental progression “a process of monitoring a student’s progress through an area of learning so that decisions can be made about the best way to facilitate future learning”

Promoting Excellence in Family Medicine Evidential Notion of multiple sampling From multiple perspectives Tool-box of “approved” methods (locally assessed and national complementary tools) Sufficiency of evidence defined

Promoting Excellence in Family Medicine Locally assessed Assessed by clinical supervisor in hospital or general practice setting Regular reviews at 6 month intervals by trainer/educational supervisor Review all the assessment information gathered Judge progress against competency areas Provide developmental feedback

Promoting Excellence in Family Medicine Triangulated Different raters Many tools (e.g. CBD, COT, mini CEX, DOPS, MSF and PSQ) Different settings (hospital and general practice)

Promoting Excellence in Family Medicine Gathering the evidence about the learner’s developmental progress

Promoting Excellence in Family Medicine Evidence from Locally assessed tools Complementary tools and… Naturally occurring information

Promoting Excellence in Family Medicine Tools for Evidence CBD (case based discussion) COT (consultation observation tool) mini-CEX (clinical evaluation exercise) DOPS (direct observation of procedural skills) MSF (multi-source feedback) PSQ (patient satisfaction questionnaire)

Promoting Excellence in Family Medicine Case-based discussion Structured oral interview Designed to assess professional judgement Across a range of competency areas Starting point is the written record of cases selected by the trainee Will be used in general practice and hospital settings

Promoting Excellence in Family Medicine COT Tool to assess consultation skills Based on MRCGP consulting skills criteria Can be assessed using video or direct observation during general practice settings

Promoting Excellence in Family Medicine Mini CEX Used instead of COT in hospital settings

Promoting Excellence in Family Medicine DOPS For assessing relevant technical skills during GP training: Cervical cytology Complex or intimate examinations (e.g. rectal, pelvic, breast) Minor surgical skills Similar to F2 DOPS

Promoting Excellence in Family Medicine MSF Assessment of clinical ability and professional behaviour ST1 Rated by 5 clinical colleagues, 2 occasions ST3 Rated by 5 clinical and 5 non-clinical colleagues on 2 occasions Simple web based tool Is able to discriminate between doctors Needs skill of trainer in giving feedback

Promoting Excellence in Family Medicine PSQ Measures consultation and relational empathy (CARE) 30 consecutive consultations in GP setting Central optical scanning and generation of results Can differentiate between doctors Needs skill of trainer in giving feedback

Promoting Excellence in Family Medicine Naturally occurring evidence From direct observation during training “tagged” against appropriate competency headings Other practice-based activities

Promoting Excellence in Family Medicine Monitoring Progress Interim reviews with trainer 6 month intervals ensure the trainee is making satisfactory progress agree training needs

Promoting Excellence in Family Medicine Monitoring Progress Deanery Panel meeting at end of ST1 and ST2 reviews the training records of every trainee face to face review with trainees when unsatisfactory achievement in either of the complementary tools or when requested by the educational supervisor

Promoting Excellence in Family Medicine Workplace-based assessment ST1 Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x MSF 1 x PSQ * DOPS ** Clinical supervisors’ report ** Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x MSF 1 x PSQ * DOPS ** Clinical supervisors’ report ** 6 month 12 month Deanery panel if unsatisfactory * if GP post ** if appropriate

Promoting Excellence in Family Medicine Workplace-based assessment ST2 Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x PSQ * DOPS ** Clinical supervisors’ report ** Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x PSQ * DOPS ** Clinical supervisors’ report ** 18 month24 month Deanery panel if unsatisfactory * if GP post ** if appropriate

Promoting Excellence in Family Medicine Workplace-based assessment ST3 Interim review Based on evidence: 6 x COT 6 x CBD 1 x MSF DOPS ** Final review Based on evidence: 6 x COT 6 x CBD 1 x MSF DOPS ** PSQ 30 month34 month Deanery sign off or panel review if unsatisfactory ** if appropriate

Promoting Excellence in Family Medicine The final judgement The trainer makes a recommendation as to whether the trainee has achieved competence in all 12 areas at the end of training

Promoting Excellence in Family Medicine Review by Deanery Panel Review of e-portfolio if satisfactory level achieved in training record Review of e-portfolio and face-to- face meeting with trainee, if satisfactory level not achieved