Work Place Based Assessment Dr Stephen Hailey

Aims Why WPBA? What tools? When applied? How to apply them? A bit of theory Pros and cons What tools? When applied? How to apply them? A bit of practice....

Miller’s Triangle

Pros: measuring actual performance Assessing the unmeasurable? What doctors do in controlled assessment conditions does not always collate with actual performance. Placing the assessment in the workplace helps to find out what a clinician actually DOES.

Pros: High educational impact Traditional assessments maximise reliability or reproducibility Negative educational impact as no meaningful feedback WPBA less reliable but can have high educational impact Assessment is ‘built in’ rather than ‘bolt on’

Pros: Multiple sampling The more measurements you take the more reliable the overall picture WPBA use multiple methods and multiple sampling

Cons: poor reliability Inter & Intra observer variability Poor application of criteria Poorly trained assessors Poor calibration of assessors

Cons: trainer vs. assessor

Trainer vs. assessor Potential for conflict of roles Being assessed by someone who has a vested interest in your performance It is vital that both trainer and the trainee understand the distinction of these roles Assessment needs to clearly defined from teaching

The tools MSF COT Mini CEX DOPs CbD PSQ CSR

Competency area MSF PSQ COT CbD CEX Supervisor structured report Communication & consultation skills Practising holistically Data gathering and interpretation Making a diagnosis / decisions Clinical management Managing medical complexities Primary care admin & IMT Working with colleagues & in teams Community orientation Maintaining performance, learning & teaching Maintaining an ethical approach Fitness to practice

ST1

DOPS: Direct Observation of Procedural Skills There are eight mandatory procedures to be covered: Application of simple dressing Breast examination Cervical cytology Female genital examination Male genital examination Prostate examination Rectal examination Testing for blood glucose Some of these procedures may be combined e.g. prostate and rectal examinations

Case Based Discussion A structured interview designed to explore professional judgement exercised in clinical cases GPStR is responsible for selecting cases Ensure that a balance of cases

CBD ST1 and 2, the GPStR will select two cases ST3, the GPStR will select four cases Including all documentation 1 week prior to discussion Trainer will select cases to be discussed

CBD About 20min for each discussion with 10min feedback

CBD Important points Selecting the cases Quality of record keeping Planning the questions Documenting the outcomes Structured feedback

Consultation Observation Tool

Multi-source feedback The MSF highlights two things: performance (areas to be commended) possible suggested areas for development  Need to do 2 SETS OF 5 PER YEAR IN ST1 (clinicians only) NONE IN ST2 2 SETS OF 10 in ST3 (5 clinicians and 5 non-clinicians)

MSF and feedback Preparation ECO model Specific Describe not judge Empathy

Patient satisfaction questionnaire 40 questionnaires Inputted by practice Results released to trainer Shared in meeting with trainee Feedback guidance applies