Enhancing Assessment – Experience from psychiatry Dr Andrew Brittlebank Associate Dean, RCPsych

What I’ll cover… Psychiatric culture Modifying the culture The results so far

“Competition can make people wary of assessment, and efforts to provide feedback on progress and attainment can unintentionally be seen as threatening...WPBA requires a change in culture” PMETB/AoMRC, 2009

The pre-MMC culture in psychiatry Supervision – Embedded Apprenticeship ‘see one, do one, teach one’? Assessment – High stake

Supervision Mandated since the 1970’s Inspected at college visits Variable degrees of compliance Embedded in psychiatry

Read about one, do one… “In Europe, being a psychiatric patient is a bit like being a passenger of a pilot who learned to fly by reading about it” Compernolle, 1998

High stake assessment MRCPsych examination – MCQ, essay, long case Supervisor report Appointment committees

Modifying the culture Patient safety MMC and training WPBA assessors Psychiatric skills should be coached

Patient safety Conversations with patient safety and experience teams – “How do you know your doctors are safe?” NHSLA – “It’s the economy, stupid”

Training in WPBA – since 2006 Reforms in medical education – Shift away from traditional apprenticeship Purpose of WPBA – Formative Tools The standards – Develop performance dimensions Observation – Assess what you see and what you hear Frame of reference – Peer discussions

Teaching psychiatric skills “Watching your trainee is a bit tiresome” People respect what you inspect – Renewed interest in teaching and learning psychiatric communication skills – Cascading examiner training – WPBA guides for Core and Advanced training

Results so far Participation Reliability Impact

Participation rates GMC Trainers’ Survey 2010 – Psychiatry WPBA

Reliability data for CbD Based on GeneralisabilityTheory Psychiatrists assess reliably

Impact Mixed picture Depends on the question and how it is asked

Impact of WPBA – qualitative Added burden for trainees: “It’s a hassle, getting people to do the assessments” Assessment drives learning: “It’s good to know where you stand, so you know what you have to do” As a tool for reflection: “It has been helpful to reflect on how much you think you’ve improved”

Impact of WPBA - quantitative Julyan, 2009 – May enhance effectiveness of supervision Menon et al, 2009 Concerns about the tools and practicalities Babu et al, 2009 – Positive view of training in WPBA – Mistrusted by trainees Menon et al, 2012 – Trainers more positive about WPBA’s than trainees – But trainers surveyed six months after trainees (2009)

In conclusion The introduction of WPBA demanded a change in practice and culture It is work in progress It takes time – five years minimum

Thank you! Questions?