Peer feedback in formative assessment to aid learning: Annie Cushing Angela Hall Olwyn Westwood Centre for Medical Education s CETL 2008

Drivers Professional Skill – Multisource Feedback as part of professional practice NSS – students want feedback Formative feedback as a support for learning Large student numbers Proximity of practice/feedback/practice s CETL 2008

Study 1 in Nov 2008 Graduate Entry Students: Year 1: n=93 Medical & Nursing [Adult, Child, Mental Health] 3 Stations Based on PEBL cases 5-minute station with 3-min feedback Actors play patient/carer

Study 1: Nov 2008 ( 78 attended = 84%) Students: Generate the marking criteria* Undertake all OSCE stations as candidates Undertake examiner role in 1 station. Actors give verbal feedback Evaluation by: Questionnaire*: 20 statement items Likert response scale Focus Group: nominal group technique. * Adapted from FAST project Open Univ/Sheffield Hallam (Brown et al 2003)

Q1. Being the examiner was helpful in providing clear instructions on what was expected of me (N=77)

Q5 The feedback was rapid and helpful N=78

Q13. I found it difficult giving feedback to my colleagues. (p <0.05)

Q20. I felt that I was able to be honest in my feedback

Focus Group (7) Helpful to Learning Seeing examples of: good and bad communications (content & delivery) (5 votes) differences between people in the same situation (5 votes) others behaviour helps you understand what you should do yourself (3 votes)

Unhelpful to Learning Checklist is too long (7 votes) Peers are amateurs, both at the skills being role-played and at feedback (5 votes) Some people are not brave enough in giving critical feedback (4 votes)

Anxiety about giving Negative Feedback I thought, You really should never say that in front of somebody, and I didnt, to be honest, have the guts to tell them. You want to say something positive but also you want to get across what they need to improve. ………. But it can be quite difficult, I think, because if youre peers, it's harder than if you're a tutor.

Feedback Sargeant J et al (2008) J Contin Educ Health Prof Three inter-related factors for acceptance of negative feedback: 1.Credibility – specificity, personal observation, explicit standards of performance 2.Emotion- when inconsistent with self perceptions of being a good doctor 3.Reflection – a good mediator of assimilation, acceptance and application

Incorporating Feedback I thought I'd done something quite well, and someone pointed out that actually I could have done it this way and it might have been better. And then seeing someone else do it, I thought, Oh well, yeah, thats a much better way of doing it. But it hadn't even occurred to me that I could improve on the way I was doing it. And I don't think it's until someone points it out to you in quite a specific way that you think, Well actually, yeah, Ill try that. And I've done it since, and it is better.

Learning Transfer With a patient who wanted a long conversation, I had to draw on the OSCE session. OK, how do I make sure that I'm actually quite interested in what you're saying now? Because I've got three other patients that I really need to deal with. (Nurse)

Study 2: May 2009 Changes in response to Study 1 Teaching on Giving Feedback ( video CETL ). What would you give feedback on? - generate a marking scheme How does the facilitator do with her feedback? - generate principles of feedback Behavioural, specific, balanced +ve with areas to improve.

Changes in Study 2 In response to Study 1 More feedback time: 4-mins interview and 4-mins feedback Actor to give written rather than verbal feedback Empathy Clarity Cases to be circulated in advance.

Feedback

Responses to Questionnaire Statements [1]

Responses to Questionnaire Statements [2]

Free Text Comments I understand it was part of the research but it definitely would have been valuable to have been given verbal feedback from the actor as well as written Really good – nice to see that we have improved since the last one, both in interviewing and giving feedback to colleagues Peer feedback: I think it is very helpful, although we are relatively at the same level so limited in depth of feedback Love the OSCE sessions with the actors. They are invaluable and a great learning tool.

I Introduced into Curriculum08 MBBS Core Curriculum in Year 3 Both Communication (3) and Clinical (3) stations: Faculty supported, supervised and guided sessions on peer feedback engage learners actively Learners need training on giving constructive feedback Learners value these sessions Resourcing – cost effectiveness

Spin-offs BSc Med Ed Effect of intervention on student performance in end of year summative OSCE marker station. Sample: GEP yr 1 cohort (formative OSCE) Graduates in Year 2

Thank you Any Questions?

References Sargeant J et al Challenges in multi-source feedback: Intended and unintended outcomes. Med Educ. 41: Nicol & MacFarlane (2004) Rethinking Formative Assessment in HE: a theoretical model and seven principles of good feedback practice Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education. 22 (2), Kurtz S, Silverman J Teaching & Learning Communication Skills in Medicine. Radcliffe Pub Oxford. (Calgary-Cambridge Guide) Weaver M. (2006) Do students value feedback? Student perceptions of tutors written responses, Assessment and Evaluation in Higher Education 31(3) pp Bing-You R.G., Paterson, P., Levine, M.A. (1997) Feedback falling on deaf ears: residents receptivity to feedback tempered by sender credibility, Medical Teacher, 19, pp

Cost effectiveness 120 students All got chance to: Examine 6 peers Practice 6 stations Be a patient in 2 stations Tutors = 8 Bodies = 12 Actor patient = 4