1. Dr Simon Watmough, SFHEA Lecturer and Senior tutor, University of Liverpool Exploring the impact of prescribing error feedback Michael Lloyd, Sarah O’Brien, Niall Furlong, Kevin Hardy St Helens CCG / St Helens & Knowsley Hospitals Trust
Following a successful MA exploring what feedback doctors receive on PEs, a proposal to undertake further research as part of a PhD was drafted to explore the impact of good quality, formative, constructive feedback on PE rates. Funding for tuition fees was received initially and the project presented to the medical director to gain further support and professional insight. This led to an opportunity to undertake the PhD fully funded as a seconded research post. Funds were generously received via research, pharmacy and most notably the diabetes department.
Prior to this project feedback would have been ’direct’ (i.e. you’ve done this, it should be this, can you change this for me?) and opportunistic or ad hoc at best lacking the key principles of ‘constructive’ feedback needed to invoke behavioural change.
I believe that the project will be sustainable with ongoing facilitation of the project, perhaps as part of a wider prescribing safety role. We are expecting a change in pharmacist culture and so to get them to deliver consistent feedback autonomously and without prompt will require time. Feedback has not been delivered on medical admissions yet although the consultant have been requesting it for the past 3 months. It is anticipated to start here in September, and logistically, it will be interesting to see how this goes.
We have recently started a “one a week” campaign for reporting of PEs. Advanced discussions with the pharmacy hierarchy suggest a similar approach can be adopted for this project and it is anticipated that we can catalyse a learning culture where delivery of feedback on PEs is part of a pharmacist’s routine clinical standards and no different than ordering a medication for a patient.

2. Exploring the Impact of formalised prescribing error feedback
Background / Methods
Summary of results
Discussion and conclusion
Next steps

3. Background
Prescribing errors (PEs) are prevalent in the hospital setting with reports suggesting error rates of 1.5% to 52%. 1,2
Recent PE audits in case setting suggest PE rates of approximately 25%
Where errors occur, resources are wasted and patient care compromised.
Interventions to address the problem have focused on educational and system interventions, yet prescribing errors remain a concern.
Depending on location, prescriber grade and the varying definitions of PEs
Which whilst higher than some seminal studies (i.e. EQUIP) , is commensurate with others (Ross et al, Reynolds et al etc)
Is feedback not an educationla intervention? Yes it is but it is aimed at changing prescribing behavior, not exclusively at prescribing education

4. Prescribers have reported a lack of awareness and feedback on their prescribing errors previously. 2,3
Without feedback, prescribers may have false perceptions of their prescribing. 4
Feedback is reported to improve task performance and underpins the experiential learning cycle to catalyze professional development 4
Feedback is considered most effective when it is constructive, raising self-awareness, identifying learning needs and motivating individuals to address those needs. 4
As pharmacists are a key defence in intercepting prescribing errors, they are considered best placed to deliver feedback to prescribers. 1

5. The project
Ward based clinical pharmacists currently based on 20/24 wards
Whiston Hospital is a 700 bed, 24 ward (or 25 with A&E) acute trust. Several hundred prescribers.
Wards were chosen where pharmacists would give prescribers feedback using a proforma
All pharmacists were briefed and trained by the researcher on;
Data collection (all pharmacists)
PE feedback (intervention group only)
Senior prescribers briefed by researcher in advance
Junior prescribers briefed by ward based pharmacists
Whiston is a 700 bed, 24 ward (or 25 with A&E) acute trust. Several hundred prescribers (MPs and NMPs):- sorry cannot get exact number but it is around 500 (230 trainees, 200 consultants, staff grade and then NMPs)
Where they will visit daily and perform medication histories, review the medications of all patients and provide medicines information to ward based teams. Where PEs are identified, they will be resolved through discussion with a prescriber. Feedback typically follows this intervention at a mutually convenient time.
On collection and rating of PEs (for inter-rater reliability), and for the intervention wards, training on delivery of constructive feedback via workshops.
The researcher was accessible on site to discuss potential feedback before delivery to the prescriber.
Researcher briefed consultant teams beforehand
Ward based pharmacists briefed junior doctors
Research information leaflets provided throughout

6. Methods
August 2014: 24 pharmacists recruited to one of four focus groups to explore attitudes of delivering prescribing error feedback prior to the intervention commencing.
Sep-Dec 2014: Pilot study. Prescribing audited on four wards (two intervention, two control/normal practice) to determine impact of feedback on prescribing error rates. Intervention group receive formal constructive feedback on their prescribing.
Oct-Dec 2014; 10 prescribers recruited to semi-structured interviews to explore attitudes on receiving feedback.
Sep-Dec 2015: Sustainability study. Prescribing audited on sixteen wards; (eight intervention, eight control/normal practice) to determine impact of feedback on prescribing error rates.
Aug-Dec 2015: 19 Pharmacists recruited to semi-structured interviews to explore attitudes of delivering feedback post-intervention.
*Ethical approval obtained for all aspects of the research.
August 2014: 32 pharmacists in department
Pilot wards: 4 wards, 2 intervention (8 prescribers total included) and 2 control (10 prescribers included)
Larger study: 16 wards, 8 intervention (36 prescribers total included) and 8 control (41 prescribers included)
Pharmacists post: All 19 involved in feedback interviewed (larger number as some had rotated and we interviewed those involved in the pilot study too)

7. Results Pharmacists views pre-intervention 1
Pharmacists recognized that feedback was essential for prescribers to learn.
Feedback was inconsistent and mainly directive.
Anxieties expressed over impact on relationships.

8. Prescribers views 2
Feedback unanimously valued.
Pharmacists considered credible facilitators.
Feedback considered an educational intervention to support professional development.
Prescribers reported increased feedback and information seeking behaviour, and a raised discretionary effort as by-products of the process.

9. Pharmacists views post intervention:
Feedback was unanimously welcomed and considered sustainable by all pharmacists.
Initial apprehensions subsided after delivery of first feedback session.
Improved self-worth and job satisfaction reported
Pharmacists felt their skills were ‘recognised’ more
Feedback process supports development of pharmacists and prescribers.
Additional educational interventions reported.

10. Quantitative results
Independent t-tests were used to determine the difference in mean change between PE rates (post minus pre PE rates) in intervention and control groups.
Pilot study: Mean PE rates were significantly lower in the intervention group (n=10) following feedback compared to the control group (n=12) (mean difference 25.45%, SD 9.68, CI to ) t(20)= , p<0.05, d=1.16).
Sustainability study: Mean PE rates were significantly lower in the intervention group (n=36) following feedback compared to the control group (n=41) (Mean difference of 23.73% (SD 3.47, 95% CI to ), t(75) = , p<0.05, d=1.6)
The independent t-test compares and determines if there is a significant difference between the means of two independent groups on a continuous dependent variable (in this example PE rate). Hence it can be used to answer our initial research question; is there a difference in mean change in PE error rates between control and intervention groups.
Why t-test? Good point: All are valid but it is focused on research question here. Repeated measures ANOVA or ANCOVA could arguably be used but they answer different questions, with more assumptions to be met without providing additional statistical inference. i.e. ANCOVA strictly would answer do prescribers with the same baseline PE rates have different PE rates after feedback? This assumes that all other controlling factors are likely to be similar but the conditions in each ward area are likely to be different akin to personality or the so called rules of the game. Additionally, as we could not strictly randomly allocate prescriber to feedback or not ANCOVA would produce likely biased results. So our research question, is there is a difference in the mean (change) PE rates is best answered with the t-test here.
d? This is cohens D or an effect size based on means and standard deviations of two groups. Instead of of saying okay its significant, it answers the so what as it is also > 0.7 and therefore a large effect size
Did you analyze results at severity of typology level? Yes we did, we used a chi-square test of homogeneity to determine if there was any difference in the binomial proportions of severity or error type. We performed these tests between and within groups and there was no difference in distribution in the intervention group although there was in the control group (if errors go unchecked in a particular area do they thus become more prominent?). We also performed chi squared tests within each group to determrien the impact of feedback on each subset of error severity and type with significant reductions reported across the board for the intervention group.
Is there an association between intensity of feedback and PE reduction? We performed a Pearson correlation and identified a non-significant association so it does not appear to be exclusively related to the number of feedback sessions.

11. Discussion
The data has shown a statistically significant reduction in PEs following delivery of constructive feedback to prescribers
The process was valued by prescribers and pharmacists who were considered credible facilitators. Feedback raised awareness of prescriber learning needs and generated further information and feedback seeking behaviour with pharmacists
This cooperation was noted by pharmacists who felt they were more integrated into the clinical team with improved self-worth
Such a co-operative approach to prescribing can only benefit patient safety and optimise patient care

12. Conclusion
Results are promising with positive impacts from feedback on prescribing, prescribers and the facilitators of feedback, pharmacists.
Further work is necessary to determine ongoing sustainability and the most effective method of delivering feedback in clinical practice

13. Thank you for listening
Any questions ?
References
1. Lloyd M, Watmough SD, O'Brien SV, Furlong N, Hardy K. Res Social Adm Pharm Sep 5. pii: S (15)
2. Lloyd M, Watmough SD, O'Brien SV, Furlong N, Hardy K. Br J Hosp Med (Lond). 2015; 2:76(12):713-8
3. Velo GP, Minuz P. Br J Clin Pharmacol 2009;67:624–628.
4. Lloyd M, Watmough SD, O’Brien SV, Furlong N, Hardy K. The Pharmaceutical Journal, 2016; Vol 296, No 7887, online