Can a Brief On-line Education Tool Improve Surgical Resident Operative Dictations? A Prospective Evaluation Alicia Kieninger, MD, Yi Wei Zhang, MD, Anna Dowdle, Amy Braddock, MD, Diego Hernandez, MD, Amy Kirby, MD, Robert Robinson, MD

Background Surgical education encompasses much more than clinical and technical education –Patel, et. al, 2005 Survey of ENT program directors and recent graduates Topics lacking in current training programs –Coding and documentation –Office management –Reimbursement –“Business of medicine”

Background Proper documentation is the key to achieving adequate coding and reimbursement –Fakhry, et al., 2007 Survey of surgical trainees and attendings regarding knowledge of coding and proper documentation Residents scored 54% correct vs. 77% for attendings –Novitsky, et al., 2005 Deficiencies identified in resident operative notes would have led to $18,200 in reduced revenues compared to attending notes

Background How do we teach residents the skills they need in less time? –Impact of the 80 hour work week Limited time spent in hospital must focus on clinical and technical skills Jones, et al., 2008 –Implementation of a practice management program improved resident coding accuracy from 36% to 88% Eichholz, et al., 2004 –Brief teaching sessions improved ob-gyn resident dictation scores

Background The operative note –Communication of details important for patient care –Proper documentation for appropriate billing –A skill important for all surgical trainees

Purpose To determine if a brief on-line education module can improve the quality and content of surgical resident operative dictations

Methods Participants –General surgery residents at a community based teaching hospital IRB approval and informed consent obtained Resident operative dictations collected for a 1 month period before and after intervention –Blinded review by 2 of 4 participating attending surgeon reviewers Evaluation tool adapted from previously published, validated tool (Vergis, et al., 2008) Dictations scored for content, quality, overall

Methods Statistical Analysis –Before and after scores compared using paired t- test (SPSS v. 21.0) Residents serve as their own controls –p  0.05 significant

Methods Module created for hospital- wide on-line compliance testing program –Brief power-point presentation Outlining keys to content and quality of operative dictations –Several multiple choice questions

Results TotalJunior (PGY 1-3) Senior (PGY 4-5) Residents1174 Dictations (pre/post) 144 (86/58) 76 (46/30) 68 (40/28)

Results All Residents NS

Results Junior vs. Senior Residents p<0.0001*

Results Paired t-Test p= 0.314p= 0.163p= 0.230

Conclusions Senior resident dictations scored significantly higher than junior resident dictations There was a trend toward improved dictations after participation in the educational module for the group as whole Using the paired t-test, individual residents showed improvement after our intervention when compared to their own pre-intervention performance

Limitations Small sample size Heterogeneity of operative procedures

Future Directions Expand information in education module based on deficiencies in operative notes Focus evaluations on single operative procedure to improve validity Expand to other residency programs Add a component of direct one-on-one feedback to residents to improve readability

References Patel, A.T., R.M. Bohmer, et al. (2005). “National assessment of business-of- medicine training and its implications for the development of a business-of- medicine curriculum.” Laryngoscope 115(1): Fakhry, S.M., L. Robinson, et al. (2007). “Surgical residents’ knowledge of documentation and coding for professional services: an opportunity for a focused educational offering.” Am J Surg 194(2): Novitsky, Y.W., R.F. Sing, et al. (2005). “Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents.” Am Surg 71(8): Jones, K., R.A. Lebron, et al. (2008). “Practice management education during surgical residency.” Am J Surg 196(6): Eichholz, A.C., B.J. Van Voorhis, et al. (2004). “Operative note dictation: should it be taught routinely in residency program?” Obstet Gynecol 103(2): Vergis, A., L. Gillman, et al. (2008). “Structured assessment format for evaluating operative reports in general surgery.” Am J Surg 195(1):

Thank You