The SOAP Note Project: Does Intensive Written Feedback Improve Student Post-Encounter Notes? Anne Walling, MB, ChB Scott Moser, MD Alan Blakely, PhD John Dorsch, MD

Rationale Accurate, organized, complete, succinct SOAP/PENs are:  Essential in patient care.  A significant element of USMLE Step II- CS grade. BUT Minimal literature available on how to teach this critical skill or evaluate outcomes of teaching interventions. Minimal literature available on how to teach this critical skill or evaluate outcomes of teaching interventions.

Our Problems  Poor student notes on CSA and OSCE  Students not prepared for USMLE-CS PENs after every encounter  Diverse styles/types of notes by faculty  Problems in scoring notes  Requires “coaching” a skill NOT lecturing  Limited opportunities/strategies to intervene  Etc, etc, etc

Research Questions 1.Does systematic written feedback on SOAP/PEN notes in formative SP sessions significantly improve these notes? 2.Are specific aspects of SOAP/PENs particularly amenable to improvement with written feedback?

Methods 1 Study Population: All 48 MS3 students on 12 week FM/Amb Med-Geriatrics block Controls: No feedback Students (24)during first half of academic yr (July-December) Students (24)during first half of academic yr (July-December) Intervention Group: Feedback Students (24) during second half of academic yr 04-05; same 7 SP encounters + written feedback on SOAPs(PENs)

Methods 2 Intervention: Written feedback from one faculty (AW) on each SOAP note (7 notes per student):  based on a “gold standard” for each case case  delivered to student within 3 days  addressed both medical content and format (style) of note. format (style) of note.

Feedback Example l Question : History: Include significant positives and negatives from history of present illness, past medical history, review of system(s), social history, and family history. l Response : Always start with age, sex, race and reason for visit. “74 white widow ….Pt was brought in by her daughter who has been having some concerns about the pts memory as l of late. Pt was apparently going to see a friend and became confused and was lost for approx two hours. Pt also forgets to give the family telephone messages. Pt is currently living at her daughter’s home and her daughter is in charge of dispensing her Zoloft. Pt herself is unaware of any problems other then what is "normal" for her age. Pt daughter is unavailable at this time to interview l Need to address the subheadings PMH, SH (includes ADLs and IADLs) FH, and RoS In elderly RoS may be only indication of serious comorbidity. See guidelines for SOAP notes l Question : Physical Examination: Indicate only pertinent positive and negative findings related to the patient's chief complaint. l Response : Always start with vitals followed by general impression of patient (see guidelines) l MMSE 10/30 (seems low) with most losses occurring on orientation to time and place. Registration and recall were also weak. as well as attention. l PE: CV RRR no m/r/g l Resp CTA-B l Question : Differential Diagnoses: In order of likelihood. List five. l Response : 1) Dementia--most likely Alizhiemer’s l 2) MDD-Abbreviation may not be recognized – interview does not have evidence for major depression. Vascular and other forms of dementia/memory impairment more likely

Methods 3 Outcomes measured: Student scores on SOAP notes in July 05, high stakes, 12-station CSA (modelled on USMLE-Step 2-CS)  3 notes per student  Trained faculty graders  Score 25 for key words + 5 organization/style  Format allowed analysis of subcategories and tracking of targeted feedback items.

Targeted Feedback Items  Medical content items  Use of sub-headings (HPI, PMH, SH etc)  “Formula” for HPI introduction ( e.g.“ 74 yr old white male c/o new onset pain……”)  Document vital signs  Provide general impression of patient  Document allergies

Results SOAPScoreStandardizedNoteContentNoteFormatTargeted Feedback Items Controln=2460%57%86%38% Interventionn=2468%62%93%65% p

Conclusions Compared to controls, feedback group had:  Higher total SOAP scores on CSA but not statistically significant (p=0.077)  Highly significant greater incorporation of targeted feedback items (p= )

Limitations of Study 1. Cohort study. Small groups 2. Scoring of CSA note heavily content-dependent (25 pts content + 5 for format/style) 3. SP programs highly individualized

Curriculum Outcomes 1.Written SOAP feedback now provided to all students on block plus the IM clerkship. 2.SOAP note workshop added to year 3 orientation 3.Explicit instructions on SOAP notes provided at orientation to block. 4.Initiated review of student SOAP notes and preceptor development during academic detailing visits. 5.Provide SOAPs and checklists as feedback to faculty teaching specific topics

Reflections  “Orphan” - invisible topic in curriculum  Feedback expensive but valuable  Consistency essential (“gold standards”, guidelines, single reviewer)  Coaching students NOT pimping/ nagging  Stressed as a core practical skill  Positive reinforcement from senior students/residents

Students Said………… Knowing the note was waiting made me do a better job of the case. When I sat down at the computer at CSE, I heard you say ‘use the subheadings, don’t forget the OTCs and allergies!’

Discussion