Practice CSE-3: A Formative, Low Stakes Experience for Third Year Family Medicine Clerkship Students Jeri R. Reid, MD Donna M. Roberts, MD Mary B. Carter, MD, PhD Carrie Bohnert, MPA

Objectives  Outline development of CSEs  Recognize the challenges of CSEs  Review formative vs. summative SP experiences  Introduce concept of “Practice CSEs” Case development Evaluation Expansion to other clerkships Future challenges

Background- Step 2 CS  “Step 2 CS uses standardized patients to test medical students and graduates on their ability to gather information from patients, perform physical examinations, and communicate their findings to patients and colleagues” –USMLE 12 patient encounters (live and telephone) 8 hours of testing Notes include history, physical exam, data interpretation, and diagnostic studies

Background- Clinical Skills Exam- CSE-3  Developed by U of L SOM to provide practice for national exam but has become a “must pass” for medical school graduation. 4 patient encounters (3 live and 1 phone) 2 hours of testing (30 minutes each case) Notes are same as Step 2 CS with additional treatment section and 2 added minutes for note writing

Expandable boxes 3 spaces for Hx and PE findings Type Dx here Please make sure patient name is correct FIRST 10 Minutes Use the arrow or mouse to move up and down

FIRST 10 Minutes Last 2 Minutes Use the arrow or mouse to move up and down Proctor Stops Clock, then Restarts

Background- Practice CSE-3  Developed by Family Medicine Clerkship Faculty in response to participation in pilot testing of CSE-3 cases and the need to change SP experience from summative to formative 2 patient encounters (1 live and 1 phone) 1 hour of testing (30 minutes each case) Notes same as CSE-3 50 points added to clerkship grade for successful completion; remediation provided

Pilot Testing  Prior to implementing the high stakes CSE-3, faculty was asked to participate in a pilot test  One of our clerkship faculty volunteered and found the experience anxiety- provoking due to: Difficulty finishing tasks in assigned time Unfamiliarity with generating computerized note Need to adapt note to different format than traditional SOAP

SP Clerkship Experience  Prior to 2011, clerkship had summative standardized patient testing  Faculty took time to carefully grade but students rarely came back to get this feedback  Building an SP experience into the middle of the clerkship would allow for feedback Verbal during mid-clerkship phone call Written at debriefing when actual graded notes and checklists could be reviewed

Case Adaptation  The faculty member who developed the CSE-3 was very excited to help with the development of our cases  Clerkship and CSE-3 faculty and staff met and began converting validated family medicine cases to the same format as the CSE-3 4 cases have been adapted to date  2 live  2 phone

Implementation  No additional SP lab time needed  Clerkship faculty did not need to do additional training with SPs Cases were already familiar to most SPs Detailed information provided to SP’s for self-study  PowerPoint orientation provided to students on Blackboard Brief recap with Q & A day of testing

Feedback  After completing each note, the students are provided with an ideal note and some case-specific educational information  After each case, students get a 5 minute feedback session with SP This does not occur after their CSE cases Many of the SPs have also done CSE-3

Grading  The notes and checklists are graded by SP staff according to strict rubrics  Faculty review the graded materials Comments added for more educational value Rubrics adjusted if needed  Results of testing are made available to students Discussed at mid-clerkship phone calls Reviewed at debriefing session

Evaluation  Students complete a survey at the end of the testing 14 questions using Likert scale 2 open ended questions  What went well  Suggestions for improvement  Survey at end of their CSE-3 exam contained 2 Likert questions relating to the Practice CSE-3 Helped them to prepare for CSE-3 Wished there were more of these

The PowerPoint provided on Blackboard before the exam helped me prepare for the Practice CSE The introduction provided by faculty helped prepare me for the Practice CSE The SPs I encountered during the Practice CSE-3 were believable 4.4 The Practice CSE-3 exam was administered in an organized fashion 4.3 The 2 cases on the Practice CSE-3 exam were sufficient to practice history and physical skills 4.0 The 2 cases on the Practice CSE-3 exam were sufficient to practice communication skills 4.3 The requirement to do teach-back during one of the cases increased my comfort level using this communication skill 3.9 The feedback session with the SP after each Practice CSE-3 case was helpful 4.3 It is important to me that my Practice CSE-3 notes and checklists are graded similarly to the CSE-3 exam 4.5 It is important to me that I receive a report of my scores on the Practice CSE I prepared for the Practice CSE-3 by reviewing the materials on Blackboard 4.2 I really tried to do well on the Practice CSE-3 exam today 4.6 I feel that the cases I saw on the Practice CSE-3 are representative of what I will see on the CSE-3 exam 4.1 I feel more prepared to take the CSE-3 exam now that I have experienced the Practice CSE The Survey Results

Open Response 1  Responses to “What went well with today’s exam?” included recurring descriptive terms Organized Believable- SPs and cases Valuable feedback- from SP & ideal note Good practice- for CSE-3 and for focused H&P in general Desire for more practice on other rotations

Open Response 2  In response to “What did not go well with exam and/or what suggestions for improvement?” Ran out of time- with SP and notes Phone case was a new, awkward experience Desired more introductory materials to review

CSE-3 Survey Results  The practice CSE-3 exam that I encountered during my family medicine clerkship helped to prepare me for today’s exam Average Likert score 3.7  I wish there were more practice CSE- 3 exams during the 3 rd year to help students prepare for the CSE-3 exam Average Likert score 3.8

Future Challenges  Providing all students with a practice experience before they take their high stakes exam Plans have been made to present this project to other clerkships  Adapting the testing to recent changes made in the in-house and national testing formats  Responding to student feedback by adding more orientation materials including sample cases and notes

Future Challenges  Lack of funding for creating practice CSE-3s during core clerkships  Lack of time and space in SP clinic to add more SP interactions

Questions/Comments???