Direct Observation: There is an App for That! Aaron W. Bernard MD Nicholas E. Kman MD Douglas A. Rund MD Sorabh Khandelwal MD The Ohio State University College of Medicine 1

Background  Direct observation of clinical skills is a critical component of performance assessment.  A portfolio of observations over time can be useful;  To better appreciate learner strengths and weaknesses allowing for accurate feedback.  To improve the accuracy of summative assessments. 2

Objective of Innovation  To promote portfolio development and improve assessment accuracy through the creation of a standardized method to document and store observations. 3

Background of Innovation  A working group of clerkship directors was formed and a standardized paper-based form was create to be used across all third and fourth year clerkships.  Weaknesses immediately appreciated regarding the paper based form; 1.Availability of form at the time of observation. 2.Logistics of keeping track of the forms. 3.Sharing of information between clerkships in a timely and useful manner. 4

5  An iPhone/iPod Touch application was created to overcome these challenges.  Prior to an observation, a student can import the demographic information. DO App 5

 The device is then handed to the attending 6

7  Different domains of observation exist to choose from DO App 7

8  Checklist are completed using touch screen technology. DO App

99

Innovation  The student and faculty can review all their observations in an electronic portfolio. 10

Feedback From Students Positive  Students appreciated not having to “find the form.”  Students found the application easy to use.  They appreciated the ability to review their observations electronically.  One place for all observations.  24 hour access Negative  Some students reported there iPod Touch given to them during the first year of medical school was broken, lost, or stolen by the fourth year. 11

Feedback From EM Faculty Positive  Drop down checklists easy to use  The option for detailed feedback was appreciated.  Most prefer this to the paper form. Negative  Privacy concerns – text messages to the students phone during the observation.  The faculty needs to inform the patient of what is being done to prevent the perception of texting during the encounter. 12

Conclusions  An application to record direct observations and store them in an electronic portfolio was created and received excellent feedback from students and faculty. 13