1. VIRTUAL PATIENT TECHNOLOGY IN A PSYCHIATRY CLERKSHIP - A PILOT STUDY Kristi S. Williams, M.D. 1, John M. Wryobeck, Ph.D. 1, Walter Edinger, Ph.D. 1, Angele McGrady, Ph.D. 1, Andrew Chin, 2 G.H. Fors, Ph.D. 3, and Nabil Zary, Ph.D. 3 1= University of Toledo, 2 = The Ohio State University, 3 = Karolinska Institutet Objective: To describe an innovative way to improve medical student ability to diagnose and treat psychiatric disorders. This poster describes a pilot project in which virtual patient (VP) technology was used to assess medical student knowledge of psychiatric disorders. Methods: The VP program (Web-SP) was presented to the Psychiatry clerks at the beginning of their 6-week clerkship. Clerks were provided individual access to the case, authored by KSW, for 6 days at the midpoint of their clerkship. Students were prompted to provide primary and differential diagnoses, justification for the diagnoses and a treatment plan. The case diagnoses were Bipolar II Disorder, Major Depressive Disorder, Generalized Anxiety Disorder, Panic Disorder, Alcohol Abuse, Chronic Back Pain, Hypertension and Hypothyroidism. Formative feedback on the correct diagnoses, differential diagnoses, treatment and rationale for each was provided to students after case completion. Results: Three groups of psychiatry clerks (total N=52) completed three different cases. Sixteen (31%) correctly identified the primary diagnosis and 31 (60%) correctly identified the secondary diagnosis across the three cases. Forty-three (83%) students listed the primary diagnoses in either the primary or differential diagnosis. Conclusions: VP programs offer an additional method to assess and evaluate medical students’ ability to diagnose and treat psychiatric illness in a cost and time effective manner. Use of virtual patients also allows for evaluation of students’ critical reasoning ability. Case results can be utilized to provide feedback to faculty on teaching effectiveness in specific clinical topics and identify areas that need further elucidation. Web-SP, an interactive, case based clinical assessment tool using a template patient scenario, was the virtual patient software program used Institutional Review Board determined the project to be exempt status research Test cases were authored by KSW, after discussion of content with the clerkship director, and designed to be of moderate difficulty Cases were reviewed by 1-2 faculty members in the Department of Psychiatry Web-SP software was presented to the Psychiatry clerks at the beginning of their 6-week clerkship Clerks were then provided individual access to the exam case at the midpoint of the clerkship (each block of students received a different case) for 6-7 days Students were prompted to provide primary and differential diagnoses, justification for the diagnoses and a treatment plan Correct diagnoses, differential diagnoses, treatment and rationale for each were provided to students in written form, after the case was closed In addition to diagnoses and treatment variables, the total number of questions each student asked and the amount of time spent asking the questions and navigating the program prior to giving a diagnosis were calculated VP programs offer an additional method to assess and evaluate medical students’ ability to diagnose and treat psychiatric illness in a cost and time effective manner Use of virtual patients also allows for evaluation of students critical reasoning ability Case results can be utilized to provide feedback to faculty on teaching effectiveness in specific clinical subjects and identify areas that need further elucidation **Had the correct primary diagnosis listed in either primary or differential DISCUSSION Our psychiatry clerkship was looking for an innovative way to educate and evaluate medical student ability to diagnose and treat psychiatric disorders without placing additional time burdens on faculty members Virtual patient technology can be used to provide formative or summative feedback to students on a variety of psychiatric diagnoses, depending on the emphasis desired Specific domains of didactic and clinical education can be targeted and enhanced to improve the educational experience Advantages of VP programs Cost effectiveness Ability to assess, modify and improve educational outcomes based on feedback from cases Ability to develop cases to assess specific diagnoses, biopsychosocial cultural, and ethical issues Ability to tailor didactics to VP case outcomes Facility of case modification to meet specific goals Authors tried to make cases “real life” to challenge students to identify diagnostic criteria rather than rely on checklists of symptoms Common requests from student feedback Provide specific timeline of symptoms Incorporate more specific information about symptoms Challenge Technological malfunctions – should have at least one test case to work through problems as well as to familiarize students with the process CONCLUSIONS DISCUSSION ABSTRACT BACKGROUND METHODS CONTACT INFORMATION Kristi Skeel Williams, M.D., Associate Professor, Residency Program Director University of Toledo - 3000 Arlington Avenue, MS # 1193 Toledo, Ohio 43614 -2598 E-Mail: Kristi.Williams@Utoledo.edu RESULTS 67% of students recommended that this form of evaluation should be continued STUDENT FEEDBACK ACROSS ALL CASES Not at all Very much CaseNNumber of Questions Accessed Minutes to Dx Correct Primary Dx Correct Primary or Diff Dx** Correct Secondary Dx #1 Bipolar II, Alcohol Abuse 1472 (37-120)25 (12-49)4 (29%)13 (93%)9 (64%) #2 MDD, Panic Disorder 1798 (28-179)26 (16-58)5 (29%)13 (76%)7 (41%) #3 Bipolar II, GAD, Alcohol Abuse 2197 (34-209)55 (24-156)7 (33%)17 (81%)15 (72%) Average 89 (33-169)35 (17-88)5 (30%)14 (83%)14 (59%)