1. Evaluation of curricular integration of virtual patients: Development of a student questionnaire and a reviewer checklist within the electronic virtual patient (eVIP) project Sören Huwendiek, Helen Rani Haider, Burkhard Tönshoff, Bas de Leng ICVP’09 conference Krakow, Poland Centre for Virtual Patients

2. Introduction Centre for Virtual Patients The curricular integration of virtual patients is essential for their success To date there are no standardized instruments available to evaluate the curricular integration of virtual patients

3. Aim of the Project Development of a student questionnaire and reviewer checklist to evaluate the curricular integration of virtual patients with the primary aim of fostering clinical reasoning which is employable in different kinds of didactic scenarios using virtual patients  offers the opportunity to compare different integration modes and elucidate strategies of effective curricular integration Centre for Virtual Patients

4. Literature review Systematic literature review: Medline, ERIC, Google Scholar Search terms included: Blended Learning, Flexible Learning, Constructive Alignment, Curricular Integration, E-Learning and Virtual Patients  no standardized instrument to evaluate the curricular integration of virtual patients could be found  theoretical frameworks supporting the development and design of the questionnaire Centre for Virtual Patients

5. Literature informing the theoretical framework of the questionnaire (1) The Community of Inquiry Model by Garrison et al. as a general framework for the questionnaire:  learning occurs through the interaction of 3 core components: - cognitive presence - social presence - teaching presence Centre for Virtual Patients

6. Literature informing the theoretical framework of the questionnaire (2) Octagonal Framework of Blended Learning by Badrul Khan (2000) Criteria categorizing didactic scenarios by Heyer (2006) Principles of curricular integration of virtual patients, a focus group study among students at the University of Heidelberg (2009) Literature concerning educational strategies to promote clinical diagnostic reasoning (Bowen 2006) Centre for Virtual Patients

7. The questionnaire (1) 20 questions, clustered in 5 main categories: (A) Teaching presence (B) Cognitive presence (C) Social presence (D) Learning effect (E) Overall judgement Additionally: (F) 3 Open ended questions Centre for Virtual Patients

8. The questionnaire (2) 5-Point-Likert-Scale: (1) strongly disagree (2) disagree (3) neutral (4) agree (5) strongly agree (6) not applicable After each question free text field: briefly reasons for answer (optional) Centre for Virtual Patients

9. “Corresponding teaching event” to describe teaching and learning activities which go along with virtual patients definition of “corresponding teaching event” according to educational scenario (e.g. virtual patients used during problem-based learning, small-group-discussions after working with virtual patients)  In this manner questionnaire employable for different kinds of educational scenarios using virtual patients Centre for Virtual Patients

10. The reviewer checklist to characterize the curricular integration scenario of virtual patients in detail by teacher 12 items describing the educational scenario in detail 24 items clustered in the same 5 subsets used in the student questionnaire Centre for Virtual Patients

11. Process of refinement by eVIP experts first draft reviewed by eVIP-partners, refined, tested on target group and again refined final review through eVIP-partners Translation in local partners language and tested with students, refined and retranslated into English Final student instrument, translated in 6 languages, available on eVIP-website Final reviewer checklist in English, available on eVIP-website Centre for Virtual Patients

12. Study among 5 th year medical students Educational scenarios: - VPs integrated into the Paediatrics Course at Heidelberg Medical School using small group discussion after VP-Work - VPs as preparation for skills lab training (Paediatrics) Results:  the questionnaire and checklist proved to be very helpful in identifying strong areas and areas with need of improvement concerning the curricular integration of VPs Centre for Virtual Patients

13. Conclusion A multi-lingual student questionnaire and checklist to analyse and compare virtual patient curricular integration scenarios is now available First studies evaluating virtual patient integration scenarios at the University of Heidelberg proved the instruments to be helpful to optimize the educational scenarios  We hope the instruments will be widely implemented around the globe Centre for Virtual Patients

14. Bibliography  Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Assessing Social Presence In Asynchronous Text-based Computer Conferencing, Journal of Distance Education, ISSN: 0830- 0445, 2001. Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Assessing Social Presence In Asynchronous 2003. Khan, B.H.: A Framework for Open, Flexible and Distributed Learning, Proceedings of the Technological Education and National Development Conference, “Crossroads of the New Millenium” (2 nd, April 8-10, 2000, Abu Dhabi, United Arab Emirates) Heyer, S. : Didaktische Szenarien und deren Verhältnis zu Lernmaterialien, Forschungsberichte des Fachbereichs Elektrotechnik & Informationstechnik, ISSN 0945 0130, Fernuniversität Hagen, 2006 (www.fernunihagen.de/etit/fachbereich/forschung/index.html)www.fernunihagen.de/etit/fachbereich/forschung/index.html Huwendiek, S. et al.: Curricular integration principles of virtual patients: a focus group study among students, 2008. Manuscript in preparation. Bowen, J.L.: Educational Strategies to Promote Clinical Diagnostic Reasoning, N Engl J Med, 355: 2217- 25, 2006. Ark, T. K., Brooks, L. R., Eva, K. W.: The benefits of flexibility: the pedagogical value of instruction to adopt multifaceted diagnostic reasoning strategies, Medical Education, 41:3, 281-287, 2007. Gruppen, L.D., Frohna, A.Z.: Clinical Reasoning. In: Norman, Geoffrey R. International handbook of research in medical education. Kluwer Academic Publishers, Dordrecht 2002. Centre for Virtual Patients

16. Thank you for your attention! Centre for Virtual Patients

17. Study among 5 th year medical students Centre for Virtual Patients Work with VPs

18. Study among 5 th year medical students Centre for Virtual Patients Work with VPs Tutor moderated small group discussion

19. Introduction Campus-Lernprogramm, Zentrum für virtuelle Patienten (ZVP), Heidelberg (for further information see: www.medicase.de)www.medicase.de eVIP Electronic Virtual Patients, Co-funded by the European Union (for further information see: www.virtualpatients.eu)www.virtualpatients.eu Centre for Virtual Patients