1. Assalam alaikum and Good Morning

2. Clinical Examiners’ Reflection about a newly developed eOSCE System: An innovative experience at Alfaisal University
Dr. Nasir Ali Afsar
MBBS, M.Phil, Ph.D.
Department of Pharmacology, College of Medicine, Alfaisal University, Riyadh, Saudi Arabia.
Honourable Chair_, ladies & gentlemen, It’s a privilege to address you in beautiful Marrakesh.
My topic is ……... Where, after briefly touching upon the basic jargons, I will walk you through a short history of what we were doing and ultimately evolved cost-effectively and indigenously based on our needs.

3. Introduction
What is an Objective Structured Clinical Examinations (OSCEs)?
Examiners evaluate students performing clinical tasks, such as on simulated patients or mannequins.
Key method to assess learning of clinical skills.
17 Jun 2015
ICTHE 2015, Hivernage

4. Advantages of an OSCE An evaluation that is,
relevant
effective
in a realistic situation
Prevents patients’ risks as well as suffering due to a long examination session.
17 Jun 2015
ICTHE 2015, Hivernage

5. Limitations of an OSCE
A resource-intensive activity, where we need to arrange for,
Venue, examiners, standardized patients, mannequins, data entry, evaluation ± feedback, organization, running cost etc.
Providing timely and personal feedback to student may be difficult.
17 Jun 2015
ICTHE 2015, Hivernage

6. Global Problem, Local Perspective
College of Medicine at Alfaisal University Riyadh, Saudi Arabia.
Regular use of OSCEs in undergraduate and postgraduate clinical examinations.
Management of an OSCE:
Pre-OSCE preparation
Conduction issues: limited space
Evaluation issues: validity of evaluation, subjective bias
Feedback issues: large number of students
Alfaisal University is an IT-friendly institution where both students and faculty extensively use various IT services through intra- and internet.
At CoM AU, we conduct clinical skills courses integrated with various organ-system modules, starting from Year-2 Medicine Program. OSCE is regularly used to evaluate students during and/or at the end of the course. However, managing an OSCE for almost 1000 students is challenging. We faced issues at all stages, ie PCEF. With a small team dealing with clinical skills courses there was no choice but to consistently look for enhancing our efficiency in one way or the other. Interestingly, I was a “permanent” member of that team and went through all different stages of evolution and hence here to tell the complete story, because, all other members, both senior and junior to me kept changing !
17 Jun 2015
ICTHE 2015, Hivernage

7. Evolution of Process at AU
To begin with, we started with the traditional way of paper-pencil OSCE, where examiners would evaluate each student with the help of a single-page-checklist. The students would bring their name and ID stickers (also prepared by ourselves) to the exam venue. Here is shown a portion of a typical OSCE checklist. But the whole exercise or organization would be very tedious, especially compiling data and its analysis.
17 Jun 2015
ICTHE 2015, Hivernage

8. Evolution of Process at AU
What is next ?
What is next? You would be familiar with this type of answer sheets used by students in different MCQ exams. However, we used this approach indigenously for examiners instead to evaluate the students. The biggest potential advantage would be quicker data entry.
17 Jun 2015
ICTHE 2015, Hivernage

9. Evolution of Process at AU
Here, a portion of such a Scantron® -transformed checklist is shown. This too was developed indigenously because luckily we could not find anyone dealing with such customization. We used MS excel to design it and then printed them on a sheet of paper. It went well, but still required human interaction to run the sheets and analyze the data. Moreover, at one instance, which happened to be the last time we used it, a very small difference in paper size, so small that we could not appreciate it, led to scanning error and then all sheets which showed error had to be marked manually after the OSCE was over. Just imagine !
17 Jun 2015
ICTHE 2015, Hivernage

10. Evolution of Process at AU
From paper-pencil OSCE to Scantron® OSCE:
Resource-intense, error-prone, cumbersome
Finally, an electronic OSCE (eOSCE) was developed utilizing the updated web-based Electronic Learning System at Alfaisal University.
17 Jun 2015
ICTHE 2015, Hivernage

11. Methods
Development of eOSCE application: helped by Alfaisal IT Department.
Testing and improvement
Examiner Training
Post-eOSCE Examiner Survey (POES) conducted anonymously
17 Jun 2015
ICTHE 2015, Hivernage

12. Methods
eOSCE
administered in five OSCEs in three different undergraduate courses (Fall 2014, years 1-3).
About 550 medical students were evaluated (more than 950 time-limited evaluation instances).
Clinical history taking and/or physical examination.
Examiners accessed the online OSCE application using laptops provided to them.
17 Jun 2015
ICTHE 2015, Hivernage

13. Typical eOSCE Interface – browser based function
17 Jun 2015
ICTHE 2015, Hivernage

14. Typical eOSCE Interface – individual evaluation
17 Jun 2015
ICTHE 2015, Hivernage

15. Methods – POES All examiners included (n=22)
Excluded; incomplete response (n=3)
Included examiners (n=19)
Seniors: Experience
> 1 Year (n=10)
Juniors: Experience
≤ 1 Year (n=9)
17 Jun 2015
ICTHE 2015, Hivernage

16. Results * * Significant difference (p = 0.03) 17 Jun 2015
ICTHE 2015, Hivernage

17. Results
All examiners
Preferred eOSCE, mainly due to convenience and automation.
Found the system easy-to-learn
Wished to use the system again
Showed motivation to share their experience,
Junior examiners showed higher enthusiasm (p=0.03).
17 Jun 2015
ICTHE 2015, Hivernage

18. Results The managers of the eOSCE appreciated this program for
Ease of use
Cost effectiveness
Easy compilation of results
Feasibility of personalized feedback to students.
Dependence on technology was a perceived disadvantage.
17 Jun 2015
ICTHE 2015, Hivernage

19. Conclusions
The eOSCE was a convenient and cost- effective application.
The faculty showed a positive response towards its preferential use, especially the younger generation of examiners.
17 Jun 2015
ICTHE 2015, Hivernage

20. Our Clinical Skills Team
Dr. Nasir Ali Afsar, Department of Pharmacology.
Dr. Muhammad Zafar, Department of Anatomy and Clinical Skills.
Dr. Amna Siddiqui, Department of Anatomy.
SPECIAL THANKS TO
Dr. M. Faisal Ikram, Assessment Office, helped to develop Scantron® OSCE
Mr. Khyam Shahzad, IT department, helped to develop eOSCE application
17 Jun 2015
ICTHE 2015, Hivernage

21. For queries write to: nafsar@alfaisal.edu
THANK
YOU
17 Jun 2015
ICTHE 2015, Hivernage