MBBS, MPH, MCPS, MRCGP (UK), FRIPH (UK), FHAE (UK) OSCE PREPARATION GUIDELINES Ass. Prof. Dr. Abdul Sattar KHAN Family & Community Medicine Department College of Medicine King Faisal University

Think ! Short & Long case Exam 4/15/2015 ASK 2 Equality Easy & difficult patients Hawks & Doves examiners Cost Getting real patients Unfair exam: complains

OSCE is an examination method OR Itself a Test? 4/15/2015 ASK 3

Objectives At the end of the session, participants are expected to: Understand the importance of OSCE Aware about advantages & disadvantages Know the purpose of OSCE Understand reliability & validity of OSCE Learn how to prepare & apply OSCE 4/15/2015 ASK 4

OSCE is a method not a test OSCE ( Objective Structural Clinical Examination) 1 OSLER (Objective structural Long Examination Record) OSPE (Objective Structural Practical Examination) TOSCE (GOSCE) - Team (group) Objective Structural Clinical Examination OSBE ( Objective Structured Biostatistics Examination) Etc……… 4/15/2015 ASK 5 1 Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ. 1979; 13(1):

Advantages Test a student’s ability to integrate knowledge, clinical skills, and communication with the patient An assessment tool that is custom-fit to the goals of a specific education program Provide chance for individualized feedback 4/15/2015 ASK 6

Disadvantages  Costly  Time consuming  Case specific evaluation  Quarantining (‘corralling’)  Serious concern (‘yellow card’) reporting systems 4/15/2015 ASK 7

What is the purpose of the OSCE? Provide feedback on performance Evaluate basic clinical skill Measure minimal competency 4/15/2015 ASK 8

Example of some skills Communication and professionalism skills (e.g. breaking bad news) bad news) History taking skills (e.g. taking a history from a patient) presenting with acute chest pain) Physical examination skills (e.g. performing a respiratory system examination) examination) Clinical-reasoning skills (e.g. interpreting clinical data and then prescribing therapy on a drug chart) 4/15/2015 ASK 9

What makes an OSCE a good form of assessment?  Reliability  Validity  Educational impact  Cost efficiency  Acceptability 4/15/2015 ASK 10 Van der Vleuten CP. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996; 1(1):

4/15/2015 ASK 11 How to prepare OSCE ? 10 – steps approach

Step 1: Selection of OSCE committee Select some peoples to be involved in following responsibilities: 1.determine the content of the examination 2.development and implementation of OSCE 3.to address decisions related to reliability and validity 4/15/2015 ASK 12

Step 2: Selection of coordinator A coordinator is the catalyst that facilitates the smooth working of the committee and make liaison with all stakes holders for good arrangement of OSCE. 4/15/2015 ASK 13

Step 3: Lists of what need to be assessed The examination will measure objectively based on objectives of the course, the competencies in specific areas of;  behavior  techniques  attitudes  decision-making strategies etc. 4/15/2015 ASK 14

A Blueprint 4/15/2015 ASK 15 Core clinical problem / outcome Clinical skills Practical procedure Patient management Health promotion Attitude & ethics Wheezing (General Medicine) ✓ ✓✓ Tiredness (General Medicine) ✓✓ ✓ Deafness (ENT) ✓✓✓ Tremor (Neurology) ✓ Collapse (Emergency Medicine) ✓✓✓

How to develop case/scenario? 1.The purpose of the station 2.Candidate instructions 3.Scoring checklist 4.Standardized patient instructions 5.Instruction for station set-up 4/15/2015 ASK 16

1. The purpose of the station State the skill and domain to be tested E.g; Skill –Physical examination. Domain –Internal medicine /cardiology. 2. Candidate instructions Candidate instruction must be clear and concise. 4/15/2015 ASK 17

3. Scoring checklist The checklist should be complete and include the main components of the skill being assess. 4. Standardized patient instructions These instruction must be detailed enough to guarantee standardization patient playing the same role. 5. Instruction for station set-up List of all equipment required for the station 4/15/2015 ASK 18

Step 4: The examinees / students The examinee is the student, resident, or fellow in training or at the end of training of a prescribed course OR During course for practice – formative assessment 4/15/2015 ASK 19

Step 5 : The Examiners Physicians/Surgeons etc. Simulated Patients Real patients Video recording only Self-assessor (Student as examiner) 4/15/2015 ASK 20

Step 6: The Examination Site Special rooms Big room with partition Outpatient facility 4/15/2015 ASK 21

Step 7: Examinations Station The total number of stations vary based on a function of the number of skills and time managment, ranging from E.g., MRCGP exam has 14 stations 4/15/2015 ASK 22

Duration of station Times ranging from 4 to 15 minutes have been reported in different examinations (MRCGP – 10 minutes & USMLE – 15 minutes) and a five minute station probably most frequently chosen. This times depend to some extent on the competencies to be assessed in the examination. 4/15/2015 ASK 23

Couplet/Linked Station Some competencies may best be assessed by coupled or linked stations. Finding Interpretation Finding Interpretation Treatment or Management Treatment or Management 4/15/2015 ASK 24 Station No. 10 Station No. 11

Couplet/Linked Station May measure competencies at different levels Basic Level Intermediate Advance level 4/15/2015 ASK 25 Station No. 10 Station No. 11Station No. 12

Step 8: Observer Assessment Method Checklist Rating scale 4/15/2015 ASK 26

4/15/2015 ASK 27

The borderline regression (BLR) method 4/15/2015 ASK 28

Step 9: Selection of Standard patient A standardized patient is an individual with a health problem that is in a chronic but stable condition; Standardized or simulated patient usually used when properly trained for history &physical assessment. 4/15/2015 ASK 29

Step 10: Ensure Reliability & Validity The patient problem relevant and important to the curriculum The station assess skill that have been taught Content experts reviewed the station 4/15/2015 ASK 30

Factor leading to lower reliability Too few station or too little testing time Checklists or items that don’t discriminate (too easy OR too hard) Unreliable patient or inconsistent portraits by standard patient Examiners who score idiosyncratically Administrative problem (disorganized staff OR noisy room) 4/15/2015 ASK 31

Ready to run the show ! Before the exam: 1.Standard setting 2.Instructions to supervisors 3.Draw a map 4/15/2015 ASK 32

Station 1 Station 2 Station 3 Station 4 Station 5 Station 6 Station 7 Station 8 Station 9 Station 10 Station 11 Station 12 Example of 10 station OSCE accommodating 12 students 4/15/2015 ASK 33

Enter in to station 2 – minute: Ending of station (A buzzer warning sound) After 10 minutes: Exit station (A buzzer sound) Enter new station 8 min 1 – minute: Standing outside of station 4/15/2015 ASK 34 1 – minute: Standing outside of station An algorithm for time management during movement in examination day

On the day of the exam: 1.Arrive early 2.Check completeness 3.Note attendance 4.Brief candidates 4/15/2015 ASK 35 Ready to run the show !

After the exam: 1.Store resources 2.Re-arrange venues 3.Feedback 4.Present results to board/committee 5.Amend / bank stations 6.Analyze the result 4/15/2015 ASK 36

Some final tips ! Spare standardized patients and examiners Back-up equipment Staff available during the examination to maintain exam security The bells or buzzers can be heard from all location with closed door Prepare an extra station 4/15/2015 ASK 37

Take Home Message Attaining clinical competence is not a one-off event but a career long learning routine 4/15/2015 ASK 38 Epstein RM. Assessment in medical education. N Engl J Med. 2007; 356(4):