Kazakhstan Health Technology Transfer and Institutional Reform Project The Objective Structured Clinical Examination

Kazakhstan Health Technology Transfer and Institutional Reform Project Objectives To become familiar with OSCE format and scoring To explore standard setting techniques To provide an overview of quality assurance methods To identify strengths and weaknesses of the OSCE To facilitate good design of OSCE stations

Kazakhstan Health Technology Transfer and Institutional Reform Project What is an OSCE? Number of stations through which students rotate Demonstration of a clinical skill or respond on paper to a series of related questions Wide variety of skills tested: History, physical examination, interpretation of tests, communication and interpersonal skills

Kazakhstan Health Technology Transfer and Institutional Reform Project Advantages Standardization Can test a range of skills Can pre set standards Flexible Does not rely on real patients

Kazakhstan Health Technology Transfer and Institutional Reform Project Disadvantages Can seem artificial Takes a lot of planning Expensive Needs well trained patients…and examiners!

Kazakhstan Health Technology Transfer and Institutional Reform Project How to run an OSCE Decide what you want to assess Identify the Team Decide the content Planning the site Patients Examiners Candidates Support staff The day before The day The day after Final follow up

Kazakhstan Health Technology Transfer and Institutional Reform Project Is it formative or summative?

Kazakhstan Health Technology Transfer and Institutional Reform Project Decide what you want to assess Communication skills History taking Examination technique Ethical issues Patient education Emergencies Technical skills

Kazakhstan Health Technology Transfer and Institutional Reform Project Identify the team Discussion

Kazakhstan Health Technology Transfer and Institutional Reform Project Decide station content Sort or long stations Mixture of the two Post encounter tasks eg read an x ray, interpret data Rest stations may be added

Kazakhstan Health Technology Transfer and Institutional Reform Project Couplet Stations Consists of patient encounters and post encounter probe Provides opportunity to test data interpretation in context Allows for an increase in test capacity without additional costs

Kazakhstan Health Technology Transfer and Institutional Reform Project You MUST talk to the standardised patient coordinator early You need to know the age and sex of available patients in order to plan the station content Define a time line for recruitment, training and testing roles

Kazakhstan Health Technology Transfer and Institutional Reform Project Test the station You MUST test the station to see if it works…try to get someone else to play the role of the student

Kazakhstan Health Technology Transfer and Institutional Reform Project For each station you will need Clear instructions to the candidate A very detailed patient role Prompts for the patient A scoring checklist A list of equipment needed in the room

Kazakhstan Health Technology Transfer and Institutional Reform Project Instructions to the candidate Who are they? Where are they? Eg You are a medical student working in the emergency room What are they to do? Do you want to give them any other information eg “The patient has diabetes”

Kazakhstan Health Technology Transfer and Institutional Reform Project How to run an OSCE Decide what you want to assess Identify the Team Decide the content Planning the site Patients Examiners Candidates Support staff The day before The day The day after Final follow up

Kazakhstan Health Technology Transfer and Institutional Reform Project Planning the site How many students? How many stations? How many tracks can you run? How long is each encounter? This will depend on your local site

Kazakhstan Health Technology Transfer and Institutional Reform Project You need – An area for registration and administration – Orientation of students, patients, examiners – Refreshment area – Area to keep students after their examination – Book the space early, a few months before!

Kazakhstan Health Technology Transfer and Institutional Reform Project Draw a map of the site Assign stations, considering the flow of students

Kazakhstan Health Technology Transfer and Institutional Reform Project Equipment lists Blood pressure cuff Reflex hammer etc Pencil for examiner Check list Water etc Clearly define who is responsible for these…or they WILL be forgotten!!

Kazakhstan Health Technology Transfer and Institutional Reform Project How to run an OSCE Decide what you want to assess Identify the Team Decide the content Planning the site Patients Examiners Candidates Support staff The day before The day The day after Final follow up

Kazakhstan Health Technology Transfer and Institutional Reform Project Examiners Find your examiners early – Faculty – Nurses? – Any other options? At least one per station and a small number of spares Send a letter explaining when they should arrive etc Record their answers and send reminders You may not have time to offer much training.

Kazakhstan Health Technology Transfer and Institutional Reform Project Instructions to examiners They should meet together, and with the patient, and run through the checklist Is it all clear to them? Rules for interacting with the student Rules for completing the list What to do with completed checklists

Kazakhstan Health Technology Transfer and Institutional Reform Project Candidates Need to know day of OSCE and time of arrival What can they bring? Eg stethoscope What can they NOT bring? Eg Phone Where do they register?

Kazakhstan Health Technology Transfer and Institutional Reform Project Orientation What signal will they hear? – To enter station – To leave station What is the time interval between stations Will there be an additional question whilst in the station?

Kazakhstan Health Technology Transfer and Institutional Reform Project Identification Best to give the a large sticker with their name or number on it Examiner MUST check it corresponds to the scoresheet they have in front of them

Kazakhstan Health Technology Transfer and Institutional Reform Project How to run an OSCE Decide what you want to assess Identify the Team Decide the content Planning the site Patients Examiners Candidates Support staff The day before The day The day after Final follow up

Kazakhstan Health Technology Transfer and Institutional Reform Project Support staff Registration Time keeper Escorts (standing in corridor etc) Clerical staff….responsible for signs, catering etc

Kazakhstan Health Technology Transfer and Institutional Reform Project The time keeper

Kazakhstan Health Technology Transfer and Institutional Reform Project The Day before Post signs…outside each room and also inside by the door Check signal system. Find a spare whistle!! Make sure all the standardized patients have remembered to come, especially teenagers

Kazakhstan Health Technology Transfer and Institutional Reform Project On the Day Have a clear schedule for when everyone arrives and where they go Where will orientation take place and who will do it? Where will the dry runs take place? When does the OSCE start?

Kazakhstan Health Technology Transfer and Institutional Reform Project Remember breaks for examiners with small bladders When you make the master schedule you must build in breaks…. A bigger break allows you to catch up if things are running over

Kazakhstan Health Technology Transfer and Institutional Reform Project Things that go wrong Patients fail to turn up – rare Examiners fail to turn up – common Scoresheets turn out to be ambiguous or confusing – common Food does not arrive Bell cannot be heard onside rooms - common

Kazakhstan Health Technology Transfer and Institutional Reform Project How to run an OSCE Decide what you want to assess Identify the Team Decide the content Planning the site Patients Examiners Candidates Support staff The day before The day The day after Final follow up

Kazakhstan Health Technology Transfer and Institutional Reform Project The day after Evaluations from examiners and participants are useful Payments to patients etc Send thank you letters

Kazakhstan Health Technology Transfer and Institutional Reform Project Scoring There is a difference between the examiners scoring of the OSCE and the final interpretation of the score afterwards.

Kazakhstan Health Technology Transfer and Institutional Reform Project 1.Checklist for important items - Done or Not done 2.Global rating section 3.Section to identify borderline candidates 4.Comments box – eg hurt patient, incorrect technique

Kazakhstan Health Technology Transfer and Institutional Reform Project Will you have “red flag” items? How much will the global section be worth eg 25% How do you define a pass mark? How many stations can be failed?

Kazakhstan Health Technology Transfer and Institutional Reform Project Group work We will design an OSCE We need to run through everything so far discussed Then we will create some roles and checklists

Kazakhstan Health Technology Transfer and Institutional Reform Project In groups Decide content for an 8 station OSCE We will split up and develop roles and stations