S tructured O bjective C linical E xamination P ractical

What is OSCE ? series of stations with tasks series of stations with tasks planned planned marking form marking form examiner examiner patient : SP patient : SP organization > examination organization > examination

Why OSCE ? before OSCE (1975) before OSCE (1975) viva (oral), long case, short case viva (oral), long case, short case  valid ? know how NOT show how know how NOT show how  reliable ? different patients different patients different examiners different examiners

Why OSCE ? more valid more valid  show how more reliable more reliable  same task | patient  same examiner or  same structured marking sheet

Basic Structure

Basic Structure : Parallel A B CD

Basic Structure : Double

How to start ? blueprint of the whole OSCE blueprint of the whole OSCE design the station design the station design the mark sheet design the mark sheet

Blueprint reversed table of specification reversed table of specification

Station Design station time 4-15 min. station time 4-15 min. total time < 2 hrs total time < 2 hrs focus task focus task examiner used : Y | N, who? examiner used : Y | N, who? pilot pilot

Type of Stations static | written static | written practical : technique practical : technique clinical clinical

Marking Sheet Design checklist checklist rating scale rating scale score score

Checklist dichotomous : Yes | No dichotomous : Yes | No Pros Pros  high objectivity  high reliability  easy to feedback Cons Cons  only quantity check

Checklist How to improve? How to improve?  stem clear clear observable observable not too long not too long  overall not too long not too long

Rating Scale rating rating quality concern quality concern lower objectivity lower objectivity  lower reliability

Rating Scale How to improve? How to improve?  3-7 scale  more clarification of each scale  more raters  rater training common errors of rating scale common errors of rating scale

Rating Scale common errors common errors  leniency error  central tendency error  halo effect  logical error  proximity error  contrast error

Examiner station developer station developer non station developer non station developer  teacher  not teacher other staff other staff SP SP participation => reliability participation => reliability

Observation direct direct indirect indirect  one-way mirror  monitor  video

Getting Feedback: How? verbal verbal marked checklist & be the subject marked checklist & be the subject marked checklist & watch video marked checklist & watch video printed answer printed answer relevant papers relevant papers

Getting Feedback : When? during the exam during the exam  intra-station  in another station  stress?  NB: too much information! after the exam after the exam  end of all stations

Station & Marking learning by doing learning by doing 8 groups : A-E in each group 8 groups : A-E in each group 2 x 4 stations 2 x 4 stations 2(p) - 4(x) - 4(d) 2(p) - 4(x) - 4(d) signal : signal : materials & ID materials & ID task (flexible) task (flexible) lunch : 3rd floor lunch : 3rd floor

Setting an OSCE learning by doing | 8 groups learning by doing | 8 groups structured task | medical student V structured task | medical student V time time  7 min. test (without feedback)  5 min. test + 2 min. with feedback available tools : pls ask available tools : pls ask draft of test and marking sheet : ~ 4 p.m. draft of test and marking sheet : ~ 4 p.m. preparation a.m. preparation a.m.

Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

Holistic Method medical school’s faculty-wide pass mark medical school’s faculty-wide pass mark e.g. 60% e.g. 60%

Modified Angoff Method group of experts group of experts get the OSCE get the OSCE “think of a group of minimally acceptable candidates” “think of a group of minimally acceptable candidates” decide the probability decide the probability brief discussion and decide again brief discussion and decide again

Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  Modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

Borderline Method marking form : checklist + global rating marking form : checklist + global rating all categorized ‘borderline’ students all categorized ‘borderline’ students mean scores of ‘borderline’ group mean scores of ‘borderline’ group

Borderline Method Mean borderline score = ( ) / 3 = 74

Relative Method 1st method : Wijnen Method 1st method : Wijnen Method  Passing mark = mean -1.96SE 2nd method 2nd method  60% of the 95th percentile rank score

Minimal Passing Stations criterion-referenced criterion-referenced

Staff & OSCE : Like emotional comfort emotional comfort validly assess validly assess consistent consistent

Staff & OSCE : Dislike too compartmentalized too compartmentalized no opportunity to observe the complete patient evaluation of the student no opportunity to observe the complete patient evaluation of the student repetitive nature => boring repetitive nature => boring

Students & OSCE fairer than other methods fairer than other methods less stressful less stressful unsure whether the important aspects tested unsure whether the important aspects tested

Limitations of OSCE lengthy preparation lengthy preparation need more observational skill of the staff need more observational skill of the staff costly costly low inter-station correlation low inter-station correlation test security? test security?

What’s next? evaluation => learning evaluation => learning summative => formative summative => formative

Innovation senior student as SP and examiner in OSCE senior student as SP and examiner in OSCE study sheet listing Dx that might appear on the OSCEs study sheet listing Dx that might appear on the OSCEs add structured oral exam into OSCE add structured oral exam into OSCE GOSCE GOSCE

GOSCE : Group OSCE Pros Pros  economy  mutual teaching  mutual support  opportunity to examine social skill Cons Cons  lack of individual assessment  different participants do different tasks

Potential Use of GOSCE formative assessment formative assessment end-of-course assessment end-of-course assessment exploring interpersonal relationship exploring interpersonal relationship teaching method for short course teaching method for short course

Re-using OSCE stations across rotation in the same academic year across rotation in the same academic year  statistically OK from year to year from year to year  statistically not OK

Conclusion : OSCE What ? What ?  stations + tasks + checklist Why ? Why ?  more valid, more reliable How ? How ?  How to organize?  How to analyze?