1. S tructured O bjective C linical E xamination P ractical

2. What is OSCE ? series of stations with tasks series of stations with tasks planned planned marking form marking form examiner examiner patient : SP patient : SP organization > examination organization > examination

3. Why OSCE ? before OSCE (1975) before OSCE (1975) viva (oral), long case, short case viva (oral), long case, short case  valid ? know how NOT show how know how NOT show how  reliable ? different patients different patients different examiners different examiners

4. Why OSCE ? more valid more valid  show how more reliable more reliable  same task | patient  same examiner or  same structured marking sheet

5. Basic Structure 12348765

6. 12348765 12435687

7. Basic Structure : Parallel 12348765 A B CD

8. Basic Structure : Double 2 123 7654

9. How to start ? blueprint of the whole OSCE blueprint of the whole OSCE design the station design the station design the mark sheet design the mark sheet

10. Blueprint reversed table of specification reversed table of specification

11. Station Design station time 4-15 min. station time 4-15 min. total time < 2 hrs total time < 2 hrs focus task focus task examiner used : Y | N, who? examiner used : Y | N, who? pilot pilot

12. Type of Stations static | written static | written practical : technique practical : technique clinical clinical

13. Marking Sheet Design checklist checklist rating scale rating scale score score

14. Checklist dichotomous : Yes | No dichotomous : Yes | No Pros Pros  high objectivity  high reliability  easy to feedback Cons Cons  only quantity check

15. Checklist How to improve? How to improve?  stem clear clear observable observable not too long not too long  overall not too long not too long

16. Rating Scale rating rating quality concern quality concern lower objectivity lower objectivity  lower reliability

17. Rating Scale How to improve? How to improve?  3-7 scale  more clarification of each scale  more raters  rater training common errors of rating scale common errors of rating scale

18. Rating Scale common errors common errors  leniency error  central tendency error  halo effect  logical error  proximity error  contrast error

19. Examiner station developer station developer non station developer non station developer  teacher  not teacher other staff other staff SP SP participation => reliability participation => reliability

20. Observation direct direct indirect indirect  one-way mirror  monitor  video

21. Getting Feedback: How? verbal verbal marked checklist & be the subject marked checklist & be the subject marked checklist & watch video marked checklist & watch video printed answer printed answer relevant papers relevant papers

22. Getting Feedback : When? during the exam during the exam  intra-station  in another station  stress?  NB: too much information! after the exam after the exam  end of all stations

23. Station & Marking learning by doing learning by doing 8 groups : A-E in each group 8 groups : A-E in each group 2 x 4 stations 2 x 4 stations 2(p) - 4(x) - 4(d) 2(p) - 4(x) - 4(d) signal : - - ---- signal : - - ---- materials & ID materials & ID task (flexible) task (flexible) lunch : 3rd floor lunch : 3rd floor

24. Setting an OSCE learning by doing | 8 groups learning by doing | 8 groups structured task | medical student V structured task | medical student V time time  7 min. test (without feedback)  5 min. test + 2 min. with feedback available tools : pls ask available tools : pls ask draft of test and marking sheet : ~ 4 p.m. draft of test and marking sheet : ~ 4 p.m. preparation 8 - 9 a.m. preparation 8 - 9 a.m.

25. Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

26. Holistic Method medical school’s faculty-wide pass mark medical school’s faculty-wide pass mark e.g. 60% e.g. 60%

27. Modified Angoff Method group of experts group of experts get the OSCE get the OSCE “think of a group of minimally acceptable candidates” “think of a group of minimally acceptable candidates” decide the probability decide the probability brief discussion and decide again brief discussion and decide again

28. Minimal Passing Score criterion-referenced ( อิงเกณฑ์ ) criterion-referenced ( อิงเกณฑ์ )  holistic  Modified Angoff norm-referenced ( อิงกลุ่ม ) norm-referenced ( อิงกลุ่ม )  borderline method  relative method

29. Borderline Method marking form : checklist + global rating marking form : checklist + global rating all categorized ‘borderline’ students all categorized ‘borderline’ students mean scores of ‘borderline’ group mean scores of ‘borderline’ group

30. Borderline Method Mean borderline score = (72+70+80) / 3 = 74

31. Relative Method 1st method : Wijnen Method 1st method : Wijnen Method  Passing mark = mean -1.96SE 2nd method 2nd method  60% of the 95th percentile rank score

32. Minimal Passing Stations criterion-referenced criterion-referenced

33. Staff & OSCE : Like emotional comfort emotional comfort validly assess validly assess consistent consistent

34. Staff & OSCE : Dislike too compartmentalized too compartmentalized no opportunity to observe the complete patient evaluation of the student no opportunity to observe the complete patient evaluation of the student repetitive nature => boring repetitive nature => boring

35. Students & OSCE fairer than other methods fairer than other methods less stressful less stressful unsure whether the important aspects tested unsure whether the important aspects tested

36. Limitations of OSCE lengthy preparation lengthy preparation need more observational skill of the staff need more observational skill of the staff costly costly low inter-station correlation low inter-station correlation test security? test security?

37. What’s next? evaluation => learning evaluation => learning summative => formative summative => formative

38. Innovation senior student as SP and examiner in OSCE senior student as SP and examiner in OSCE study sheet listing Dx that might appear on the OSCEs study sheet listing Dx that might appear on the OSCEs add structured oral exam into OSCE add structured oral exam into OSCE GOSCE GOSCE

39. GOSCE : Group OSCE Pros Pros  economy  mutual teaching  mutual support  opportunity to examine social skill Cons Cons  lack of individual assessment  different participants do different tasks

40. Potential Use of GOSCE formative assessment formative assessment end-of-course assessment end-of-course assessment exploring interpersonal relationship exploring interpersonal relationship teaching method for short course teaching method for short course

41. Re-using OSCE stations across rotation in the same academic year across rotation in the same academic year  statistically OK from year to year from year to year  statistically not OK

42. Conclusion : OSCE What ? What ?  stations + tasks + checklist Why ? Why ?  more valid, more reliable How ? How ?  How to organize?  How to analyze?