1. Lunelle L Pienaar & Gregory Doyle Education Development Unit, Faculty of Health Sciences, University of Cape Town

2. o Multiple choice format remain one method of assessment that is most commonly used to test theoretical knowledge in medical education (Palmer EJ and Devitt PG. 2007, Mavis et al, 2001). o Increasingly computer based assessment is used more within the FHS UCT-student numbers, drive increase IT use

3. o R-type (OBA) Traditionally used more often and is written with question in the form of a clinical case scenario with 4 or 5 option, questions are thought to test recall of knowledge. o A-type (EMI) The questions are written with a theme and questions are constructed around the focus theme (Beullens et al, 2002, Palmer and Devitt, 2007).

4. o EMIs is considered a better format to test problem solving and integration of knowledge( Swanson et al, 2008, Swanson et al, 2006). o EMI has demonstrated better discrimination between students as well as more difficulty compared to OBA (McCoubrie 2004).

5. o What is the performance of in house test administered to medical students during one of their clinical rotations in 5 th year. o What is the test item difficulty and discrimination of EMIs and OBA used in summative assessment. o Is there any differences between item difficulty and discrimination between the EMI and OBA.

6. o The questions were administered to all 5 th year students at the end of their clinical block. o We evaluated OBA items and extended matching items that were developed by the 5 th year course lecturer and evaluated as appropriate for testing by the course convener(not the researcher). o The item difficulty and discrimination for the EMIs and OBA was determined

7. o Results were from the end of block test with 5 groups of students. The same questions were administered to students all 5 groups of students. o The 193 students answered a computer based test containing five EMI sets with 34 items and 16 (OBA)-Type MCQs. o The OBA -had 4 options with each of the 5 EMIs sets had 9 options with 9 questions.

10. Item difficultyItem Discrimination MeanStd DevMeanStd Dev OBA (N=16)0.710.220.240.11 EMI (N=34)0.790.190.230.12

11. o The study did not show any statistical significance between the 2 types of MCQs – o small sample of test items. o Possible reason for this may also be that writing OBA format were used more often in the past. o Both EMI and OBA test items wide range of item difficulties o The results raises 2 issues o when questions are reused to select the best performing items. o evaluation of test post administration is important to determine how to improve quality of the assessment items.

12. o Beullens J, Van Damme B, Jaspaert H, Janssen PJ. 2002. Are extended-matching multiple-choice items appropriate for a final test in medical education? Medical Teacher 24 (4):390-5. o Mavis BE, Cole BL, Hoppe RB. 2001 A survey of student assessment in U.S. medical schools: the balance of breadth versus fidelity. Teaching and Learning in Medicine, 13 (2), 74-79. o McCourbrie P. 2004. improving the fairness of multiple- choice questions: a literature review. Medical Teacher, vol 26. 8 (709-712) o Palmer EJ and Devitt PG. 2007. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ; 7: 49. o Swanson, D.B., Holtzman, K.Z., Allbee, K. 2008. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to number and source of options. Journal of the Association of American Colleges, 83(10 Suppl):S21-4 o Swanson, D.B., Holtzman, K.Z., Allbee, K., Clauser, B.E. 2006. Psychometric characteristics and response times for content-parallel extended-matching and one-best-answer items in relation to number of options. Journal of the Association of American Colleges. Abstract 81(10 Suppl):S52-5

13. http://edusum.edublogs.org/category/thought s-on-assessment/