1. Overview of Student Assessment and its Guidelines in the College
Nadia Al Wardy
Chairperson, College Examinations Committee
Assessment orientation
5/2/2019

2. Standards of “Sound Assessment”
Continuous assessment
Evidence of linkage to course objectives
Diversity of assessment methods
Assess different levels of achievement
Criterion referenced
Marking accurate and consistent
Feedback meaningful and helpful
Should enhance student learning
Students’ awareness of criteria
Proportional to the workload of the course

5. The assessment process
Specifying the purpose
Determining what is to be measured and how
Measuring it
Setting standards of expected performance
Reporting results
Making a judgement

6. Specifying the Purpose
Formative
Summative
Diagnosis
Promotion
Certification

7. Determining what is to be measured and how
Blueprinting
A matrix of:
What to assess?
knowledge, skills, attitudes
How to assess?
Different assessment tools
MCQ
EMI
MEQ
SAQ
OSPE/OSCE
Mini-CEX
CbD
OLC ….

8. Guidelines
Vetting: For all exams, questions / stations / cases etc. should be reviewed and emended by the relevant committees before the examination takes place
Relevance (Scenarios)
Types (at least 2 different modalities)

9. IDEAL Office International assessment bank
Over questions available

10. Formats of assessment in Phases I, II and III
Domain/Outcome
Phase I
Phase II
Phase III
Clinical Skills
OSCE
Mini-CEX, CbD, OLC, OSCE, Seminars, WRP, BSP, SC, OSPE
Health Promotion & Disease Prevention
MCQ, Reports
Seminars, OSCE, OLC, CbD
Communication Skills
TBL
Present
Seminars, OSCE, OLC, CbD, MCEX
Medical Informatics
CBA
MCQ, CBA
MCQ, CBW
Basic, Social & Clinical Sciences
MCQ, Spotter
Lab rep, SAQ
MCQ, SAQ, CBP
Quiz, problems, Spotter, EMI, OBE, OSPE, MEQ, Viva
MCQ, EMI, MEQ, KFQ, SAQ
Attitudes, Ethics & Legal Responsibility
Assignment, Project,
Proposal, Thesis, SAQ
MCQ, Mini-CEX, CbD, OLC
Personal & Professional Development
TBL, OBE
Assignment, Project
Poster, Thesis, TBL, Present
Portfolio
Miller’s Pyramid
Knows
Knows How
Shows How
Does

11. Principles of assessment (Reliability)
Sampling
No. of assessments
(min of 3 components)
No. and quality of questions
(at least 30 MCQs or equivalent/credit hour)
No. and quality of examiners
Weighting of components
(no single component to exceed 60% of total marks)
Time per question
( min/MCQ)

12. Example of assessment in courses of Phase I, II, and BMS
Course/Module
No. of credits
No. of components
Weighting
Format
Intro to Anatomy 4
20%, 20%, 20%, 40%
MCQ, Spotter, online
Locomotor system 3
30%,10%, 60%
MCQ, OSPE
Integrated Modules
30%, 20%, 50%
MEQ, OSCE
Molecular Diagnostics
20%, 20%, 60%
MCQ, practical

13. Assessment in Phase III
Stage
Block
Continuous (50%)
Final (50%)
Stage I
Pre-Clerkship
3 Mini-CEX
MCQ and EMI
Junior Clerkship:
Med/CH & allied block
Surg & allied block
BM/Fam block
Mini-CEX, OLC, KFQs (40%)
Portfolio (seminars, ward round presentations, bedside procedures, case write-ups) (10%)
80 MCQ & EMI,
20 SAQ (30%)
8 OSCE (20%)
Stage II
Senior Clerkship:
Med/Famco & allied block,
Surg & allied block,
Women & Child block
Mini-CEX, OLC, (CbD), KFQs (40%)
Seminars, Ward round presentations, Bedside Procedures, Case write-ups (10%)
60 MCQ & EMI,
4 OSCE (20%)
Elective
Selective
Pre-Internship
FMD
120 MCQ (20%)
120 SAQ (20%)
15 OSCE (20%)
8 Short Cases (40%)

14. Stage 1 3 block exams Stage 2 3 block exams MED/CH SURG BM/Fam MED/Fam
Women & Child

15. Junior Clerkship A B C D E F Week 1 2 3 4 5 6 7 8 Med x CH Surgery
Ophthal BM
FAMCO 2 3
Mini-CEX 4
OLC
Orthopedics 5 6
Dermatol 7
Genetics
Anesthesia
Mini CEX
SAQ
KFQ 8
Lab
EBM
MEQ
Forensic
MCQ
Geriatric
Oral Health

16. Marking MCQ and EMI: MEQ, SAQ and Essays:
if not done on-line, then optical marking.
MEQ, SAQ and Essays:
model answers
double marking is encouraged
OSCE/OSPE, Mini-CEX, OLC, Presentations, Reports…
checklists and rating scales
several examiners should be available, but one examiner should mark the same question for all students for consistency.

17. Standard setting method Fixed percentages (eg 60%)
Angoff
Hofstee
Borderline group
Fixed percentages (eg 60%)
For all the absolute methods here we first have to define the borderline group of students. Those students who have a 50% chance of passing.
Discuss
Or a group that is minimally competent
Take first item judge difficulty and importance of the item.
Estimate the percentage of borderline candidates that that will respond correctly ie get it right.
Fix your estimate
Proceed to next item
Take each estimate and average across the examination to obtain the cut point

18. Grade descriptions Grade Description A
Exceptional performance; all course objectives achieved; objectives met in a consistently outstanding manner. B
Very good performance; significantly more than the majority of the course objectives achieved (majority being at least two-thirds); objectives met in a consistently thorough manner. C
Satisfactory performance: at least the majority of course objectives achieved; objectives met satisfactorily. D
Minimally acceptable performance: less than the majority but more than the minimum required course objectives achieved; objectives met at a minimally acceptable level. F
Unacceptable performance: minimum required course objectives not met; objectives not met at a minimally acceptable level; no credit earned. FW
Failure to meet the attendance requirement, no credits earned
FSP
Failure with Supplemental Privilege

19. Programme for conversion of scores to grades

20. Item analysis and test statistics
IDEAL Office
Determination of difficulty, discrimination and reliability indices
Results of these analyses should be discussed by the relevant committees to assist in making informed decisions

21. Reporting results What? To whom?
Grade sheets filled online to be submitted to the Examination Office before and after confirmation of grades (timeline)
Separate spreadsheet showing scores of all ICAs and final accompanied by course report form
Grade report form
To whom?
Examinations Committee through Examinations Office

22. Feedback ICAs: Finals not discussed within 1 week timetabled
should not involve release of questions but a discussion of points of weaknesses
Finals not discussed

23. Management College Board College Examinations Committee
Phase II Exam Subcommittee
Phase III Exam Subcommittee
Written group
OSCE group
SC group
IP and BMS Exam Subcommittee

24. Management College Board College Examinations Committee
Phase II Exam Subcommittee
Phase III Exam Subcommittee
Written group
OSCE group
SC group
IP and BMS Exam Subcommittee
Chairman
Chairman (CC)
Assistant Deans (Pre-clinical and Clinical)
Phase Coordinators
Senior staff members
Examinations Officer

25. Management College Board College Examinations Committee
Phase II Exam Subcommittee
Phase III Exam Subcommittee
Written group
OSCE group
SC group
IP and BMS Exam Subcommittee
PC, JC and SC Block coordinators
Chairs/Reps Dept Exam Committees
Course/module coordinators

26. Examinations Office Examination timetables
Examination booklets, optical mark sheets, etc
Administrative forms related to examinations
Copy, store (electronically), and ensure security of all examination question papers
Assignment of invigilators
Prepare, print, and distribute information relating to assessments
Coordinating the visit of External Examiners
Collection of grades
Liaising with A&R in matters related to assessment
…..