INTRODUCTION TO ASSESSMENT METHODS USED IN MEDICAL EDUCATION AND THEIR RATIONALE

Objectives Explain why assessment Distinguish formative and summative assessment Classify different assessment methods Describe characteristics of a good assessment

“ Assessment Drives Student Learning.” George E Miller

“Assessment drives learning in at least four ways: its content, its format, its timing and any subsequent feedback given to the examinee.” van der Vleuten, C. (1996) The Assessment of Professional Competence: Developments, Research and Practical Implications, Advances in Health Sciences Education, 1, pp. 41–67.

“The ‘law’ of educational cause and effect states that: for every evaluative action, there is an equal (or greater) (and sometimes opposite) educational reaction.” Schuwirth, L.W.T. (2001) General Concerns About Assessment. Web address:

Evolution of Medical Students Website by NUS students: “Assessment drives learning in the direction you wish.”

Assessment Feedback Learning Linking Learning, Assessment, and Feedback

We should assess what we teach and teach what we assess.

Assessment Fundamentals Why do we assess? What should we assess? When should we assess? How should we assess?

Why Do We Assess? Determine whether learning outcomes are met Support of students’ learning Certification and competency judgment Teaching program development and implementation Accountability Understanding the learning process

Assessment Serves Multiple Stakeholders Students Teachers Department, Faculty; University; Administrators Public; Governmental Agencies Stakeholders’ interest in assessment is not necessarily aligned.

Stakeholders’ Priorities Students Teacher Faculty, University Public, Government

Effective Education KnowledgeAttitudeSkills What Should We Assess?

Knowledge and Performance Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Professional authenticity Cognition Performance

Shows how Knows how Knows Does Factual tests Context-based tests Performance Assessment in vitro Performance Assessment in vivo What Should We Assess?

Concept of Mastery ‘All or none state’ – not really When Should We Assess? Date of Examination

Continuum of Performance –‘Learning Curve’

A examination that attempts to test students’ mastery at a given point of time is less preferable than one that tests the mastery over a span of time.

How Should We Assess? Utility of Assessment Instruments: –Validity –Reliability –Educational Impact –Cost –Acceptability Utility = validity x reliability x educational impact x cost effectiveness x acceptability

Validity Validity: Ability of the assessment instrument to test what it is supposed to test. Example: The course aims to determine whether the students are able to communicate effectively. What assessment instrument would you choose for the given purpose?

Content validity: ability of the assessment instrument to sample representative content of the course. Course contentAssessment

Reliability Reliability refers to the consistency of test scores and the concept of reliability is linked to specific types of consistency. –Over time –Between different examiners, –Different testing conditions Instruments for student assessment needs high reliability to ensure transparency and fairness

Ex 1Ex 2Ex 3Ex 4Ex 5 Q 1X Q 2X Q 3X Q 4X Q 5X Examiner Question

Ex 1Ex 2Ex 3Ex 4Ex 5 Q 1XXXXX Q 2 Q 3 Q 4 Q 5 Examiner Question

Ex 1Ex 2Ex 3Ex 4Ex 5 Q 1X Q 2X Q 3X Q 4X Q 5X Examiner Question

Educational impact

There is probably more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education. Boud, 1995

..… this would not be so bad if it were not for the fact that the effects of bad practice are far more potent than they are for any aspect of teaching. Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition, if they want to graduate) escape from the effects of poor assessment. Boud, 1995

Principle One: “Learning >> Assessment >> Feedback” in that order! Learning should drive assessment and feedback; not the other way round Teaching is easy bit; defining the purpose of assessment is critical Balance between “assessment of learning” and “assessment for learning”

Principles Two: Validity, Validity, Validity Validity is the single most important determinant in in assessment Our interest is in validity of inference; validity can not be compromised Balance between educational needs and institutional needs

Principle Three: Educational Impact Any assessment is anxiety provoking for the students and (staff) Assessment has potential positive and negative steering effects on learning and professional development “Curriculum instructs teacher what to teach; exam instructs students what to learn.” Donald Melnick, 1991

Assessment is a moral activity. What we choose to assess and how shows quite starkly what we value. Knight, 1995