1. Students’ Assessment and Evaluation Prof. Namir G. Al-Tawil M.B.Ch.B., F.I.C.M.S./CM Hawler Medical University College of Medicine namiraltawil@yahoo.com

2. Contents Objectives Definitions of assessment and evaluation Types of assessment – Formative assessment – Summative assessment Assessment of students’ knowledge Assessment of students’ performance

3. Objectives At the end of the lecture, the audience must be able to know: The differences between formative and summative assessment. How to assess students’ knowledge. How to assess students’ performance.

7. Definitions Assessment – It is the process of data gathering by instructors about their teaching and their students’ learning. – Pre-test, observations, examinations are forms of assessment. Evaluation – It is the decision making process based on the assessment data.

8. Types of assessments Diagnostic assessment Formative assessment Summative assessment

9. Diagnostic assessment Helps to identify students’ current knowledge, skills, and capabilities; and to clarify misconception before teaching takes place. Types: – Pre-tests – Self assessment – Interview

10. Formative assessment Provides feedback during a teaching process. It assesses student’s and teacher’s progress. These assessments are typically not graded. E.g: if after 2 weeks of the start of the course, you made a test (by clickers for e.g) and you discovered that the majority didn’t answer correctly, then you have to repeat this point, and to correct yourself in the next course

11. Types of formative assessment Observations during in-class activities. Homework exercises. Question and answer sessions (planned and spontaneous). Conferences between instructor and students. Student feedback about their instructors.

12. Summative assessment Takes place at the end of learning process. It assesses the final product. Grades are usually an outcome of it. – To decide whether the student is eligible for the next step or not.

13. Types of summative assessments A mid-term exam. A final exam. A final project. A paper.

14. Criteria of the assessment tool Validity Reliability Educational impact, on teaching and learning Acceptability to staff, students, and others Cost A variety of assessment methods are needed to test competency

15. “When the cook tastes the soup, that’s formative evaluation. When the guests taste the soup, that’s summative evaluation” Bob Stake 1991

16. Assessment of knowledge and its application The most common method for knowledge assessment is the written method. Several formats are available like MCQs and essays…. The content of the question is more important than the format. A variety of formats are preferable.

17. Questions formats MCQs (one best answer) Single choice questions are more preferred than the true/false (multiple response item). Anatomy of the item (question): – Stem Vignette (e.g case study) Lead-in question (what is the most likely Dx?) – Options Correct answers Distractors (wrong-response options)

18. Tips for designing MCQs Use Plausible Distractors Use a Question Format (rather than incomplete sentence format) Keep Option Lengths Similar Balance the Placement of the Correct Answer Be Grammatically Correct

19. Avoid Clues to the Correct Answer Avoid Negative Questions Use Only One Correct Option Give Clear Instructions Avoid the "All the Above" Option Avoid the "None of the Above" Option Don't Use MCQs When Other Item Types Are More Appropriate

20. MCQs (Extended matching items) Extended matching questions consist of lettered options followed by a list of numbered problems/questions. For each numbered problem/question select the one lettered option that most closely answers the question. You can use the lettered options once, more than once, or not at all.

21. Extended matching, example For each case below, choose the SINGLE most appropriate treatment option from the above list of options. Each option may be used once, more than once, or not at all. Options: a.Laser treatment as laparoscopy b.Danazol c.Oral contraceptives d.Mefenamic acid (Ponstan) e.Gonadotrophin releasing hormone agonist (Soladex) f.Total abdominal or vaginal hysterectomy g.No treatment h.Pelvic sympathectomy i.Laparotomy

22. 1. A 30 year old woman complains of painful periods and pain during intercourse. She is afebrile and has a firm, tender nodule into the pouch of Douglas. Small foci of endometriosis are found at the time of laparoscopy. Answer: A 2. A 28 year old woman is diagnosed with endometriosis. Her symptoms are not incapacitating. She would like to start a family. Answer: G 3. A 45 year old woman complains of intractable dysmenorrhoea and menorrhagia. She does not want any more children. Answer: F 4. A 40 year old woman is diagnosed with endometriosis. The pain is severe, and she refuses surgical treatment. She does not want any more children. Answer: B 5. A 20 year old woman presents with severe left lower abdominal pain, increasing abdominal girth. Painful periods, and menorrhagia. Ultrasound demonstrates a 20 cm left ovarian cyst. Answer: I

23. Key features questions They are short clinical cases or scenarios which are followed by questions aimed at key features or essential decisions of the case. The questions could be MCQs or open ended questions. More than one correct answer could be provided.

24. e.g. Key features questions Fatima, aged 70 years, has been seeing you intermittently over the past 2 years since her previous GP (a colleague) retired. She presents this morning complaining of pain in her left knee. She explains she has had the pain for 4-5 years but it is now getting worse. Pain is worse with exercising and is relieved at rest. She also has pain at night, with stiffness in the morning and after an afternoon lie down. She now finds it difficult to walk to her local supermarket and mosque. Always she was active in the past, she now finds she looks for reasons to avoid walking.

25. Question 1: (2 marks) What possible differential diagnosis would you consider with this limited information: 1.Osteoarthritis 2.Crystal induced inflammatory arthropathy such as Gout or pseudogout 3.Rheumatoid arthritis and other causes popliteal cyst(baker cyst) or septic arthritis

26. Question 2: (1 marks) What should you look for on physical examination? 1.body weight 2.tenderness, swelling, effusion 3.reduced range of movement, presence of crepitus, deformity, involve other joints.

27. Question 3: (1 marks) What investigation may be helpful? 1.full blood count, ESR, CRP 2.urate level, joint aspiration and microscopy if effusion is present 3.plan x-ray of knee, rheumatoid factor, Antinuclear antibody,

28. Short answers questions Open ended questions. Their answer is usually 1 or 2 words. Usually used when closed formats are not possible. Each teacher should correct the same question for all the students. So more reliable scores.

29. Essay questions Requires more time to answer. Less reliable scoring. Using correction scheme (checklist) can improve reliability.

30. Modified essay questions Is a special type of essay question that consists of a case followed by a series of questions that relate to the case and that must be answered in the sequence asked. A student answering the first question incorrectly is likely to answer the subsequent questions incorrectly too.

31. Assessment of performance Categorized into: – In vitro assessment (using simulation) – In vivo assessment (real condition) In both, the student make a skill or behavior and observed by the examiner.

32. Assessment of performance, cont Many methods like check-lists and rating scales can be used to report observation. – OSCE and OSPE – Direct observation of procedural skills (DOPS) – Peer assessment – Self-assessment

33. Methods for assessing performance Checklists – E.g OSCE Rating scales – Problem of subjectivity and low reliability – Use more than one rater (observer) OSCE Short cases to assess clinical competence

34. Long cases (to assess clinical competence) – No more used in many colleges – Of help for formative examinations – Modification done in many places to improve reliability 360 degrees evaluation – Of help in formative assessment – Multiple evaluators like supervisors, peers, students, administrative staff, and patients. – Time consuming and not practical.

35. Mini Clinical Evaluation Exercises (MINI-CEX) – Of help in formative assessment – They are relatively short observations (15-20 minutes) in which performance is recorded on a 4 point scale where 1 is unacceptable, 2 is below expectation, 3 is met expectations, and 4 is exceeded expectations. – Several competencies are evaluated: history taking, physical examination, clinical judgment, counseling, professionalism and other generic qualities

36. Portfolio – A portfolio is a collection of student work which provides evidence that learning has taken place. – Mainly for formative assessment.

37. Questions ?

38. Conclusion Various assessment methods that test a range of competencies are available for examiners The choice depends on purpose of examination, i.e. either summative, formative or both, in addition to other factors.

39. Thanks for listening