1. Assessment Tools

3. Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions

4. Educational Triangle Needs / Objectives/Outcomes & Contents Strategies Assessment Curriculum

5. Assessment Formative v/s Summative

6. Formative vs Summative It is continues assessmentAt the end of a course/ program Feedback is an essential partUsually NO feedback session Usually NO gradsMarks OR grades are essential There is no pass-fail decisionThere should be a pass-fail decision

7. Assessment V/S Evaluation

8. Evaluation: is more focused on making a judgment or determination concerning the quality of a performance, work product or use of skills against a set of standards. Assessment is more focused on measuring a performance, work product, or skill in order to offer feedback to document strengths and growth and to provide directives for improving future performance. Assessments are nonjudgmental and are designed and intended to be helpful to produce improvement.

9. Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Cognition/ Knowledge Skills & Behaviour Assessment

10. Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Knowledge Competence Performance Action Assessment

11. One method can’t do it all –One measure is no measure –We need a mixture of methods to cover the entire pyramid –We can choose different assessment tools

12. Criteria for different assessment methods 1.Validity: 2.Reliability 3.Educational impact 4.Cost effectiveness 5.Acceptability/ Practicability

13. Criteria / Requirements 1.Validity:Does it measure what it is supposed to measure? 2.Reliability:Does it produce consistent results? 3.Practicability:Is it practical in term of times & resources.

14. Educational Impact Assessment may have facilitative or adversive effect on learning Emphasizes the important learning outcomes Provide feedback about the success of study methods

15. What Should Be Tested ? Important topics are weighted heavily The sample of items reflect the instructional goals and the process.

16. Assessment

17. Written Exam

18. Written Written tests can assess knowledge acquisition and reasoning ability, They cannot so easily measure skills

19. …all students from (all classes) are submitted to the same test…… Progress test

20. End of course assessment Multiple Choice Open ended questions Combination MCQ/OEQ Modified Essay Questions Computer assessment Assignments Construct a patient information folder Design a Poster Integration essay Prepare a forum discussion Write a research proposal

21. Clinical 1.Short cases 2.Long cases 3.OSCE 4.Simulated Clinics 5.Video consultation

22. Clinical (cont’) 6. Slides Interpretation 7. Lab Data Interpretation 8. Portfolios 9.SOE (simulated oral exam) 10.Oral

23. Computerized Case-based Testing Computer is used to increase effectiveness in: –Cost –Test administration –Scoring –Authenticity (e.g. by multimedia) –Written, practical, clinical,…….

25. “Long case" and “Short case" The basic requirements for reliability and validity have not always been achieved in traditional long & short cases

26. Observing candidates' interaction with patients

27. OSCE Objective structured clinical examination

29. Observing Candidates' Interaction with Patients

33. Oral

35. Work Place-Based Assessment The oral examination is traditionally an unstructured face to face session with the examiners Direct observation: Clinical work sampling measures: Mini-CEX, Mini-Clinical Evaluation Exercise DOPS, Directly Observed Procedural Skills Directly ObservedDirectly Observed P-MEX, The Professionalism Mini-Evaluation Exercise Global performance measures Multi-Source Feedback (MSF or 360) Aggregation and reflection measures Logbook Portfolio

36. observable behaviour behaviouroutside valuesnormsinsideattitude professional behaviour= Assessment of professional behavior

37. PEER EVALUATION The oral examination is traditionally an unstructured face to face session with the examiners Peer Assessment & Self Assessment

38.  Regular feedback of performance  For all the students  Helping students to grow as a professional  Starting from the beginning of the studies Assessment of professional behavior

39. Professional behaviour consists of three aspects Dealing with workDealing with othersDealing with oneself Assessment of professional behavior

40. Student gathers evidence about aspects of his competence of his competence Criteria to meet the different aspects of competence are described by faculty results of assessment (longtudinal or tailored) reports, video projects peer assessment staff assessment articles, presenting papers Student reflects on the different aspects of competence (written statements) Portfolio

41. Student proves his competence On several aspects described by faculty Role as expert results of assessment reports projects peer assessment staff assessment Student reflects on the different aspects of competence (written statements) Student discusses his competence and reflection on it with the mentor Mentor = coach Portfolio assessment

43. 1. Contineous Assessment 40% Assignment 2. End of the Course Assessment TOTAL a. MCQ- 10 20% b. Written Short Answer Questions 40% 100%

44. Conclusion

45. The oral examination is traditionally an unstructured face to face session with the examiners Comprehensive Assessment Different methods to be applied Standardized / structured Not to depend only on few methods

46. Knows Shows how Knows how Does Knows Knows how MCQ, SAQ, Short essy, MEQs….Simple recall questions

47. Knows Shows how Knows how Does Knows howShows how Simulations requiring behavioural performance OSCE, OSPE, Simulated Clinics,

48. Knows Shows how Knows how Does Shows how Does Assessing performance Work-based assessment: Clinical work-sampling, Mini-CEX, Portfolio, practice visits, case orals….

49. Knows Shows how Knows how Does Self-, peer, co-assessment, portfolio, multisource feedback, learning process evaluations……