Dr. Mehdi Sayyah 1 Assessment of Competence An Introduction

Dr. Mehdi Sayyah 2 Miller - Van der Vleuten learning assessment pyramid Tests of Knowledge: Clinical Based Tests: Does Shows How Knows How Knows Competence Assessment: Performance Assessment: Real patients, Videos, Logs, Undercover SPs Observations, OSCEs, SPs PMP, Practicals, Essays, Orals MCQ, CRQ, EMI, SAQ

Dr. Mehdi Sayyah 3 Competence vs. Performance Competency-based assessments measures what students/doctors can do in controlled representations of professional practice Performance-based assessments measures what students/doctors do in actual professional practice

Dr. Mehdi Sayyah 4 Mechanisms of Thought Skill-BasedRule-BasedKnowledge-Based ConsciousUnconscious Rasmussen

Dr. Mehdi Sayyah 5 Skill-Based Skill-based cognitive processing and performance refers to actions that are automatic and easy due to an acquired skill. They usually happen quickly and without express effort on the part of the actor. These are unconscious actions that we don't need to explicitly "think about" in order to accomplish. Any decisions are usually automatic as well

Dr. Mehdi Sayyah 6 Rules-based Rules-based processing involves matching the context and problem currently facing the actor. These rules are typically of the "if X then Y" form, and can be based on past experience, explicit instructions, and so forth.

Dr. Mehdi Sayyah 7 Rules-based For example, if you want to treat hypocalcaemia, you normally give the patient calcium gluconate (an automatic skill). If the hypocalcaemia does not respond, however, you start to go down your list of reasons why it didn't works. –Think about hypomagnesaemia, –Think about hypoalbominaemia, …

Dr. Mehdi Sayyah 8 Knowledge-based If rules-based processing doesn't solve the problem, we fall back on knowledge-based processing (we tend to prefer rules-based solutions since they require less cognitive effort on our part).

Dr. Mehdi Sayyah 9 Knowledge-based This is what happens when we are truly faced with novel or unfamiliar situations, or where low-level rules aren't appropriate (e.g. making strategic decisions, or establishing a medical diagnosis). In general, this kind of processing involves the processing of symbolic information. –What about a refractory hypocalcaemia and hepatomegaly

Dr. Mehdi Sayyah 10 Knowledge-based As with rule-based processing, knowledge-based processing is a conscious process. It refers to what we typically think of as "analytic thought" the process and analysis of personal subjective knowledge.

Dr. Mehdi Sayyah 11 Competence and capability in complex adaptive systems Degree of Certainty Degree of Agreement High Low Environment Unfamiliar Environment Familiar Task FamiliarTask Unfamiliar Zone of Complexity Capability Competence