Assessment of Clinical Competencies

Slides:



Advertisements
Similar presentations
Assessing Ethics in CbDs
Advertisements

NMRCGP Assessments on the e portfolio A summary for hospital clinical supervisors Maggie Eisner June 2009.
Workplace assessment Dr. Kieran Walsh, Editor, BMJ Learning. 2.
Assessment of Professionals M. Schürch. How do you assess performance? How do you currently assess the performance of your residents? What standards do.
A Model for Assessment ASSESSMENT TYPES. OBJECTIVE Assessment instruments vary considerably in their uses to test different levels of competence. 5/10/2015.
Workplace-based Assessment. Overview Types of assessment Assessment for learning Assessment of learning Purpose of WBA Benefits of WBA Miller’s Pyramid.
Workplace-based Assessment (WBA) – Beyond the numbers
Promoting Excellence in Family Medicine nMRCGP Workplace-based Assessment March 2007.
An overview of Assessment. Aim of the presentation Define and conceptualise assessment Consider the purposes of assessment Describe the key elements of.
. Workplace-Based Assessment: a true reflection of competency ? Trevor Gibbs.
An overview of Assessment. Aim of the presentation Define and conceptualise assessment Consider the purposes of assessment Describe the key elements of.
Constructing a test. Aims To consider issues of: Writing assessments Blueprinting.
Keele University School of Medicine Reliability of the L eicester C linical procedure A ssessment T ool (LCAT), a tool to support holistic generic assessment.
1 Tools of clinical assessment. 2 Presentation outline Introduction Introduction Our daily practice Our daily practice Types of assessment tools Types.
Clinical assessment in the Health Sciences David Taylor School of Medical Education University of Liverpool.
Assessment of Clinical Competence in Health Professionals Education
Workplace-based Assessment
EVALUATION OF DR.MOHAMMED AL NAAMI, FRCSC, FACS, M Ed. Using O bjective S tructured C linical E xamination (OSCE)
Training the OSCE Examiners
Assessment Tools. Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions.
Work based assessment Challenges and opportunities.
Bob Woodwards SAC Chair, Oral and Maxillofacial Surgery.
Mini-CEX Mini-clinical evaluation exercise لیلا پاداش
Assessment tool OSCE AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
R 3 P Colloquium American Board of Pediatrics Jan. 31 – Feb. 2, 2007 The Past, Present and Future Assessments of Clinical Competence A Canadian Perspective.
Student assessment AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Student assessment Assessment tools AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Developing measuring and maintaining competence in new nursing roles, skills and advanced practice Ann Close, Care Quality Commission May 26 th 2009.
Developing an Assessment System B. Joyce, PhD 2006.
Programme Objectives Analyze the main components of a competency-based qualification system (e.g., Singapore Workforce Skills) Analyze the process and.
Student assessment Assessment tools AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Assessment Tools.
Assessment tools MiniCEX, DOPS AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Assessing Learners The Teaching Center Department of Pediatrics UNC School of Medicine The Teaching Center.
Workplace based assessment for the nMRCGP. nMRCGP Integrated assessment package comprising:  Applied knowledge test (AKT)  Clinical skills assessment.
INTRODUCTION TO ASSESSMENT METHODS USED IN MEDICAL EDUCATION AND THEIR RATIONALE.
Reliability in assessment Cees van der Vleuten Maastricht University Certificate Course on Assessment 6 May 2015.
Dr. Shruti Mohanty, FAIMER 2008,GSMC, KIMS, Narketpally
Henry M. Sondheimer, MD Association of American Medical Colleges 7 August 2013 A Common Taxonomy of Competency Domains for the Health Professions and Competencies.
Maria Gabriela Castro MD Archana Kudrimoti MBBS MPH David Sacks PhD
Focus Questions What is assessment?
1. Mini-Clinical Evaluation Exercise (mini-CEX) 2.
FUTURE PRACTICE & RESEARCH
Curriculum Development for Outcome Based Learning
PORTFOLIO ASSESSMENT (Undergraduate Medical Education)
Workplace Based Assessments
“PRACTICE BASED ASSESSMENTS” An update for 2017/18
Clinical Evaluation Tools
Longitudinal Clerkship
Classroom Assessment A Practical Guide for Educators by Craig A
Fundamentals of a Vocational Assessment
Empathy in Medical Care Jessica Ogle (D
“You teach best what you most need to learn…..”
Clinical Assessment Dr. H
A Practical Approach to Evaluation in the Ambulatory Setting in the Era of the New ACGME General Competencies Eric S. Holmboe Stephen Huot Yale University.
Workplace Based Assessment
The New Approaches with Cognitive Abilities Assessment
Work Place Based Assessment
Assessment 101 Zubair Amin MD MHPE.
Competency and Performance: What Should We Measure and How
Competency and Performance: What Should We Measure and How
Take Home Implementation Tools for Safety Evaluation
nMRCGP Assessments on the e portfolio
Changes in Training and Assessment
General Competencies Committee September 21, 2007
Oral Exams & Interviews
Assessing Ethics in CbDs
Assessment David Taylor.
ASSESSMENTs DURING TRAINING.
Presentation transcript:

Assessment of Clinical Competencies Zubair Amin MD MHPE Associate Professor; Dept of Pediatrics Yong Loo Lin School of Medicine Senior Consultant; Dept of Neonatology National University Hospital

Overview of clinical assessment Basic psychometric principles on clinical assessment Bias Sampling Context specificity Basic educational principles about clinical assessment Educational impact Feedback Quality assurance General properties of innovative instruments

“ Assessment Drives Student Learning.” George E Miller 1919-1998

“Assessment drives learning in at least four ways: its content, its format, its timing and any subsequent feedback given to the examinee.” van der Vleuten, C. (1996) The Assessment of Professional Competence: Developments, Research and Practical Implications, Advances in Health Sciences Education, 1, pp. 41–67.

Knowledge and Performance Professional authenticity Does Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.

Examiner Question Ex 1 Ex 2 Ex 3 Ex 4 Ex 5 Q 1 X Q 2 Q 3 Q 4 Q 5

Examiner Question Ex 1 Ex 2 Ex 3 Ex 4 Ex 5 Q 1 X

Examiner Question Ex 1 Ex 2 Ex 3 Ex 4 Ex 5 Q 1 X Q 2 Q 3 Q 4 Q 5

One case: a patient with diabetes mellitus Scenario A One student One case: a patient with diabetes mellitus Examiner 1 Examiner 2 30 minutes for long case Decision: Pass Total Examiners’ Time: 60 minutes (30X2)

15 minutes for each long case Decision: Pass Scenario B Diabetes Mellitus Rheumatoid Arthritis Asthma Stroke Examiner A Examiner B Examiner C Examiner D 15 minutes for each long case Decision: Pass Total Examiners’ Time: 60 minutes (15X4)

You are the Chairperson of the examination committee You are the Chairperson of the examination committee. The particular candidate is known to be weak. In which situation (Scenario A or Scenario B) you are more comfortable in examiners’ decision to pass the candidate? What are the comparative advantages and disadvantages between Scenario A and Scenario B?

Context Specificity There is no generic problem solving skills Geoff Norman Management of Diabetes Ketoacidosis = Outpatient management of diabetes Management of COPD = Management of rheumatoid arthritis Diagnosis of asthma = Counseling of asthma patient Typical correlation of “skills” across problems is 0.1 – 0.3

“Assessment drives learning in the direction you wish.” “The ‘law’ of educational cause and effect states that: for every evaluative action, there is an equal (or greater) (and sometimes opposite) educational reaction.” Schuwirth, L.W.T. (2001) General Concerns About Assessment. Web address: www.fdg.unimaas.nl/educ/lambert/ubc “Assessment drives learning in the direction you wish.”

Unintended Consequences of Assessment Evolution of Medical Students Website by NUS students: http://medicus.tk Unintended Consequences of Assessment

Educational Impact and Feedback Any test is anxiety provoking for the students and (staff) Test has potential positive and negative steering effects on learning and professional development We tend to assess more, but provide considerably less feedback Provision for feedback should be strongly considered in performance-based assessment

Linking Learning and Assessment

‘All or none state’ – not really Concept of Mastery ‘All or none state’ – not really

Continuum of Clinical Competencies Student Trainee Doctor in Practice

A examination that attempts to test students’ competency at a given point of time is less preferable than one that tests the competency over a span of time

Some Common Features in Assessment of Clinical Competency Multiple examiners Multiple cases (patients, problems) Context free Over a span of time Feedback and improvement of learning

Traditional Clinical Examination One/two examiners evaluate a candidate Candidate takes one patient Tasks Take history Complete physical examination Interpret data Generate differential diagnosis Discuss the patient problem with examiners

Issues Single patient: context specificity Single examiner: bias Patient: mostly in-patients “Exam” patients Artificial Unlike most physician-patient encounters Limited or no opportunity for observation of communication Limited or no opportunity for feedback

Instruments for Clinical Competency Assessment OSCE Mini-CEX DOPS 360 degree evaluation Portfolio Many others

Mini-CEX Assessor observes a trainee in any setting Takes into account of contexts Complexity Focus of visit (diagnosis, management, follow-up) Uses standard form (not customized to particular patient/problem) Completes 6-10 encounters

Implementation Issues Generally reliable and reproducible At least 4-6 encounters Different assessors, different patient, different context Feedback

The mini-clinical evaluation exercise ( mini -CEX) form Norcini, J. J. et. al. Ann Intern Med 2003;138:476-481

Direct Observation of Procedural Skills Mini-CEX equivalent for procedural skills assessment Similar psychometric properties Observation of procedures with real patients Minimum of 4-6 procedures, different assessors Provision of feedback

3600 Assessment / Multi-Source Feedback Involves evaluation of the candidate by multiple individuals within the sphere of influence Self, superior, peers, co-workers (nurse, ward clerk) Assessment of professionalism

3600 Assessment Multiple observations by multiple observers Over a period of time Provide evidence, as opposed to impression Highly valued as a developmental tool

3600 Assessment Focus on behavior: action taken by the individual Not used for knowledge or skills Nurses often are better discriminators Difficult to standardized grades Unsatisfactory Satisfactory Superior Customize to your needs

Fundamental Principles Assessment should be driven by purpose Consider both psychometric and non-psychometric parameters of assessment tool Create a backbone of assessment with few high quality, practical tools Strive towards holistic profiling of a candidate