Workplace-based Assessment (WBA) – Beyond the numbers James Kwan Associate Professor, Emergency Medicine and Medical Education

Acknowledgements A/Prof Lukas Kairitis Dr Karuna Keat Professor Jenny Reath Professor Wendy Hu Members Year 3/5 Committee

Outline Competence vs. Performance Overview WBA at UWS Assessor judgements on performance Delivering Feedback

Definitions Competence Competence in medicine “Ability to do something successfully” (Oxford Dictionaries, 2012) Competence in medicine Ability to perform a specific clinical task Requires integration of competencies Measurable in terms of observable behaviour Context specific Process of growth and development

Miller’s pyramid Does Shows how Knows how Knows Professional authenticity Non- standardised assessment (emerging) Does Standardised Assessment (mostly established) Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.

Goals of WBA Make a judgement on a student’s performance of a specific task in the clinical setting Provide guidance to the student through effective feedback to improve future performance of the task

Examples Encounter based Global Case Discussion (Long case in the workplace) Mini-CEX (direct observation clinical/procedural skill) Global Clinical Attachment Assessment Multi-Source Feedback (360) Shift Feedback Form

Year 3 Mini-CEX Student (or assessor) selects a patient Student performs a focused clinical task e.g. history taking, physical examination, performing a procedural skill, counseling a patient Assessor directly observes the encounter Assessor rates the student’s performance in a range of domains and provides feedback No longer than 30 min including feedback

Year 3 Mini-CEX

Challenges Competence is developmental vs. normative (not satisfactory, borderline, satisfactory, good and excellent) Assessors do not read the back page – use their own internal rubric Reluctance to rate a student as “not satisfactory” – “Failure to fail” Feedback is often limited and not as helpful as it could be A “tick box” exercise resulting in students chasing a number

Crossley, J. , Johnson, G. , Booth, J. , & Wade, W. (2011) Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ, 45(6), 560-569.

Construct aligned scales Judgement on extent to which a supervisor would trust a trainee (or student) to perform a task independently Descriptors reflect the need for close or more distant supervision Different contexts will have a different way of construing “independence” Greater reliability with fewer assessments 3 vs. 6 Mini-CEXs required to achieve a Generalisability coefficient 0.7

Domains of assessment

Construct aligned Mini-CEX

Providing effective feedback

Effective Feedback Characteristics Perceived need for change Technique Creating an action plan Coaching

Delivering Feedback Enable learner reaction Encourage self assessment Provide an assessment of strengths and weaknesses Develop an action plan Document the feedback

Year 5 Emergency Medicine Shift Feedback Form

Summary Competence is developmental Construct aligned scales may help align both assessor judgements on performance and the development of competence by the student Delivering effective feedback is an important part of the assessment process to guide student learning Action plan and coaching

Questions? j.kwan@uws.edu.au