1. Workplace-based Assessment (WBA) – Beyond the numbers
James Kwan
Associate Professor, Emergency Medicine and Medical Education

2. Acknowledgements A/Prof Lukas Kairitis Dr Karuna Keat
Professor Jenny Reath
Professor Wendy Hu
Members Year 3/5 Committee

3. Outline Competence vs. Performance Overview WBA at UWS
Assessor judgements on performance
Delivering Feedback

4. Definitions Competence Competence in medicine
“Ability to do something successfully”
(Oxford Dictionaries, 2012)
Competence in medicine
Ability to perform a specific clinical task
Requires integration of competencies
Measurable in terms of observable behaviour
Context specific
Process of growth and development

5. Miller’s pyramid Does Shows how Knows how Knows
Professional authenticity
Non- standardised assessment
(emerging)
Does
Standardised
Assessment
(mostly established)
Shows how
Knows how
Knows
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.

6. Goals of WBA
Make a judgement on a student’s performance of a specific task in the clinical setting
Provide guidance to the student through effective feedback to improve future performance of the task

7. Examples Encounter based Global
Case Discussion (Long case in the workplace)
Mini-CEX (direct observation clinical/procedural skill)
Global
Clinical Attachment Assessment
Multi-Source Feedback (360)
Shift Feedback Form

8. Year 3 Mini-CEX Student (or assessor) selects a patient
Student performs a focused clinical task e.g. history taking, physical examination, performing a procedural skill, counseling a patient
Assessor directly observes the encounter
Assessor rates the student’s performance in a range of domains and provides feedback
No longer than 30 min including feedback

9. Year 3 Mini-CEX

10. Challenges
Competence is developmental vs. normative (not satisfactory, borderline, satisfactory, good and excellent)
Assessors do not read the back page – use their own internal rubric
Reluctance to rate a student as “not satisfactory” – “Failure to fail”
Feedback is often limited and not as helpful as it could be
A “tick box” exercise resulting in students chasing a number

11. Crossley, J. , Johnson, G. , Booth, J. , & Wade, W. (2011)
Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ, 45(6),

12. Construct aligned scales
Judgement on extent to which a supervisor would trust a trainee (or student) to perform a task independently
Descriptors reflect the need for close or more distant supervision
Different contexts will have a different way of construing “independence”
Greater reliability with fewer assessments
3 vs. 6 Mini-CEXs required to achieve a Generalisability coefficient 0.7

13. Domains of assessment

14. Construct aligned Mini-CEX

15. Providing effective feedback

16. Effective Feedback Characteristics Perceived need for change Technique
Creating an action plan
Coaching

17. Delivering Feedback Enable learner reaction Encourage self assessment
Provide an assessment of strengths and weaknesses
Develop an action plan
Document the feedback

18. Year 5 Emergency Medicine
Shift Feedback Form

19. Summary Competence is developmental
Construct aligned scales may help align both assessor judgements on performance and the development of competence by the student
Delivering effective feedback is an important part of the assessment process to guide student learning
Action plan and coaching

20. Questions?