1. Authentic Performance Assessment Allison D.Spinelli,D.O. Assistant Professor of Anesthesiology Residency Program Director

2. Authentic Performance Assessment Background What is competency How to assess competency Examples of APA model Competency rating scale Pros/cons Looking ahead

3. Background Why change evaluation system Standardizes faculty approach to evaluating residents Helps to define resident’s competency level for a particular year/rotation Better defines competency

4. What is Competency ‘Know it when I see it”

5. How to Assess Competency Authentic Performance Assessment Model

6. Real-time assessment of resident performance during actual patient care Clearly defines the skills/knowledge the resident must possess to be deemed competent at a task Specific to resident year of training Specific to rotations

7. Examples of APA Model CAY 1 Mentorship period Occurs 2 weeks after start of 1st year Resident should be able to set-up room,perform machine check,start iv,intubate uncomplicated airway

8. Examples of APA Model Four month APA for CAY1 resident All of previous skills/knowledge plus: Induction of GA

9. Examples of APA Model 8 month APA for CAY1 resident All of previous skills/knowledge plus Maintenance of GA and intra-op differential dx of vital sign abnormalities

10. Examples of APA Model 12 month APA for CAY1 resident All of previous knowledge/skills as well as emergence of patient from GA and post-op care

11. Examples of APA Model CAY 2 resident on cardiac anesthesia rotation Pre-op,line placements,induction,airway Manage prebypass period Manage postbypass period Manage transfer of care to CSICU Diagnose and treat postop complications

12. Examples of APA Model CAY 3 residents on any rotation and for the most acute and complex patients Pre-op assessment and anesthetic plan Line placement Induction and airway management Intra-op management of data Emergene and transfer of care Post-op complications

13. Competency Rating Scale Rubric system Highly competent Competent Becoming competent Not competent Gauges the degree of resident’s ability to perform independently vs. need for attending intervention

14. Examples of Competency Rating Scale Highly competent The resident is smooth and efficient The resident adapts knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately The resident does not require attending intervention

15. Examples of Competency Rating Scales Competent The resident lacks smoothness and efficiency but adapts knowledge,judgement,and skill to start the case and induce anesthesia with no significant errors and most appropriately The resident does not require attending interevention

16. Examples of Competency Rating Scales Becoming competent The resident lacks smoothness and efficiency and some knowledge,judgement or skill and exhibits some limits in adaptability. The resident makes some signifiant errors but is able to start the case although not most appropriately May require attending intervention

17. Examples of Competency Rating Scales Not competent The resident can only start the case and induce anesthesia with extensive coaching.The resident requires attending intervention

18. APA Model Logistics Scheduled to occur at the 4,8,and 12 month intervals for the CAY1 and CAY3 residents Scheduled to occur at the end of each subspecialty month for theCAY2 residents Requires 3 different cases and faculty members

19. APA Model Goal is for all residents to be competent or above Becoming competent can be acceptable at start of new rotation but need to see improvement otherwise,remediate Not competent=remediation/probation/ Daily evaluations til competent or no credit

20. APA Model Pros Helps to simply define competency Residents know what is expected of them and what goals are Helps faculty gauge competence in a uniform way Easy form to fill out Helps faculty give timely and constructive feedback Works well with one-to-one system

21. APA Model Cons Can be difficult to match patient /case complexity to resident year Faculty/resident cooperation to assure timeliness of assessments Residents free to pick and choose attendings who evaluate them Residents(and faculty)wait until they have a perfect case to assess Creates paperwork

22. Future of APA Model Randomize assessments Assessment teams Expand concept to use for resident evaluation of faculty teaching