1. Novel Teaching Methods and Innovative Fellow Evaluation Tools Joanne Filicko-O'Hara, MD; Thomas Jefferson University

2. Disclosures for Joanne Filicko-O’Hara, MD Employment Kimmel Cancer Center Thomas Jefferson University Consultancy None Equity Ownership None Research Funding NIH Honoraria None Patents & Royalties None Speakers Bureau None Membership on Board of Directors/Advisory Committee None Other Investigational Drug Supplied by Millenium Pharmaceuticals on current trial Presentation includes a description of the following off-label use of a drug or medical device None In compliance with ACCME policy, ASH requires the following disclosures to the session audience: 52 nd ASH Annual Meeting ♦ Orlando, FL

3. Objectives Discuss ways to make evaluations of trainees meaningful in real time. Discuss ways to make faculty participation in the evaluation process and other fellowship activities meaningful.

4. The Evaluations Of Trainees By faculty on each rotation By faculty in continuity experience By faculty – Other? By peers By patients and families By other staff (RN, Social Work, Pharm D) By program director By others (students? residents?) By trainees Of rotations faculty total experience Of faculty Rotations Continuity clinics Other Of peers Of program Of other staff

5. The Evaluations Medical Knowledge (faculty, peers?) Patient Care (faculty, patients/ families, other staff, peers) Practice Based Learning and Improvement (faculty, other staff, peers) Systems Based Practice (faculty, patients/ families, other staff, peers) Interpersonal Skills (faculty, patients/ families, other staff, peers) Professionalism (faculty, patients/ families, other staff, peers)

6. The Evaluations Medical Knowledge (faculty, peers?) Patient Care (faculty, patients/ families, other staff, peers) Practice Based Learning and Improvement (faculty, other staff, peers) Systems Based Practice (faculty, patients/ families, other staff, peers) Interpersonal Skills (faculty, patients/ families, other staff, peers) Professionalism (faculty, patients/ families, other staff, peers)

7. The Evaluation Process Fellow provides patient care under supervision of Dr. Jones Dr. Jones provides ongoing feedback about fellow performance Dr. Jones provides a final evaluation of fellow performance at the end of rotation that is: “in sync” with ongoing feedback and reviewed with the fellow Evaluation of Fellow Performance During Rotation

8. The Evaluation Process Fellow provides patient care under supervision of Dr. Jones Dr. Jones provides ongoing feedback about fellow performance Dr. Jones provides a final evaluation of fellow performance at the end of rotation that is: “in sync” with ongoing feedback and reviewed with the fellow

9. Why isn’t it done “right”? Too busy No one wants to be critical

10. Why isn’t it done “right”? Too busy No one wants to be seen as critical On line evaluation systems are great for record keeping, but you can’t carry the system in your pocket

11. One solution Self assessments mirror the faculty end-of- rotation evaluation Fellows do self-assessment on paper version Fellows review with attending during the final few days of rotation and sign Fellows hand in signed version to program coordinator Faculty often keep a copy for reference when doing the online version

12. Why isn’t it done “right”? Too busy (forces the issue) No one wants to be seen as critical On line evaluation systems are great for record keeping, but you can’t carry the system in your pocket

13. The problem of too few evaluations It’s easy to match evaluations to individual evaluators One good/bad evaluation can skew data Many evaluators grade up Many evaluators don’t want to be the one who’s “not nice”

14. Hematology/ Medical Oncology Fellowship Faculty Quarterly Evaluation of Fellows During the last three months, I have worked with this fellow (check all that apply):  In the outpatient setting  In the inpatient setting  During night and weekend call  I have not worked with this fellow during the past three months. Scale: 1 – Severe Deficits, 2 – Marginal Competence, 3 – Expected Competence, 4 – Above Expected Competence, 5 - Outstanding

15. 12345 NA Arrives on time for office hours/ rounds Completes dictations in a timely fashion Demonstrates professionalism in dealing with patients; Communicates effectively with patients and families ; shows compassion and empathy Demonstrates professionalism in dealing with nurses/physician extenders; relates well with other health care professionals Responds to requests for consults in a timely fashion Writes an appropriate letter/ progress note – all necessary components, grammar/ spelling Remains focused on tasks at hand (not having constant interruptions/ phone calls, etc) Is competent at performing history and physical exam

16. 12345 NA Has an appropriate knowledge of basic science related to hematology/ oncology Shows appropriate clinical judgment in developing differential diagnosis and treatment plan Is sensitive to patient’s age, gender, culture and disabilities Is knowledgeable about indications/ contraindications of procedures Is skilled in performing procedures Adheres to professional ethical standards Teaches medical students and residents effectively Is willing to admit errors and accept feedback

17. 12345 NA Considers costs of diagnosis and management Assesses own strengths and weaknesses; Has good self directed learning skills Adapts to new technology Is committed to quality assurance and improvement Is effective at managing health care resources Is an effective team member Is an effective consultant

18. Fellow Quarterly Evaluation of Faculty 1. Discusses patient care plans w/me at the time of ADMISSION 2. Treats me with RESPECT as a colleague 3. TEACHES about oncologic or hematologic issues and directs me to appropriate reading 4. Involves me in patient care decisions 5. Interacts well with the other faculty – shows them respect and is collegial 6. Provides me with appropriate support/back-up 7. Responds to calls at night – in a reasonable time, is helpful and appreciative 8. Is accessible 9. Provides helpful input at conferences 10. discusses end-of-life issues with patients, families and me

19. Faculty Name NANeverRarelySome- times OftenUsuallyAlways 012345 Dr. Axelrod Dr. Besa Dr. Carabasi Dr. Carr Dr. Filicko-O’Hara Dr. Flomenberg Please rate the attending physicians on the following aspects of patient care and interaction with you: In each case, please feel free to provide comments as needed to explain answers or to provide examples showing why interacting with any individual faculty member is a good or bad experience. This is completely anonymous. …..discusses patient care plans with me at the time of admission

20. 12 fellows; 10 questions – 120 data points Twice each year – 240 data points Anonymous and untraceable Reviewed with chiefs Faculty evaluation of fellow – 23 questions; on line; more faculty  harder to enforce

21. Does it help? Data presented at department meeting Incentives for bonus Faculty need to score at the median or better Target participation in educational activities

22. Why isn’t it done “right”? Too busy No one wants to be seen as critical On line evaluation systems are great for record keeping, but you can’t carry the system in your pocket Incentives for faculty help get the job done

23. Other solutions? For discussion