1. STACEY T. GRAY, MD PROGRAM DIRECTOR, HARVARD MEDICAL SCHOOL

2.  STEVEN PLETCHER, MD  Program Director, UCSF  VYVY YOUNG, MD  Associate Program Director, University of Pittsburgh  IRAM AHMAD, MD  PGY-7 Resident, University of Iowa

4.  RESIDENT SURVEY  Questions about the clinical and educational experience in the training program  Questions about duty hours  FACULTY SURVEY  Questions about the experience working in the training program  Interactions with residents

5.  Administered annually for residents, fellows and core faculty  NOTIFICATION  Program is responsible for notifying and reminding trainees and faculty about the survey  TIMING  Mid-January thru early June  5 week window to respond

6.  REQUIRED RESPONSE RATE  Resident/Fellow Survey ▪ 70% completion rate  Faculty Survey ▪ 60% completion rate

8.  Duty Hours  Faculty  Evaluation Systems  Educational Content  Resources  Patient Safety/Teamwork  Overall Evaluation of Program

9.  Supervision and Teaching  Educational Content  Resources  Patient Safety  Teamwork  Overall Evaluation of Program

10.  Sample questions are not available  Major categories remain the same  Questions change each year – picked from a question bank  Not all survey takers will see the same questions

13.  ANNUAL PROGRAM REVIEW DATA ENTRY  ANNUAL ADS UPDATE ▪ Program Changes ▪ Program Attrition ▪ Scholarly Activity – faculty and residents ▪ Major changes ▪ Response to citations  CASE LOGS  RESIDENT SURVEY  FACULTY SURVEY  MILESTONES  BOARD PASS RATE

14. “The ACGME’s Resident/Fellow and Faculty Surveys are an additional method used to monitor graduate medical clinical education and provide early warning of potential non- compliance with ACGME accreditation standards”

15.  RESPONSE RATE  Resident survey – 70%  Faculty survey – 60%

16.  RESIDENT SURVEY  Duty hour violations  Negative responses compared to national ORL program results in 3 major categories  Negative responses compared to national ORL program results in 20% of the overall survey  Negative response to final summary question

17.  FACULTY SURVEY  Negative responses compared to national ORL program results in 2 major categories  Negative responses compared to national ORL program results in 20% of the overall survey

19. 2014-2015

20.  106 programs  Resident Response  1442/1506  96% response rate  Faculty Response  1406/1625  87% response rate

21.  Focus on areas of less than 95% compliance  Potential areas for improvement  National level – how are we doing as a specialty?  Program level – compare to national ORL standards

22. % Program Compliant % National Compliant 80 hours96%94% 1 day free in 799%97% In-house call every 3 rd night 100%99% Night float no more than 6 nights 100%99% 8 hours between duty periods 96%97% Continuous hours scheduled 98%96%

23. % Program Compliant % National Compliant 80 hours96%94% 1 day free in 799%97% In-house call every 3 rd night 100%99% Night float no more than 6 nights 100%99% 8 hours between duty periods 96%97% Continuous hours scheduled 98%96%

24.  Do you review duty hour rules with residents before the survey?

25. % Program Compliant % National Compliant Sufficient supervision98%92% Appropriate level of supervision 98%96% Sufficient instruction94%85% Faculty and staff interested in residency education 94%84% Faculty and staff create environment of inquiry 91%78%

26. % Program Compliant % National Compliant Sufficient supervision98%92% Appropriate level of supervision 98%96% Sufficient instruction94%85% Faculty and staff interested in residency education 94%84% Faculty and staff create environment of inquiry 91%78%

27. % Program Compliant % National Compliant Sufficient supervision98%92% Appropriate level of supervision 98%96% Sufficient instruction94%85% Faculty and staff interested in residency education 94%84% Faculty and staff create environment of inquiry 91%78%

28. % Program Compliant % National Compliant Sufficient time to supervise residents/fellows95%94% Residents/fellows seek supervisory guidance92% Interest of faculty and PD in education98%96% Rotation and educational assignment evaluation 99% Faculty performance evaluated99%98% Faculty satisfied with personal performance feedback 89%86%

29. % Program Compliant % National Compliant Sufficient time to supervise residents/fellows95%94% Residents/fellows seek supervisory guidance92% Interest of faculty and PD in education98%96% Rotation and educational assignment evaluation 99% Faculty performance evaluated99%98% Faculty satisfied with personal performance feedback 89%86%

30.  Resident Supervision vs. Independence  Different perception  Learning Environment Faculty Development  Mentorship  Teaching techniques  Faculty Feedback  How to provide it?

31. % Program Compliant % National Compliant Able to access evaluations99% Opportunity to evaluate faculty members 98%96% Satisfied that evaluations of faculty are confidential 92%84% Opportunity to evaluate program 99%98% Satisfied that evaluations of program are confidential 91%78% Satisfied that program uses evaluations to improve 88%73% Satisfied with feedback after assignments 84%70%

32. % Program Compliant % National Compliant Able to access evaluations99% Opportunity to evaluate faculty members 98%96% Satisfied that evaluations of faculty are confidential 92%84% Opportunity to evaluate program 99%98% Satisfied that evaluations of program are confidential 91%78% Satisfied that program uses evaluations to improve 88%73% Satisfied with feedback after assignments 84%70%

33.  Resident Feedback  Not just a semi-annual review  In the moment  Confidentiality  Reassurance for residents  Evaluations need to be meaningful  Issues identified by residents should be addressed  Improvements should be documented and reviewed with the residents

34. % Program Compliant % National Compliant Provided goals and objectives for assignments 96%95% Instructed how to manage fatigue97%92% Satisfied with opportunities for scholarly activities 92%76% Appropriate balance for education89%79% Education (not) compromised by service81%69% Supervisors delegate appropriately100%99% Provided data about practice habits73%68% See patients across variety of settings98%95%

35. % Program Compliant % National Compliant Provided goals and objectives for assignments 96%95% Instructed how to manage fatigue97%92% Satisfied with opportunities for scholarly activities 92%76% Appropriate balance for education89%79% Education (not) compromised by service81%69% Supervisors delegate appropriately100%99% Provided data about practice habits73%68% See patients across variety of settings98%95%

36. % Program Compliant % National Compliant Worked on scholarly project with residents/fellows 86%76% Residents/fellows see patients across a variety of settings 99%98% Residents/fellows receive education to manage fatigue 99% Effectiveness of graduating residents/fellows99%97% Outcome achievement of graduating residents/fellows 99%

37. % Program Compliant % National Compliant Worked on scholarly project with residents/fellows 86%76% Residents/fellows see patients across a variety of settings 99%98% Residents/fellows receive education to manage fatigue 99% Effectiveness of graduating residents/fellows99%97% Outcome achievement of graduating residents/fellows 99%

38.  Providing data about practice habits?  Evaluations?  Patient outcomes?

39.  Time for research and scholarship  How protected is the time?  How do you expose residents to all of the potential research projects and interested faculty?  Balance between education and service  How protected is the time?  How do you evaluate the quality of didactics?  How involved are the residents in the didactics?

40. % Program Compliant % National Compliant Access to reference materials100%99% Use EMR in hospital98%97% Use EMR in ambulatory setting100%97% EMR integrated across settings89%82% EMR effective96%94% Provided a way to transition care when fatigued 91%80% Satisfied with process to deal with problems and concerns 91%80% Education (not) compromised by other trainees 96%91% Residents can raise concerns without fear91%80%

41. % Program Compliant % National Compliant Access to reference materials100%99% Use EMR in hospital98%97% Use EMR in ambulatory setting100%97% EMR integrated across settings89%82% EMR effective96%94% Provided a way to transition care when fatigued 91%80% Satisfied with process to deal with problems and concerns 91%80% Education (not) compromised by other trainees 96%91% Residents can raise concerns without fear91%80%

42. % Program Compliant % National Compliant Program provides a way for residents/fellows to transition care when fatigued 100%99% Residents/fellows workload does not exceed capacity to do the work 98%99% Satisfied with faculty development to supervise and educate residents/fellows 96%95% Satisfied with process to deal with residents/fellows problems and concerns 95%93% Prevent excessive reliance on residents/fellows to provide clinical service 98%

43.  Fatigue Management  Care Transitions  Resident Concerns  Confidential reporting  Free of fear of intimidation or retaliation

44. % Program Compliant % National Compliant Tell patients of respective roles of faculty and residents 99% Culture reinforces patient safety responsibility 100%99% Participated in quality improvement88%86% Information not lost during shift changes or patient transfers 98%97% Work in interprofessional teams99%98% Effectively work in interprofessional teams100%99%

45. % Program Compliant % National Compliant Tell patients of respective roles of faculty and residents 99% Culture reinforces patient safety responsibility 100%99% Participated in quality improvement88%86% Information not lost during shift changes or patient transfers 98%97% Work in interprofessional teams99%98% Effectively work in interprofessional teams100%99%

46. % Program Compliant % National Compliant Tell patients of respective roles of faculty and residents 99% Culture reinforces patient safety responsibility 100%99% Participated in quality improvement88%86% Information not lost during shift changes or patient transfers 98%97% Work in interprofessional teams99%98% Effectively work in interprofessional teams100%99%

47. % Program Compliant % National Compliant Information not lost during shift changes or patient transfers 93%89% Tell patients of respective roles of faculty and residents/fellows 88%90% Culture reinforces patient safety responsibility 97%96% Residents/fellows participate in quality improvement or patient safety activities 92%90%

48. % Program Compliant % National Compliant Information not lost during shift changes or patient transfers 93%89% Tell patients of respective roles of faculty and residents/fellows 88%90% Culture reinforces patient safety responsibility 97%96% Residents/fellows participate in quality improvement or patient safety activities 92%90% f f

49. % Program Compliant % National Compliant Residents/fellows communicate effectively when transferring clinical care 99%98% Residents/fellows effectively work in interprofessional teams 100%99% Program effective in teaching teamwork skills 100%99%

50.  Patient Care Transitions  Quality Improvement  Interprofessional teams

51.  Which of the following best summarizes your opinion of your residency program?

53.  Residents and Faculty generally have a positive view of our training programs  High compliance with most major categories  New questions highlight ACGME concerns with training

55.  Remind the residents AND faculty before the survey  Duty hours  Confidentiality of your evaluation system  Availability/accessibility of Goals & Objectives  Resources & systems for communication/support ▪ Faculty mentors ▪ Chief or senior residents ▪ Institutional  Sleep deprivation education  Quality Improvement

56.  Assess what the program is doing well  Assess opportunities for improvement  Topics for national discussions on training  Gain insight into what the ACGME is looking for in training programs