1. ARCOS 16th Annual Meeting
The ACGME Site Visit: Partners in the GME Community
ARCOS 16th Annual Meeting
March 15, 2019
Lee S. Segal, MD
ACGME Accreditation Field Representative

2. The ACGME Site Visit: Partners in the GME Community
I have no financial disclosures
I am employed by the ACGME
Former Program Director, Department of Orthopaedic Surgery and Rehabilitation, Penn State University College of Medicine

3. Program Director and Coordinator
ACGME Site Visit
Program Director and Coordinator
ACGME
Review
Committee

4. Department Of
Orthopaedic Surgery
ACGME
Site
Visitor

5. Learning Objectives
Describe the elements and objectives of different types of accreditation site visits.
List key attributes of ongoing readiness for a site visit, and key elements of site visit preparation.
Review the Common Program Requirements effective July 2019 as they relate to the Annual Program Evaluation
Prepare for the 10-Year Accreditation Site Visit

6. Types of Site Visits Application, Re-Application
Pre-Accreditation, Continued Pre-Accreditation (AOA  ACGME)
Initial Accreditation
Continued Accreditation
10-Year Accreditation
Data-Prompted
Complaint-Prompted

7. Overall Goal of a Site Visit
Obtain complete, accurate, meaningful information about the program to assist the Review Committee (RC) in making an accreditation decision

8. Goals for Site Visits
Verify/clarify information provided by the program, by program's ADS site, and by Resident / Faculty Survey results
Address all relevant elements of the Program Requirements
Conduct educationally helpful, non-adversarial interview sessions
Inform program of RC review process and accreditation decision options
Make program improvement suggestions, not decisions or citations

9. Site Visit Preparation – PROGRAM RESPONSIBILITIES
Knowledge of Program Requirements (Common and Specialty-specific)
Provision of requested documents
Accurate completion of program information in ADS
Meaningful responses to all previous citations
Description of subsequent major changes to improve program
Update of faculty and resident rosters
Construct correctly-formatted block diagram
Review of all material by DIO

10. Site Visit Preparation– SITE VISITOR RESPONSIBILITIES
Site Visit Letter of Instruction
Research additional information in program’s ADS site:
e.g., program accreditation history, LONs, other RRC correspondence, ACGME Resident and Faculty Survey results, rosters, case lists, institutional accreditation status, core program status, faculty/resident attrition, program director turnover
Study all program-provided information
Plan interview sessions and discussion items for each

11. Resident / Fellow Interviews
Trainees may be interviewed in a single group, in 2 groups (seniors vs. juniors), or individually
Trainee interviews occur early in the site visit day, followed by other interviews to confirm the resident perspective

12. Two Sources of Conversation-starters
Resident and Faculty ACGME annual surveys
Confidential resident and faculty “consensus lists” of 5 strengths and 5 areas for improvements are requested before most site visits

13. Resident Survey Review Challenges
Small programs (< 4 )  multi-year aggregated results +/- helpful
Recent program graduates are not interviewed
PGY-1 Residents, fellows in 1-yr programs did not complete the RS available for review
Residents unwillingness to be open with site visitor may require reorganization of resident interviews on day of the site visit
RC thresholds of concern not known and RS items weighted differently
RS considered by some to be more than a screening tool
Results not PGY-stratified

14. Resident Survey Review Challenges
Corrected concern carry-over
Use as a retaliation tool
Expression of program, department, institutional unrest
Items/questions not understood by residents
Compare to national aggregate data, not specialty data
My understanding that RS will be changing

15. Resident Survey Review
 How has the program addressed the RS results?
Were RS results discussed
with trainees/faculty?
What RS-stimulated
improvements were made?

16. Document Review
Focused document review to reduce burden and increase meaning
Some documents (e.g., evaluations) will be reviewed in resident files or online

17. Internal Drivers for Program Improvement
Lauren M. Byrne, Rebecca S. Miller, and Thomas J. Nasca (2016) Implementing the Next Accreditation System: Results of the 2014–2015 Annual Data Review. Journal of Graduate Medical Education 2016; 8(1):
Internal Drivers for Program Improvement
Number of
Citations

18. Next Accreditation System (2013) Nasca TJ, et al: NEJM 2012: 366: 1051-1056
Former accreditation process prescriptive, not promote innovation
Programs were meeting, not raising educational standards
Transition from episodic SV (4-5 years) to annual data collection and 10 year SV, with a self-study done before scheduled 10 year SV.
Continuous process of quality improvement, educational outcomes based.

19. 10 year Site Visit / APE
Citations - Internal Drivers for Program Improvement
ACGME – External Driver for Program Improvement
Self-Study complements the Annual Program Evaluation

20. New Common Program Requirements July 1, 2019
IV.A.: The curriculum must contain the following educational components - a set of program aims consistent with the Sponsoring Institution’s mission, the needs of the community it serves, and the desired distinctive capabilities of its graduates V.C.1.b.: The PEC’s responsibilities must include a review of current operating environment to identify strengths, challenges, opportunities, and threats as related to the program’s mission and aims – SWOT analysis V.C.2.: The program must complete a Self-Study prior to its 10- Year Accreditation Site Visit

21. Program Aims
What does the program want their program graduates to look like?
Align with the community of patients for which the program provides care
Aims can change over time
Intentionality – “mission statement”

22. Context / SWOT Analysis/APE
Current operating environment (context)
Strengths, challenges, opportunities, and threats
Devise strategic plan - interventions
Area(s) for improvement
Advance strength, eliminate weakness, take advantage of an opportunity, mitigate a threat

23. Annual Program Evaluation
Interventions address:
Program requirements
What will take us to the “next level”
Long-term projects
Use PDSA approach (Plan, Do, Study, Act)
What is program trying to change, what interventions, how to measure outcomes
Improvement model – onus on each program to raise educational standard.

24. APE template

25. Self Study Where have we been? Where are we now? Where are we going?
Notification of Self-Study, Submit SS Summary, Self-Study Visit usually two years later … Intentionality:
Where have we been?
Where are we now?
Where are we going?

26. Self Study Have 5+ years of APE’s to incorporate
Self Study Group ≠ PEC (increase stakeholders, residents)
Strategic plan (Aims, SWOT, Interventions/strategies, action plans)
Opportunity to set direction and priorities, communicate mission/aims, alignment of program. Simplify decision-making.

27. The 10-Year Site Visit
A full site visit with review of all applicable program requirements Submit Summary of Achievements prior to scheduled 10-year SV 2-4 areas where improvements have been achieved Review both the Self-Study Summary and the program improvements described in the Summary of Achievements

28. Site visitor provides feedback to program leadership
Aspects of the Self-Study may be inserted into Site Visitor’s Report (SVR)
Site visitor provides feedback to program leadership
How well your Aims fit your program
Achievement areas link to aims and operating environment (SWOT)
Use of data to measure outcomes
Stakeholder engagement

29. ACGME Review Committee
Reviews to SVR, ADS documents, Surveys, SSS, Summary of Achievement
Makes accreditation status decision
May commend the program
Issues any citations
May request progress report
May note areas for concern / improvement
Sends Letter of Notification (LON)

30. Questions?

31. Thank You