1. Program Evaluation Committees (PEC), Annual Evaluations, and the 10 year Self Study and Visit in the Next Accreditation System
Cuc Mai MD FACP
August 2014

2. Disclaimers No part of my salary is supported by ACGME.
I am only a messenger and also a program director.
Information may change in the Next Best Accreditation System.

3. Goals and Objectives
Review the goals of the accreditation process in the Next Accreditation System for Maintenance of Accreditation
Define current description of the Annual Program Evaluation, the 10 year self study, and 10 year self study visit
Define the requirements for the Program Evaluation Committee
Describe some actions programs should consider for the 10 year self study

4. The Old Days… Internal Reviews Program Information Form
Duration between Site Visits based on Results of Accreditation Visit

5. Maintenance of Accreditation in the Next Accreditation System
Self-
Study
visit
Ongoing Improvement AE
Yr 0
Yr 1
Yr 3
Yr 4
Yr 5
Yr 6
Yr 7
Yr 8
Yr 9
Yr 10
Yr 2
Annual Program Evaluation (PR-V.C.)
Resident performance
Faculty development
Graduate performance
Program quality
Documented improvement plan
Programs must conduct an Annual Program Evaluation to examine various components, including resident performance, faculty development, program quality, and improvement plans, as a ramp-up to the Self-Study (conducted by the program) at Year nine and the Self-Study Visit (conducted by ACGME Field Staff members) at Year 10.
Focused assessments possible at any time based on data from AE.
Full site visit at any time if data from AE eggregious.
AE: Annual Program Evaluation

6. Aims of the Next Accreditation System
Enhance the ability of the peer-review system to prepare physicians for practice in the 21st Century
To accelerate the movement of the ACGME toward accreditation on the basis of educational outcomes
Reduce the burden associated with the current structure and process-based approach

7. 10 Self-Study and Self-Study Visit
Conducted by the program
Annual Program Evaluation
Review of program goals and improvement efforts
Self-Study Visit
Conducted by ACGME Field Staff members
There are essentially two parts to the Self-Study. The Self-Study itself, which is conducted by the program and should include both a process similar to a Plan-Do-Study-Act cycle, and the Annual Program Evaluation that is an outcome of the work conducted by the Program Evaluation Committee. The Self-Study is partly a preparation for the Self-Study visit at Year 10.
The Self-Study visit is conducted by the ACGME Field Staff members at the assigned tenth-year date and the site visit team will evaluate the program with regard to response to citations, faculty development activities, CQI, and review the future goals for improvement set by the program.

8. Annual Data Review Elements Policy 17.61 Review of Annual Data
Continuous Data Collection/Review
ADS Annual Update
Resident Survey
Faculty Survey
Milestone data
Certification examination performance
Case Log data/Clinical experience
Hospital accreditation data
Faculty member and resident scholarly activity and productivity
Other: ACGME complaints, Institutional Quality Metrics
There are a number of data elements collected/reviewed annually. A number of these are already being collected. Faculty Survey and Milestones are new. Collection of scholarly activity information has been changed from inclusion in faculty members' curriculum vitae (CVs) to a Scholarly Activity template in ADS that simplifies reporting. In the old accreditation system, scholarly activity was not collected annually.

9. The Next Accreditation System
Continuous
Observations
Promote
Innovation
Assess
Program
Improvement(s)
Identify Opportunities for Improvement
The NAS seeks to provide continuous observations and identify opportunities for programs to improve and ensure that programs in good standing are free to innovate with regard to Detail requirements.
Program
Makes
Improvement(s)

10. Next Accreditation System: Ten-Year Self-Study and Self-Study Visit
Ongoing Improvement AE
Yr 0
Yr 1
Yr 3
Yr 4
Yr 5
Yr 6
Yr 7
Yr 8
Yr 9
Yr 10
Yr 2
Programs must conduct an Annual Program Evaluation to examine various components, including resident performance, faculty development, program quality, and improvement plans, as a ramp-up to the Self-Study (conducted by the program) at Year nine and the Self-Study Visit (conducted by ACGME Field Staff members) at Year 10.
AE: Annual Program Evaluation

11. Objectives of the Ten Year Self-Study Visit
A comprehensive review of the program
Using the Annual Program Evaluation and 10 year self study
Information on how the program creates an effective learning and working environment
How this leads to desired educational outcomes

12. Objectives of the Ten Year Self Study Visit
Judge program success at Continuous Quality Improvement
Learn future goals of program
Will verify compliance with CORE requirements
Core and subspecialty programs will be reviewed together

14. Next Accreditation System: Ten-Year Self-Study and Self-Study Visit
Ongoing Improvement AE
Self-
Study
Yr 0
Yr 1
Yr 3
Yr 4
Yr 5
Yr 6
Yr 7
Yr 8
Yr 9
Yr 10
Yr 2
Programs must conduct an Annual Program Evaluation to examine various components, including resident performance, faculty development, program quality, and improvement plans, as a ramp-up to the Self-Study (conducted by the program) at Year nine and the Self-Study Visit (conducted by ACGME Field Staff members) at Year 10.
AE: Annual Program Evaluation

15. Objectives of the 10 year Self Study
Conducted one year before self study visit
Assess ongoing compliance and improvement in all dimensions, specifically on outcomes
Consider program aims and strengths/weakness/opportunities/threats (SWOT) of program
Uses annual program evaluations and ACGME data to review program improvement activities, successes achieved, and areas still in need of improvement

27. Next Accreditation System: Ten-Year Self-Study and Self-Study Visit
Ongoing Improvement AE
Yr 0
Yr 1
Yr 3
Yr 4
Yr 5
Yr 6
Yr 7
Yr 8
Yr 9
Yr 10
Yr 2
Annual Program Evaluation (PR-V.C.)
Resident performance
Faculty development
Graduate performance
Program quality
Documented improvement plan
Programs must conduct an Annual Program Evaluation to examine various components, including resident performance, faculty development, program quality, and improvement plans, as a ramp-up to the Self-Study (conducted by the program) at Year nine and the Self-Study Visit (conducted by ACGME Field Staff members) at Year 10.
AE: Annual Program Evaluation

28. Annual Program Evaluation
Annual but may be more frequent
Done by Program Evaluation Committee
Document Review of Program Outcomes from Annual Data
Document Review of Program Achievement of Aims, Program Strengths, Program Threats, and Program Opportunities
Develop Continuous Program Improvement Projects and Track/Document Follow-up

30. Maintenance of Accreditation in the Next Accreditation System
Self-
Study
visit
Ongoing Improvement AE
Yr 0
Yr 1
Yr 3
Yr 4
Yr 5
Yr 6
Yr 7
Yr 8
Yr 9
Yr 10
Yr 2
Annual Program Evaluation (PR-V.C.)
Resident performance
Faculty development
Graduate performance
Program quality
Documented improvement plan
Programs must conduct an Annual Program Evaluation to examine various components, including resident performance, faculty development, program quality, and improvement plans, as a ramp-up to the Self-Study (conducted by the program) at Year nine and the Self-Study Visit (conducted by ACGME Field Staff members) at Year 10.
Focused assessments possible at any time based on data from AE.
Full site visit at any time if data from AE eggregious.
AE: Annual Program Evaluation

31. Program Considerations for the 10 Year Self Study Visit
How are you and your subspecialty programs working together?

32. Program Considerations for the 10 year Self Study
Do you have defined program aims?
Have you ever analyzed your program with a SWOT analysis?

33. Program Considerations for the Annual Evaluation?
Do you have a Program Evaluation Committee and policy?
Are you reviewing all the Program Outcomes evaluated by the ACGME and looking for areas of program improvements?
Do you create, follow through on, and document program improvement projects?

34. The Next Accreditation System
Continuous
Observations
Promote
Innovation
Assess
Program
Improvement(s)
Identify Opportunities for Improvement
The NAS seeks to provide continuous observations and identify opportunities for programs to improve and ensure that programs in good standing are free to innovate with regard to Detail requirements.
Program
Makes
Improvement(s)

35. ACGME Webinars and Other Resources
ACGME webinars are available at:
CLER
Overview of Next Accreditation System
Milestones, Evaluation, CCCs
Specialty-Specific Webinars (Phase I)
Phase I Coordinator Webinars (surgical and non-surgical)
Specialty-specific Webinars (Phase II): Nov 2013-Dec 2013
Slide presentations for distribution to the GME community: NAS, CCC, Milestones, Annual Program Evaluation/PEC, Updates on Policy– December 2013
Upcoming
Specialty-specific Webinars (Phase II): Jan 2014-May 2014
Self-Study (what programs do)
Self-Study Visit (what ACGME site visitors do)