1. 8 th Annual Track 1.0 ART Program Meeting Maputo, Mozambique August 10, 2010

2. PEPFAR II On July 30, 2008, the United States Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act was signed into law Authorized $48 billion over the next 5 years to combat HIV/AIDs, tuberculosis, and malaria $39 billion (81%) were authorized for PEPFAR bilateral HIV/AIDS programs and U.S. contributions to multilateral partnerships To support treatment for at least 3 million people To support prevention of 12 million new infections To care for 12 million people by Sept. 20, 2014

3. Assuring the treatment goals of PEPFAR II USG agencies and funded partners must demonstrate strong stewardship of the USG funds and programs Reporting how funds are being used Ensuring the quality of services Accurately reporting the number of persons on ART To enhance the long-term sustainability of PEPFAR programs, there is a process to transition the administrative and fiscal responsibilities of USG-supported ART programs to the MOH and other local partners A strong supervision system for the ART program is fundamental for fiscal reporting, quality services, and accurate results reporting.

4. The current state of ART program supervision and monitoring Key areas of support Infrastructure improvement and maintenance Human resources Training Mentoring Laboratory Pharmacy Data and information systems Commodities availability Community linkages

5. The current state of ART program supervision and monitoring Partners expected to conduct visits to facilities and districts on a regular basis Assess overall functioning of HIV clinics Provide technical assistance Ensure quality of care Current state of supervision varies from country to country Implementing partners have worked with the MOH to develop and pilot supervisory programs Partners themselves are providing supervision and mentoring Supervisory visits may be rare due to insufficient staff, shortage of skills, authority, and transportation

6. The current state of quality management in ART programs In many countries, USG funds support quality management (QM) and performance improvement and ( PI) initiatives Some use an experienced QM partner to assist them with their QM/PI initiatives Implementing partners integrate QM with clinical guidelines and standards of care (SOC) An effective QM program continuously validates the reliability of performance measurement data, and uses this data to improve the quality of care services Supportive site supervision around quality may focus on one or more of the following: data quality assessment clinical consultative back-up, education, and mentoring facilitation of additional resources to improve care and services

7. The current state of ART monitoring and evaluation Implementation partners expected to collaborate with MOH staff to develop, implement, and maintain patient and program monitoring systems at their supported facilities Health facilities expected to follow MOH or National AIDS Control Program minimum standards for patient monitoring which include: a minimum standard data set a unique national HIV program ID an HIV care and treatment patient record pre-ART and/or ART registers (unique patients organized chronologically by month of enrollment/initiation) standard indicators Standard monthly and/or quarterly reports on cross-sectional and cohorts Minimum standards are not always met Some USG country teams fund external partners to provide M&E support to specific levels of the system Recent data quality assessments suggest that the quality of data collected in the ART monitoring systems needs improvement

8. Standards Standard 1: There is a regular schedule of joint (USG- MOH) supportive supervisory site visits. Standard 2: There is a reliable results monitoring system for oversight of partners and programs. Standard 3: All ART sites have a functional and sustainable patient monitoring system. Standard 4: There is a regular schedule and reporting of ART data quality assessment activities. Standard 5: All ART sites have QM/PI processes in place.

9. Standard 1 A regular schedule of joint (USG-MOH) supportive supervisory site visits Rationale: To provide adequate and informative oversight, the USG PEPFAR staff and the MOH should be familiar with the treatment programs at individual sites. Experience during the first 5 years of PEPFAR suggests that this is best achieved by a schedule of routine supportive supervisory site visits. Supportive supervision is a practical and collaborative approach to supervision that aims to improve the quality of clinical care, efficiency of services, and satisfaction of staff. It focuses on ongoing identification and resolution of problems through joint observations, team discussions, and direct site involvement and emphasizes dissemination of results and the use of data for decision-making.

10. Standard 1 A regular schedule of joint (USG-MOH) supportive supervisory site visits Ideally, the supportive supervision framework should comprise at least two supervisory levels in each country. District level team including the district MOH office and the USG partner; this team conducts routine and systematic visits to every site in their area in a predefined period. Central supervisory team including national MOH staff and a USG member will: monitor overall activities support the mid-level supervisors modify guidelines and policy as indicated by supervisory data and reports. accompany the district teams on visits to pre-selected sites on a routine basis (e.g., every 3 months a percentage of clinics will be visited) Site visits will be scheduled and standardized so that staff can plan for and fully participate in the visit. A national supervisory tool is required to support standardization and supervision.

11. Standard 1 Action Steps to Achieve Standard Action StepsPerformance MeasuresResponsibilityTimeline Establish a schedule of joint USG-MOH site visits so that each USG site is visited at least once during FY 2010-FY 2011. Detailed reliable schedule available for review Documentation that sites are aware of schedule Develop a National Supportive Supervision Tool that outlines a standardized approach to site visits. Document available either electronically or via hard copy Identify staff at the central, district, and partner levels that will be stable members of the supervisory team. Clearly delineate roles and responsibilities of each member with formal allocation of time for supervisory activities Supervisory team rosters available with responsibilities and relationships between members defined

12. Standard 1 Action Steps to Achieve Standard Action StepsPerformance MeasuresResponsibilityTimeline Conduct training on supportive supervision concepts for supervisory team with refresher courses as necessary Training scheduled, implemented, and attended by teams Establish annual budget to support supervisory team and their activities Budget includes money for supervisory activities, including team members’ time for visits, writing of reports, and analysis of reports at central level to inform program development Identify practical and logistical impediments (e.g., reliable transportation) to routine site visits and a plan to address them Plan available for review Identify technical assistance (internal or external to USG) that might be needed to actualize the schedule of site visits. Technical assistance needs documented

13. Standard 2 Results monitoring system to improve oversight of partners and programs Rationale: In the absence of fully functional MOH systems that can deliver needed information to the USG, the USG agencies and their funded partners must support systems to monitor the quantity and quality of services provided at supported sites.

14. Standard 3 All ART sites have a functional and sustainable patient monitoring system Rationale: The facility-based system should have the tools and staff needed to support patient care and to report on a timely basis on the required indicators - national, USG, and otherwise. Thus the system must include the following: Tools: Standard records, registers, reports, and databases are in use at site. Electronic databases are likely needed for sites with more than 500 active ART patients. Patient ID: Patients can be uniquely identified at the site (preferably across sites with unique national health or HIV patient ID if possible. Organization: Patient records are stored in a way that allows for systematic management, retrieval and update of records. Interoperability of Clinic, Lab, and Pharmacy Systems: Data within clinic, lab, and pharmacy systems can be exchanged and compared among those systems via common data elements, forms, processes, or electronic programs. Confidentiality/Security: Records, registers and databases are kept in secured space with restricted access. Staff: Sufficient and qualified staff to maintain patient monitor system. Reporting: Timely production and transmittal of all monthly and quarterly reports

15. Standard 4 The regular conduct and reporting of ART data quality assessment activities Rationale: Recent multi-country rapid data quality assessments showed that the quality of data collected in ART monitoring systems needs improvement. Substantial over- reporting of persons on ART was discovered at selected large USG-funded ART sites in several countries, due largely to the failure of systems to subtract from the “active” or current roster of patients those persons who had not had a visit or drug pick-up in the recent period. These findings reveal challenges to other indicators of ART program performance, including the number ever on ART and the number retained in ART programs.

16. Standard 4 Action Steps to Achieve Standard ACTION STEPSPERFORMANCE MEASURES RESPONSIBILITYTIMELINE Review, develop, and disseminate additional data quality assessment protocols and tools, as needed. Data quality assessment protocol and tools exist. Provide technical support (with external partner) to country teams, partners, and sites for thorough data quality assessments and data quality improvement plans. Technical support has been identified and is available to country team, partners, and sites for data quality improvement. Require all USG care and treatment partners to conduct specified data quality assessments on all directly supported sites in FY 2010-FY2011 and report back site-specific findings with action plans for addressing challenges of underperforming sites. Partners have conducted data quality assessments on all directly supported sites in FY2010-FY2011. Site specific findings with action plans have been reported.

17. Standard 4 Action Steps to Achieve Standard ACTION STEPSPERFORMANCE MEASURES RESPONSIBILITYTIMELINE Incorporate into contracts/cooperative agreements the expectation to incorporate standard operating procedures (SOPs) to check quality of data at every step in data flow AND to periodically conduct thorough data quality assessments on key indicators (e.g. current on ART and retention in ART program) and report the results of those assessments to the funding agency. Existing contracts/cooperative agreements include requirement for periodic data quality assessments with reporting of results to the funding agency.

18. Standard 5 All USG-funded partners and ART sites have formal QM/PI plans and reporting mechanisms Rationale: QM plans and PI activities promote the local review of the quality of data, the local use and understanding of data, and the collaboration of data and clinical staff on PI activities Each national ministry or department of health should have written standards of care (SOC) and a national QM/PI plan, which are disseminated throughout the system and demonstrable at the site level. The SOCs should be used to guide the development of specific quality indicators for each plan. The language of the national plan should be reflected in the plans of all USG-funded partners.

19. Standard 5 Action Steps to Achieve Standard ACTION STEPSPERFORMANCE MEASURES RESPONSIBILITYTIMELINE Identify and describe existing country standards of ART care and treatment Country SOCs exist and are available for review. Identify and describe existing QM/PI plans QM/PI plan exists and is available for review. Plan has been updated within 2 years. Engage an external partner(s) to work in collaboration with the MOH and other USG-funded partners to facilitate adoption and implementation of a national QM/PI plan, if needed An assessment has been conducted to determine the need for an external partner to work with the MOH and other partners to facilitate adoption and implementation of a national QM/PI plan and documentd. An external partner has been engaged, if needed.

20. Standard 5 Action Steps to Achieve Standard ACTION STEPSPERFORMANCE MEASURES RESPONSIBILITYTIMELINE Work with USG-funded partners to evaluate QM/PI plans throughout directly supported facilities The QM/PI plan contains the elements described above, that larger (above PHC level) sites have a QM/PI committee, that at least one staff member at each site has received QM/PI training, and that recent PI activities have been reviewed by higher level staff Incorporate the expectation of updated QM/PI plans into contracts and cooperative agreements Contract/cooperative agreements contain a quality section. These contain a requirement for quarterly and annual reports of QM/QI activities.

21. Standard 5 Action Steps to Achieve Standard ACTION STEPSPERFORMANCE MEASURES RESPONSIBILITYTIMELINE Request information in the quarterly and annual reports from the partner to the agency on the number of participating sites and current PI projects based upon national indicators At least one PI activity has been conducted and documented, based on national indicators, in each quarter of the past year, with evidence of an improved outcome. Request information in the quarterly and annual reports from the partner to the agency on current PI projects based upon a site-specific indicator At least one PI activity has been conducted, based upon a site- specific indicator, in the past 6 months, with documented evidence of an improved outcome

22. External Assessment of ART Program Supervisory Systems In order to support and sustain the provision of high quality clinical services and accurate program and fiscal reporting, USG headquarters proposes a regular schedule of site assessments to evaluate the progress of implementation and/or maintenance of the country-specific ART supervision plan. Each review would be conducted over the course of 2-3 weeks, potentially including staff from: the Ministry of Health USG country HIV/AIDS Adult Care and Treatment and M&E Teams USG headquarters (CDC, USAID, DOD, etc.) Adult Treatment and Monitoring and Evaluation (M&E) Technical Work Groups implementing partner (e.g., Columbia, EGPAF) US headquarters offices implementing partner (e.g., Columbia, EGPAF) country or regional offices

23. External Assessment of ART Program Supervisory Systems Review Process: Each country visit would begin with a preliminary planning meeting of the review team in the country capital. The review team would then be divided into several smaller site review teams, which would be deployed to the sites. In general, each review team would review one to two sites per day, depending upon the size of the site and distance between sites, using standardized procedures. We propose a site visit methodology in which sites would be chosen using a random sampling method of at least 5 larger sites and 5 smaller sites (> 50 patients on ART), which would be equally distributed between implementing partners. Logistics of site visits would be developed by in- country staff prior to the review period.

24. External Assessment of ART Program Supervisory Systems Review Process: An existing supervisory tool (such as the one described in the Standard 1 sample action plan), or one developed for the purpose, would be used by all teams and address each of the [five] proposed standards. Each site review team would provide the site with a verbal review report and draft a site report with recommendations (following a standardized outline). The review teams would participate in a group debrief at the end of the review period with all stakeholders. A final report would be drafted by the review team for submission to the in-country stakeholders and USG headquarters.

25. 6 month to 1 year plan Pilot standards-based approach in 3 countries Hold meetings with USG country staff at Maputo meeting Summarize feedback on approach and share with relevant TWGs Meet with MOH and other country stakeholders to develop an implementation plan Conduct external assessment visits at randomly selected sites Evaluate pilot project results and modify process based on these results Expand Approach to other PEPFAR countries Enlist and train assessors for review teams Schedule preliminary meetings in countries Conduct assessments Report findings of assessments to relevant stakeholders and USG headquarters

26. Thanks To the team members who have participated in this project to date and who have greatly contributed to it. Laura Porter (LPorter@cdc.gov) Carla Johnson (CJohnson17@cdc.gov) Laura Broyles (LBroyles@cdc.gov) Joseph Barker (JBarker1@cdc.gov) To all of you who will need to work with us on this project Please give any comments that you may have to members of the team during or after the meeting