Data Quality Assessment of PEPFAR ART Sites In Nigeria Final Report February 17, 2006 Nigeria/Monitoring and Evaluation Management Services in collaboration with MEASURE Evaluation

What is Data Quality? Data Quality fundamentally means that the information collected as part of a program’s monitoring and evaluation system adequately represents the program’s activities.

Adequately Represents? The information is accurate and reliable. Accurate information is interpreted as measuring what we intend to measure (that the information is correct).

Role of Data Quality in the Emergency Plan PEPFAR emphasizes data quality because the Emergency Plan is explicitly evidence based and results oriented Good data are needed to inform the design of interventions and monitor and evaluate the plan’s quantitative progress toward pre-determined treatment, prevention, and care targets Hence, if the effectiveness of the emergency plan is evaluated on the basis of numbers, then any doubt about those numbers makes the entire program vulnerable to criticism

Data Quality Assessment of PEPFAR ART Sites in Nigeria

Assessment Purpose Determine quality of data used to report on PEPFAR ART indicators Examine data collection, management and reporting procedures Ascertain site capacity for M&E Document degree of data sharing and harmonization among PEPFAR, GON, and other HIV/AIDS Programs Assess cross-agency data comparability

Methods and Facilities Field Visits – August 30 – September Seven out 9 PEPFAR’s targeted States covered 16 facilities out 23 assessed Qualitative and Quantitative techniques  Mapping of patient flow within facility  Structured interview  Observations  Accuracy checklist  Geographic Information System (GIS) coordinates

Nnewi ART Assessments took place:

Data Quality Assessments Measure Validity Validity: Does data adequately represent performance? Reliability Reliability: Are data collection and management procedures consistent? Precision Precision: Do data have sufficient detail to present a fair picture of performance? Timeliness Timeliness: Are data current and collected with required frequency? Integrity Integrity: Are data free of manipulation for political, professional or personal reasons?

DQA Methods Mapping

DQA Findings: Successes M&E personnel were well trained and highly motivated Written/posted M&E guidelines and flow charts at many sites All sites had or were about to implement an electronic data reporting system Well coordinated ART patient administration, testing, treatment, counseling and tracking systems

DQA Findings: Challenges ART eligibility guidelines not uniform among sites Physicians complained about number and redundancy of required forms Forms and databases not uniform across IPs, preventing unified reporting system Tensions between PEPFAR IPs providing services at same site

DQA Findings: Challenges (cont.) Outsourcing of lab tests hampered document monitoring Lack of coordinated supervisory system led to fragmented data monitoring and processing delays Inadequate Staff and equipment

Conclusion The provision of ART services was a ‘work in progress.’ Most sites had only just started operating when assessments occurred –two clinics had not yet opened PMM harmonization across all ART sites in progress For the complete ART Assessment report, log on to:

Updates after Assessment USG-SI-Advisor convened SI-working group USG through IPs have engaged GON on needs for harmonization of tools USG-supported GON in the harmonization process Stakeholders’ meeting held Current PMM tools under review and awaiting approval by GON New harmonized tools to be piloted in scale facilities

DISCUSSION and QUESTIONS