1. School of Health Systems and Public Health Monitoring & Evaluation of HIV and AIDS Programs Data Quality Wednesday March 2, 2011 Win Brown USAID/South Africa Slide 1 of 18

2. Objectives of the Session To Review and Discuss: –A Data Quality approach to M&E –Six important elements of data quality –Practical applications Slide 2 of 18

3. Why Data Quality? Program is “evidence-based” Data quality  Data use Accountability Slide 3 of 18

4. Real World In the real world, activities are implemented in the field. These activities are designed to produce results that are quantifiable. Data Management System An information system represents these activities by collecting the results that were produced and mapping them to a recording system. Data Quality: How well the DMS represents the real world Real World Data Management System Data Quality ? Slide 4 of 18

5. Validity Valid data are considered accurate: They measure what they are intended to measure. Reliability The data are measured and collected consistently; definitions and methodologies are the same over time. Completeness Completely inclusive: the DMS represents the complete list of eligible names and not a fraction of the list. Precision The data have sufficient detail; in this case the “accuracy” of the data refers to the fineness of measurement units. Timeliness Data are up-to-date (current), and information is available on time; the DMS produces reports under deadline. Integrity The data are protected from deliberate bias or manipulation for political or personal reasons. ? = Dimensions of Data Quality Slide 5 of 18

6. Good Data are Valid and Reliable X X X X X X X X X X XXX XXXX XXX XXX XXXX XXX Valid Reliable ≠ Valid Reliable ≠ Valid ≠ Reliable Slide 6 of 18

7. What are: – Valid data? – Reliable data? – Complete data? – Precise data? – Timely data? – Data with integrity? Slide 7 of 18

8. Framework for Enhancing Data Quality Data Management System Data Management Processes / Procedures Data Quality System Data Quality Processes / Procedures Auditable System Document! Risk Verification Source Validity Reliability Completeness Precision Timeliness Integrity Paper Trail that allows verification of the entire DMS and the data produced within it Collection Collation Analysis Reporting Use Slide 8 of 18

9. The South Africa Approach Data Quality Assessment Training Data Warehouse SASI Manual Standard M&E plan  DQ Plan Slide 9 of 18

10. Data Quality Assessment PMTCT Data; District focus Trace and Verify Routine Data Quality Assessment Tool (RDQA) Slide 10 of 18

11. M&E Training? M Routine data collection Data quality Results reporting Strategic planning E Internal validity Operations research Instrument design Survey sampling Data analysis for data use Local training partners Participant follow-up User’s groups/networks Slide 11 of 18

12. PEPFAR Reporting Issues Are PEPFAR’s results valid & reliable? How do you know? Are your patient numbers valid & reliable? How do you know? Slide 12 of 18

13. 0%10%20%30%40%50% Percent reporting: “I understand statistics.” 100 50 1 # random samples drawn Data and Statistics are Empowering Slide 13 of 18

14. Data Warehouse Online results reporting system Standardized data capture Control of data quality Customized reporting tool Online indicator guidance Slide 14 of 18

15. South Africa Strategic Information Manual (SASI Manual) Operational manual Standard definitions for PARTNERS Addresses common data quality problems Slide 15 of 18

16. Try Making a Data Quality Plan Component of the M&E plan Strategically think about data quality Slide 16 of 18

17. Measurement With monitoring of progress in a clinic or in a community, always try to hit the bull’s eye. Paper Trail Always document progress. Data Use Who is using the data? Slide 17 of 18

18. Thank you Slide 18 of 18