1. Linking Data with Action Part 2: Understanding Data Discrepancies

2. What are data discrepancies?  Different data sources with different estimates for the same indicators  Example – HIV prevalence among men and women of reproductive age, 2007  DHS, 4.1%  Sentinel surveillance, 6.4% Does the difference matter? Which estimate should I use? ?

3. What contributes to data discrepancies? Population – understand study population  Who is in the study?  Where are they located?  How were they selected?  How do they compare with the greater population?

4. How are study samples selected? 1. Probability Sampling - each individual has an equal chance of being chosen because they are randomly selected. 2. Non-probability Sampling – chance of selecting any individual is not known because they are NOT randomly selected.

5. Study Populations – RDHS: 4.1% HIV prevalence ANC Surveillance: 6.4% HIV prevalence QuestionsDHS 2007Antenatal Surveillance 2007 Who is in the study?6,641 Men & women of repro age 13,321 Pregnant women attending ANC How do they compare?Not all are sexually active, variable contraceptive use All are sexually active, none using contraception How were they selected? Random sampleAll women attending ANC Where are they located? Nationally representative household sample 30/359 health centers – 2 capital, 12 other urban, 16 rural

6. What contributes to data discrepancies?  Error  Random  Systematic  Definitions  Indicators  Terminology  Data Quality

7. 2 Studies – Same indicator: No. of women who complete PMTCT services Study A: 95% Criteria:  Counseled  Tested  Received test result Study B: 75% Criteria:  Counseled  Tested  Received test result  Positive women & babies treated

8. What contributes to data discrepancies? Terminology  2010 Behavioral Surveillance Survey: 51% HIV prevalence (n = 1,338 FCSW)  ‘High Burden of Prevalent & Recently Acquired HIV among Sex Workers & Female HIV VCT clients in Kigali, Rwanda’ by Braunstein, et al.: 24% HIV prevalence (n = 800 FCSW)

9. What contributes to data discrepancies?  Bias  Random  Systematic  Definitions  Indicators  Terminology  Data Quality

10. Validity Accuracy: Does the data reflect what it is intended to measure? Reliability Consistency: Does the data measure a concept or characteristic consistently? Completeness Complete: Is all the data collected and considered? Precision Precise: Is the data described in sufficient detail? Timeliness Current: Is the data current? Does it reflect actual program activities? Integrity Data are protected from deliberate bias or manipulation for political or personal reasons. Dimensions of Data Quality Slide 5 of 18

11. Program Outcome Errors  “False Positive”: program had an effect when it did not (linked to significance level or p-value)  “False Negative”: failing to detect a true program effect (linked to significance level or p-value)  “Implementation Error”: No program effect due to lack of or inappropriate implementation.

12. What can help you interpret data discrepancies?  Probe, question, investigate  Confidence Intervals  Amount of uncertainty of an estimate, example – 3% (1%, 5%), 95% confidence intervals OR 3% (±2%), p ≥.05  Compare to other data sources  Ask an expert

13. Confidence intervals StudyEstimate95% Confidence interval Study A53%49-57 Study B61%59-63

14. Confidence intervals StudyEstimate95% Confidence interval Study A53%48-58 Study B61%57-65

15. What can help you interpret data discrepancies?  Probe, question, investigate  Confidence Intervals  Comparison to other data sources  Ask an expert

16. Small Group Exercise: Data Discrepancies  2005 Demographic Health Survey (DHS)  2005 Priorities for Local AIDS Control Efforts (PLACE) assessment

17. Small Group Exercise: Data Discrepancies  Who is in the study?  Where are they located?  How were they selected?  How does each study group compare to the greater population?  How are the 2 studies different?  What is the use of each type of data for program planners? For policy makers?

18. Small Group Exercise: Data Discrepancies  Who is in the study?  Where are they located?  How were they selected?  How does each study group compare to the greater population?  How is the data in the 2 studies different?  What is the use of each type of data for program planners? For policy makers?

19. This research has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement GHA-A-00-08-00003-00 which is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed are not necessarily those of PEPFAR, USAID, or the United States government.