1. Introduction Governments and donors supporting social development programmes rely on implementing partners to produce high quality data for reporting on programmatic performance. Data derived from monitoring helps managers and funders to judge the efficiency and effectiveness of programme efforts. However, if data is not of high quality, the quality of programme efforts is questionable. RELIABILITY – the measure of consistency of data management processes – has been shown to be one of the most important aspects of overall data quality. This is particularly true of programmes that operate in multiple sites. Based on Khulisa’s experience in assessing data quality in SA Government programmes and among PEPFAR/South Africa partners, inconsistent data collection processes between sites represents one of the largest risks to Data Quality. Khulisa’s work in Data Quality is funded by the President’s Emergency Plan for AIDS Relief (PEPFAR) through a contract with USAID/South Africa. Tips for Improving Data Reliability  Comprehensive data management procedures and guidelines are given to all personnel who handle data (across sites).  Data Management Guidelines include Indicator Protocol Reference Sheets to fully define and operationalize indicators being measured.  An audit trail exists of all the data management steps, so that the organization itself can retrace the steps.  Documents and tools are version controlled with dates provided.  All decisions made with regard to the data management system are documented and incorporated into the IPRS and data management procedures and guidelines.  Same data collection tools used at all programme sites.  Ideally, same tool used for data collection, collation and reporting (thus eliminating transcription steps).  All data collectors are trained prior to data collection. Ideally, all data collectors are trained by the same person to ensure consistency in instructions.  Double entry of data into database, system designed to catch any discrepancies.  Quality control reviews for any data transcription or manipulation steps.  Secondary and tertiary data are checked for validity and reliability. Programme Monitoring: Data Collection Methods that Enhance Reliability J. Welty Mangxaba 1 and M.P. Selvaggio 1 1 Khulisa Management Services, P.O. Box 923, Parklands 2121, South Africa Common Pit-Falls Affecting Data Reliability  No documented standard operating procedures, thus resulting in inconsistencies in data handling methodology.  Not ensuring inter-tool reliability when different tools are used at different sites.  Different manipulation, analysis, or aggregation techniques used from one period to the next.  No standard collation tool used.  No audit trail kept of manual aggregations.  No version control of documents – can cause confusion if obsolete documents are inadvertently used.  An Indicator Protocol Reference Sheet contains multiple indicators; can cause confusion on specific process for each indicator.  Weak or no existing performance review process for data management personnel. Accurately Measuring Progress 26 - 7th Avenue Parktown North Johannesburg South Africa 2193 Tel: +27 11 447-6464 Fax: +27 11 447-6468 Web: www.khulisa.com