Quality Control Issues During Data Analysis Demographic Analysis Tom McDevitt, Population Studies Branch, International Programs Center, U.S. Census Bureau

Anatomy of a Survey/Census - 5 Phases Contract Negotiation Design and Development Data Collection Post-Collection Processing Analysis and Dissemination Each phase has its own: Objective Key tasks Deliverables Documentation

The Data Analysis and Data Collection Link Data analysis generates information constrained by census/survey design. Data evaluation identifies limitations. Census/survey design builds on the previous two operations.

Quality Control Issues During Data Analysis Strategies for Identifying Errors and Adjusting International Demographic Data Strategies of the U.S. Census 2000 Plan

Pay now or pay later.

There is no such thing as a free lunch. Pay now or pay later.

Strategies for Identifying Errors and Adjusting International Demographic Data

Demographic data must be evaluated and a decision taken

Demographic data must be evaluated and a decision taken: Accept the data without modification.

Demographic data must be evaluated and a decision taken: Accept the data without modification.  Adjust the data.

Checking data quality using a population pyramid

Checking inter-censal consistency of data with cohort analysis

Quality Control in Analysis Product control –In order to control quality in census/survey analysis, we need definitions of what is acceptable for each product, decision rules to determine which products are accepted or rejected, and appropriate actions to take based on the results of the decision. Process control –Control the methods used to monitor the operation. –Control the steps that determine product acceptance and, in the extreme, when an employee needs to be retrained or released.

Quality Control Systems, Demographic Analysis at IPC Methodology standardization – PAM & PAS Standardized documentation procedures –Within work files –Paper documentation Verification Supervision Review of analyst’s work by others

Quality Control Points in Demographic Analysis at IPC Analyst-- data re-entry and analysis

Quality Control Points in Demographic Analysis at IPC Analyst-- data re-entry and analysis Verifier100% verification

Quality Control Points in Demographic Analysis at IPC Analyst-- data re-entry and analysis Verifier100% verification E&P country coordinator Branch chief Senior demographer

Quality Control Points in Demographic Analysis at IPC Analyst-- data re-entry and analysis Verifier100% verification E&P country coordinator Branch chief Senior demographer

Quality Control Points in Demographic Analysis at IPC Analyst-- data re-entry and analysis Verifier100% verification E&P country coordinator Branch chief Senior demographer Author Statistical review Publications and printing Author Product

Strategies of the U.S. Census 2000 Plan

Strategies of the U.S. Census 2000 Plan 1. Build partnerships. 2.Keep it simple. 3.Use technology intelligently. 4.Use statistical methods.

Alternative strategies exist for addressing the issue of the quality of demographic data, whether these be from the United States or from another country. It's a matter of choice:  Do it right the first time. Collect data using procedures that minimize error and maximize data quality.  Attempt to correct the data, to compensate for error, at the analysis stage. (Work with data collection staff to improve data collection next time). Quality in Demographic Data

Alternative strategies exist for addressing the issue of the quality of demographic data, whether these be from the United States or from another country. The choices are essentially 3 in number:  Do it right the first time. Collect data using procedures that minimize error and maximize data quality.  Attempt to correct the data, to compensate for error, at the analysis stage.  Ignore data quality Quality in Demographic Data

Lessons of Experience In demographic analysis, use multiple techniques and compare results. Use multiple datasets where possible, and recognize that reference “standards” must also be evaluated. Graph data. Verification – “Trust, but verify.” Monitor and review an analyst’s work. Learn from experience, and provide feedback to the data collection planning effort.

Two Pathways and the Goals of Data Quality: Relevance Accuracy Timeliness Accessibility Interpretability Coherence

There is no such thing as a free lunch. Pay now or pay later In most instances, in order to obtain good population data, it is cheaper, easier, better to pay “up front.”