1. Copyright 2010, The World Bank Group. All Rights Reserved. Managing Data Processing Section B

2. Copyright 2010, The World Bank Group. All Rights Reserved. What is Editing? Inspection and correction of responses according to predetermined rules either manually or by computer programs. 2

3. Copyright 2010, The World Bank Group. All Rights Reserved. Types of Edits Enumerator edit Field edit Preliminary office edit Manual central office edit Computer edit 3

4. Copyright 2010, The World Bank Group. All Rights Reserved. Types of Errors Omission Out of scope entries Inconsistent entries Unreasonable magnitudes 4

5. Copyright 2010, The World Bank Group. All Rights Reserved. Principles of Editing Minimum number of changes possible Eliminate obvious inconsistencies Supply missing entries when possible 5

6. Copyright 2010, The World Bank Group. All Rights Reserved. Correction Methods Cold Deck Method - static imputation Hot Deck Method – dynamic imputation Not Reported – no imputation 6

7. Copyright 2010, The World Bank Group. All Rights Reserved. Editing Instructions Should cover all known possible errors and how to correct them Written by subject matter experts upon review of data Manual editing training should include test decks Editing updates should be distributed systematically Computer edit instructions are a collaborative process 7

8. Copyright 2010, The World Bank Group. All Rights Reserved. Manual Editing Considerations Consistency is mandatory No record of changes means loss of original data Each clerk’s work must be monitored and verified Reduce all manual processing to absolute minimum to reduce: –Time –Cost –Introduction of errors 8

9. Copyright 2010, The World Bank Group. All Rights Reserved. Structure Edit Performed after data conversion to electronic data file Control totals used to check completeness Corrections done with questionnaires Limit checks to rendering questionnaires ready for computer processing (processability) 9

10. Copyright 2010, The World Bank Group. All Rights Reserved. Pre-Edit Data Consolidation Follow pre-established geographical priorities Check control totals Use operational control data base Avoid geographic coding (geocode) conflicts in joining files 10

11. Copyright 2010, The World Bank Group. All Rights Reserved. Computer Editing Team Data are reviewed to identify errors and overall data trends Edit specifications are written by subject matter specialists and implemented by programmers Edited data is reviewed to assess impact of editing decisions on data Process continues until staff are satisfied with edits No editing decision has to be final as all data versions are kept for comparability 11

12. Copyright 2010, The World Bank Group. All Rights Reserved. Consistency Edit Controls Develop consistency specifications Prioritize variables Monitor corrections Use control tabulations Re-run output file 12

13. Copyright 2010, The World Bank Group. All Rights Reserved. Example of Consistency Edit Specification Variable: Sex Valid Responses: –1 Male –2 Female Possible Problems: –Head same sex as spouse –Male with fertility information –Missing code –Invalid code (3) 13

14. Copyright 2010, The World Bank Group. All Rights Reserved. Pre-Tabulation Consolidation Controls Consolidate to facilitate tabulation Check control totals for each record type Use standardized forms for operational control Avoid geocode conflicts in joining files 14

15. Copyright 2010, The World Bank Group. All Rights Reserved. Tabulation Specifications Table shell with headings, rows and each cell defined in terms of how it is calculated from variables Shows layout, provides text for headings, and ensures no mistakes in calculation of table cells Facilitates communication and expedites programming 15

16. Copyright 2010, The World Bank Group. All Rights Reserved. Production of Tabulations Table specifications Table programming and refinement by tabulation team Final tabulation number checks Special request tabulations 16

17. Copyright 2010, The World Bank Group. All Rights Reserved. Tabulation Controls Computer Program Specifications User Approval of Tables Tables Grouped by Characteristics Tables Checked Against Control Totals Control Tables Show Weighted and Un-weighted Numbers Geographic Subtotals Match Final Table Review Data Dissemination to User Community 17

18. Copyright 2010, The World Bank Group. All Rights Reserved. Dissemination Plans Publication working group Description of publication program Directives Calendar of activities 18

19. Copyright 2010, The World Bank Group. All Rights Reserved. Publication Preparation Exhibits Standard portions of text Methodology Maps Historical data 19

20. Copyright 2010, The World Bank Group. All Rights Reserved. Section B Quiz 1.Briefly describe what is checked during structural and consistency editing? 2.What are the types of errors that need correction? 3.What statistical staff should develop edit programs? 4.Are table specifications necessary? 20