Copyright 2010, The World Bank Group. All Rights Reserved. Managing Data Collection Section B 1

Copyright 2010, The World Bank Group. All Rights Reserved. Quality Control Tools Listing sheet controls Interviewer observations Formal / Informal reviews Reinterview Address listing QC 2

Copyright 2010, The World Bank Group. All Rights Reserved. Listing Quality Control Tools Dependent Quality Control, including: Advance listing Suppression If errors are found, choices include: Listing mistakes corrected Re-list enumeration area 3

Copyright 2010, The World Bank Group. All Rights Reserved. Dependent Quality Control Supervisor or another enumerator checks a sample of the listing. Copy listing information for select units without enumerator knowledge. Review a randomly chosen area to verify coverage, check for omissions and completeness. 4

Copyright 2010, The World Bank Group. All Rights Reserved. Other Listing Quality Control Tools Advance Listing –Supervisor lists sample before enumerator –Compare work when enumerator completes the sample section. Suppression –Enumeration units are removed from listing –Supervisor checks for enumerator updating of suppressed (omitted) units 5

Copyright 2010, The World Bank Group. All Rights Reserved. Interviewer Observations Supervisor watches enumerator work Does not interact to minimize bias Makes notes during observation Provides feedback after observation 6

Copyright 2010, The World Bank Group. All Rights Reserved. Interviewer Observations Assess effectiveness of training Ensure enumerators can do their work Provides continuous feedback Corrective actions may be needed 7

Copyright 2010, The World Bank Group. All Rights Reserved. Formal / Informal Reviews What are the goals of reviews? To ensure the quality of the data collected on forms and questionnaires To ensure interviewers and clerks understand and follow procedures for completing forms and questionnaires 8

Copyright 2010, The World Bank Group. All Rights Reserved. Informal Reviews Supervisor reviews enumerator forms during enumeration Check for legibility, completeness and accuracy Provide feedback Repair errors 9

Copyright 2010, The World Bank Group. All Rights Reserved. Formal Reviews Conducted by field office staff upon completion of EA Provides check of supervisory work Repair errors or return to field where necessary 10

Copyright 2010, The World Bank Group. All Rights Reserved. Random Reinterview Another field staff member interviews an enumeration unit after enumerator Finds same respondent and completes new questionnaire Questionnaires are compared to verify content 11

Copyright 2010, The World Bank Group. All Rights Reserved. Random Reinterview Sample Supervisor takes simple random sample from each enumerator’s workload Above supervisory level uses two-stage sample: –Sample of enumerators in their area –Sample of enumeration units from those enumerators 12

Copyright 2010, The World Bank Group. All Rights Reserved. Reinterview Errors Falsification (release interviewer) Interviewer error (retraining/feedback) Respondent error (no action) Rework may be necessary for falsification or other serious errors 13

Copyright 2010, The World Bank Group. All Rights Reserved. Administrative Reinterview Accumulates data on work characteristics for all interviewers Vacant enumeration units Partial interviews Enumeration units by type/location Deleted enumeration units Average size per enumeration unit 14

Copyright 2010, The World Bank Group. All Rights Reserved. Administrative Reinterview Provides a more targeted and informed approach to identifying outliers Does not replace other reviews Management tool and provides organizational operational data 15

Copyright 2010, The World Bank Group. All Rights Reserved. Administrative Reinterview Compile data for all enumerators Compare individual data to compiled data Note areas of variance for investigation Could be that individual is performing better than group 16

Copyright 2010, The World Bank Group. All Rights Reserved. Administrative Reinterview Actions Cannot take corrective action without investigation Increase suspect enumerator reviews: –Informal/formal review –Reinterview 17

Copyright 2010, The World Bank Group. All Rights Reserved. Supplemental Reinterview Purposeful selection of a particular enumerators work for reinterview Supervisor selects sample size –Ensure quality –Avoid over-sampling 18

Copyright 2010, The World Bank Group. All Rights Reserved. Supplemental Reinterview Means for checking work of selected enumerators Especially useful for suspected falsification Should be stopped once quality is of acceptable level 19

Copyright 2010, The World Bank Group. All Rights Reserved. Section B Quiz 1.What are the five quality control tools available for data collection operations? 2.What are three listing quality controls? 3.Explain how to conduct interviewer observation. 4.Name the three types of reinterviews that can be used to assess the content and quality of data collection. 5.Is falsification a serious problem? Yes or No 20