AADAPT Workshop South Asia Goa, December 17-21, 2009 Maria Isabel Beltran 1

2  For evaluation purposes:  Administrative data  Surveys  our focus, we can complement with other sources of information ▪ Household ▪ Plot ▪ Associations ▪ Community  Census and other country surveys

3  Who collects the data? 2 main cases:  The ministry ▪ Hiring of enumerators? Who are they going to be? ▪ People inside the project have incentives to present a better or worse picture for their areas ▪ A lot of effort to follow the process  An agency (statistical office or private firm) ▪ OK, this is the type of work they do, but STILL A LOT OF EFFORT is needed to ensure quality (TORs, sample, questionnaire, training, supervision)

4  Questionnaire design  Training  Pilot test (and re-training)  Field work  Supervision  Data entry & data cleaning

5  Who defines it? YOU (the IE team, not the firm)  Purpose of survey? Define: respondents, indicators, level, modules. Time & quantity trade off  Internal consistency  Omission of key issues & skip patterns  Clear and explicit questions for all circumstances  Avoid open questions (pre-code) / recall period  Respondent burden, sensitive issues last

 Often underestimated part of the process.  Training  reduce variability in data collection  Pilot  ensures the questionnaire is collecting all information needed to answer questions, all correct information, flows and logic of the questionnaire.  Test the instruments  cover all conceivable situations  Involve the enumerators in the project  the importance of the data collected.

8  Almost always, it is better if organized in groups of enumerators (2-3)  Time Vs. quality  Have a clear field work plan and division of responsibilities among the group  Daily targets  Gambia: Enumerator 1Enumerator 2Enumerator 3 Talks to head teacherChildren, math test Classroom observation Head teacher question. Children, reading testTeacher tests Oral tests

9  Supervision protocol, 1 per 2 teams?  Have a supervision strategy: 10% of the sample, 100% ? Only non valid responses?  Use an independent firm or team; that has received the training  Supervise the supervisors

10  No need to wait for data collection to finish to start data entry. Make corrections while the data is still being collected. (Missing values, inaccuracies)  Integrated concurrent data entry Vs. Concurrent Centralized data entry Vs. Computer assisted interviews  Data entry: ONE TIME NOT ENOUGH  double entry at the same time, one after the other, one with supervision, … etc  If not planned… data cleaning = long & frustrating  Data is lost, quality decreases (decisions not documented)

11  Integrate the data collection and data entry.  Timely data  Feedback on field work on real time  Early detection of errors (like lack of uniform criteria)  The Medical Advice, Quality and Absenteeism in Rural India  project of the Center for Policy Research, New Delhi

3 separate firms: data collection, supervision, data entry Define all possible error per questions and program them type0: No error type1: ID error type2: Formattin g error type3: entered skip code but not skipped type4: skipped but no skip code type5: cross check error type6: header ID does not match page 1 type7: blank instead of -99 type8: one digit instead of two Total s1q s1q s1q s1q s1q s1q s1q

14  Relevant data  Reliable data  Data that is ready when needed… ON TIME, to answer operational and policy questions. Need to have staff dedicated to the project in all phases (design, preparation, implementation, dataset documentation & validation)  field coordinator.

15