MICS Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop Data Entry and Processing

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files Data archiving

Content of the Presentation Overview of the MICS data processing system

MICS Data Processing System: Actors and Roles –Country data processing manager and country team: Customization of data entry programs, data entry, editing, and production of datasets Customization of tabulation syntaxes and tabulation –Regional Office MICS Coordinator Coordination and supervision, organization of the Data Processing workshop –Regional Office Data Processing Consultant Technical support and review of customized programs and close work with country teams –HQ Data processing unit Development of standard programs, templates and coordination of Data Processing workshops

MICS Data Processing System The data-processing system can be divided into following phases: –Customization of MICS data entry/collection program and tabulation syntaxes, –Establishing the data entry system locally, –Primary data processing (data entry/data collection), –Secondary data processing (creating analysis files), and –Tabulation.

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires

MICS Data Processing System Designed to deliver the first results of a survey within several weeks after the end of fieldwork simultaneouslySuch rapid turnaround time is possible when completed questionnaires are entered simultaneously with survey fieldwork all the questionnaires from a clusterData for each cluster is stored in a separate data file and is processed as soon as all the questionnaires from a cluster are returned from the field This approach breaks data processing down into discrete segments and allows it to progress while fieldwork is ongoing By the time the last questionnaires are finished and returned to headquarters, most of the data have already been processed

Customization of data entry files Adapting MICS standard data entry program files to the country-specific questionnaire –Second Regional Workshop [Data Processing] Adapting MICS standard tabulation syntax files Customisation of the programs during and immediately after the workshop

Establishing the data entry system The goal is to be ready to begin data entry shortly (about one to two weeks later) after the fieldwork commences The preparation phase involves the following steps: –Obtaining computer equipment and setting up a data- processing room –Identifying and recruiting appropriate personnel so that they participate to fieldwork personnel training –Setting up a system for managing the questionnaires and data files

Computers Operating system: Microsoft Windows XP, Vista, Windows 7, or Windows 8. Computers adequate to run Windows XP or better versions –Adequate hard disk space (265 megabytes memory and at least 70 megabytes of free disk space) –Adequate memory Network Printer

CSPro Windows software tool for census and survey processing Developed jointly by IPC-US Census Bureau, Macro International, SerPro Public domain Free download at – orwww.cspro.org – Software

CSPro Modules –Data entry –CAPI Data collection: laptops/tablets (PDAs v4.1) –Data editing and imputation –Frequencies and crosstabulations –Data manipulation utilities –Data dissemination –Export to statistical packages

CSPro –Data Entry Interactive Range, skip, consistency checks –Structure check Ensures completeness of data –Verification 100% double entry –Secondary editing Complex consistency checks –Export to SPSS

Primary Data Processing The goal of primary data processing is to produce clean, edited data files. Primary data processing involves the following steps: –Entering all questionnaires for a cluster onto a data file –Production of field check tables –Checking the structure of the data file –Entering the data a second time and then verifying the data file –Backing up the checked and verified data file –Performing secondary editing on the data file –Backing up the edited, or final, data file

Data processing requirements Computers –1 per data entry operator –Plus 1 for supervisor Arrange day/night shifts if workload is heavy Staff –Data processing supervisor (1 or 2) –Data entry operators (variable) Can enter about 1 cluster per day –Data editors (1 or 2) 6-7 hours per day including breaks

Flow of data entry

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system

Mobile data collection In the recent years we have seen development of many innovative data collection tools using handheld PocketPC personal digital assistants (PDAs), smartphones or tablet personal computers.

Tablet Personal Computer Tablet – sized computer with the key features of a full – size personal computer Various operating systems Computer vs. cell phone capability

Personal Digital Assistants Handheld computers Various operating systems –Windows mobile –Palm –Others (e.g. iPhone, Nokia) Cell phone capability –Plus: good communications –Minus: security

Tablet vs PDA Choosing a tablet over a PDA -Tablets have much larger screens, better resolutions, more space for on screen typing making things easier to see and more useful for data entry -Tablets and PDAs have similar battery life, though if PDAs are used for making and receiving calls battery will need more frequent charging -Better security

Software requirements MICS survey requirements –Keep track of the “data path” –Multiple languages on the fly –Roster manipulation –Saving an interview in the middle –Dynamically customized questions and response categories –Powerful programming language Able to handle complex consistency checks and skips –Ability to run the same application in PDAs, tablets or PCs –Capability to administer instruments at different times –Able to handle long/complex questionnaires

Hardware requirements for CSPro Tablets: Required configuration: Full Microsoft Windows 7 or 8 (NO windows RT tablets) PDAs: Required configuration: Windows Mobile versions 5 and 6 (note that UNICODE is not supported)

MICS CAPI System All applications to collect and administer data are written in CSPro Three systems: –Interviewers - data collection – Tablet/PDA –Supervisors – data monitoring and control – Tablet/PDA –Central Office - centralized data and monitor fieldwork - PC

Screen Reports Interviewer’s System Supervisor’s System Int 2 Int 5 Supervisor Central Office System System Updates Reports Central Office Repair Utility Update Utility

Questionnaires One question per screen –Question area –Answer area –(for text or numeric responses) –List of categories (where appropriate)

Customizable questions Multiple languages –Language can be changed during interviewing Questions customized to the individual Response categories also customizable Radio buttons for categorical variables Check boxes for multiple response variables

A Typical Screen Navigation/tools Display/enter values Possible Responses Questions Area

Virtual Keyboard Values in ranges or textual text needs to be typed Soft Keyboard

Dynamically Generate Answers Colors have meanings Value Sets populated

Benefits –Improved data quality? Pro – checking of data in real time during interview Con – possibility for interviewer’s to ‘avoid’ work Overall – positive improvements, but only with good quality controls in place –Eliminates missing information –Shortens interviews –Data ready almost immediately –Survey indicators monitored early in fieldwork

Equipment Power - battery charging is an issue –Tablets/PDAs generally give close to a full day of work –May require charging or swapping batteries during the day –Spare battery needed –Separate charger Need for extra tablets/PDAs –Needed for all trainees (not just number of interviewers/supervisors planned for fieldwork), plus IT staff –In case of problems/loss in the field Cases/covers –Good protection needed depending on expected conditions

Equipment Security issues –Theft of Tablets/PDAs is a risk –Increased risk if PDAs are also cell phones (smartphones) Virus risk –Tablet/PDA viruses from downloaded programs –PC viruses infecting SD cards (e.g. when SD cards are used in PCs in Internet cafés) Cell phones to communicate with field teams –Built in if PDA is smartphone

Data loss risk –Lost/stolen PDAs/Tablets –Hardware failure –Viruses –User error Risk management –Backup to SD card immediately after interview –Copy to supervisor at end of day –Paper questionnaires can be used in emergency

PDA Costs Basic Tablet/PDAs cost approx $450 with no accessories Plus approx $100 in needed accessories: SD card Spare battery Case Screen protector Spare stylus Vehicle chargers (1 per team)

Costs Costs balanced out by –Printing of only 10% of questionnaires –No field editors –No data entry –No computers required for data entry However –Technical assistance costs are higher! Cost savings –if Tablets/PDAs are re-used –Savings on secondary processing, data editing

Technical assistance Significant extra programming compared with classic data entry –Questionnaire programming is more complicated Addition of question text More customized response categories More complicated flow to facilitate interviewing Possible need for multiple languages –Data management programming much more complicated (even than questionnaires) Three systems (interviewer’s, supervisor’s, central) compared with two (data entry, data coordinator) Logistics of data management much more complicated

Preparation and Pretesting Early design of questionnaires needed –Last minute changes to questionnaires are difficult Programs prepared well in advance and thoroughly pretested Full pretest required to properly test all aspects of the system Programs must be ready and thoroughly tested before training begins –Updating applications during training is problematic Making the change is easy, but Preparing update for all Tablets/PDAs takes time Updating all Tablets/PDAs is very time consuming

Training –Paper questionnaires still important Discussions and version control Documentation Training –Facility for training (power, desks) –Good documentation –Train on paper first, then Tablet/PDA Gives the ‘big picture’ Paper can also be used in emergency in the field –Fieldwork testing very important –Recommended – 4 week training, Including 4-5 separate days of field practice Extensive practice during training is needed

Data transfer Logistics of transferring data to central office –Cell phone Only if PDA is smartphone –Wireless Limited Wifi locations –Internet cafés Virus risk on PCs –Field control teams Via bluetooth or SD cards More reliable, but slower

Field Staff –Interviewers should be computer literate Younger interviewers who are comfortable with cell phones quickly learn to use PDAs –Interviewers need an introduction to computers and to PDAs –Supervisors need to be very comfortable with applications –Balance between good interview experience and good computer knowledge

DP staff Several local data processing staff needed to troubleshoot in the field –User errors in using the system –Program malfunctions – usually related to user error –Lost data problems – usually found on the backups –PDA viruses –PC viruses on SD cards –Frozen PDAs –Problems transferring data –Problems related to data inconsistencies that cannot be resolved by the team –Upgrading programs Central office staff for secondary processing

Conclusions Quicker, better results are possible Many complications –Costs likely higher –Logistics much more complicated –Technical assistance needs increased Don't underestimate how much work it takes to properly implement a survey, even with Tablets or PDAs. –It is very easy to conduct a poor survey

MICS Pilot experience: Duration of interview Household questionnaire –33 minutes to administer paper household questionnaire. –19 minutes to administer PDA household questionnaire. Women questionnaire –15 minutes to administer paper women’s questionnaire. –12 minutes to administer PDA women’s questionnaire. Men questionnaire –15 minutes to administer paper men’s questionnaire. –9 minutes to administer PDA men’s questionnaire. Under 5 questionnaire –18 minutes to administer paper U5 questionnaire. –10 minutes to administer PDA U5 questionnaire. Important to add around 5-10 minutes needed for data transfer after completing household questionnaire

MICS Pilot experience: Duration of interview Household questionnaire –33 minutes to administer paper household questionnaire. –19 minutes to administer PDA household questionnaire. Women questionnaire –15 minutes to administer paper women’s questionnaire. –12 minutes to administer PDA women’s questionnaire. Men questionnaire –15 minutes to administer paper men’s questionnaire. –9 minutes to administer PDA men’s questionnaire. Under 5 questionnaire –18 minutes to administer paper U5 questionnaire. –10 minutes to administer PDA U5 questionnaire. Important to add around 5-10 minutes needed for data transfer after completing household questionnaire.

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files

Secondary Data Processing The goal of secondary data processing is to produce analysis data files and to create the MICS standard tables. Secondary data processing involves the following steps: –Bringing together all cluster data files into one data file –Exporting the data to the SPSS software –Recoding some variables to be used in analysis –Calculating sample weights and adding to data files –Computing wealth index and adding to data files –Creating the tables required to analyse the data –[Archiving and distributing the data files]

Software SPSS v 20 –Construction of analysis files HH: Household HL: Household Listing WM: Women MN: Men CH: Children –Production of tabulations –Analysis of sampling errors/confidence intervals

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files Data archiving

Data archiving: rationale Collecting data is expensive. Data should be used beyond producing basic report. Survey microdata are valuable resources for government departments and academic researchers. Survey data constitute valuable and irreplaceable assets which should be managed in a way that encourages their widest possible use and re-use. At the same time, data collectors main focus should be protecting respondents while making microdata assessable.

Role of MICS data archiving Preservation Documentation Anonymization Dissemination of microdata

Role of MICS data archiving

Role of MICS data archiving 11 Piped into dwelling 12 Piped into compound, yard or plot 13 Piped to neighbour 14 Public tap / standpipe 21 Tube Well, Borehole 31 Protected well 32 Unprotected well 41 Protected spring 42 Unprotected spring 51 Rainwater collection 61 Tanker-truck 71 Cart with small tank / drum 81 Surface water (river, stream, dam, lake, pond, canal, irrigation channel) 91 Bottled water 96 Other (specify)

What do users expect? Well documented data Comprehensive Clear, consistent, easy to use data Information to be able to –Fully understand the survey, especially Sample design, selection and weighting Field procedures Data processing Datasets –Accurately analyze and use data

When to archive? Start archiving when you start the survey Typically, datasets are documented after completion of the survey

Microdata Management Toolkit Tools to facilitate archive and dissemination of surveys International Household Survey Network [IHSN] Established in September 2004 International organizations actively sponsoring household surveys + NSOs Coordination: ILO, DFID, Paris 21, UNICEF, UNSD, WHO, World Bank

Data sharing and ownership of data Data are the property of the country UNICEF agreements with countries should permit UNICEF (CO, RO and HQ) and country partners to: –Receive datasets and tabulations as soon as data processing is complete –Receive reports, including drafts, as soon as available and prior to publication –Distribute datasets, tabulations and reports to other users and researchers after publication of report

Why share data? Permits further analysis of data by researchers Leads to greater benefit in terms of maximizing the results of the survey by –Providing more insights into situation of children and women in each country –Providing more valuable information for planning of programmes Leads to greater confidence in survey results

MICS Data Processing Workshop Goal: get familiar with software and standard programs, practice customization of the application and for countries with draft questionnaires, modify standard data entry and processing programs to reflect country questionnaires Structure –Introduction of software –Lecture: Overview of data processing components –Practical: customization of the standard programs

MICS Data Processing Workshop Who will participate? –Only staff responsible from data entry and/or data processing (from implementing agency) Experts in data entry and/or processing Prior expertise in data entry supervision and/or data processing 2-3 participants per country –MICS UNICEF consultant is expected to participate since she/he is directly involved in all phases of the survey

Thank you