Multiple Indicator Cluster Surveys Data Processing Workshop Overview of Data Processing System MICS Data Processing Workshop

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files Data archiving

Content of the Presentation Overview of the MICS data processing system

MICS Data Processing System: Actors and Roles –Country data processing manager and country team: Customization of data entry programs, data entry, editing, and production of datasets Customization of tabulation syntaxes and tabulation –Regional Office MICS Coordinator Coordination and supervision, organization of the Data Processing workshop –Regional Office Data Processing Consultant Technical support and review of customized programs and close work with country teams –HQ Data processing unit Development of standard programs, templates and coordination of Data Processing workshops

MICS Data Processing System The data-processing system can be divided into following phases: –Customization of MICS data entry/collection program and tabulation syntaxes, –Establishing the data entry system locally, –Primary data processing (data entry/data collection), –Secondary data processing (creating analysis files), and –Tabulation –Archiving

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires

MICS Data Processing System Designed to deliver the first results of a survey within several weeks after the end of fieldwork simultaneouslySuch rapid turnaround time is possible when completed questionnaires are entered simultaneously with survey fieldwork all the questionnaires from a clusterData for each cluster is stored in a separate data file and is processed as soon as all the questionnaires from a cluster are returned from the field This approach breaks data processing down into discrete segments and allows it to progress while fieldwork is ongoing By the time the last questionnaires are finished and returned to headquarters, most of the data have already been processed

Primary Data Processing Flow Main Data Entry Structure Check Verification Data Entry Difference Listing Backup Raw Data Secondary Editing Backup Final Data

Primary Data Processing Main data entry –First time data is entered Structure check –Checks structure of data files Verification data entry –Second time data is entered Difference listing –Two data files are compared; differences resolved

Primary Data Processing Raw data backup –Verified data are backed up to a directory containing raw data files Secondary editing –Complex inconsistencies are investigated and resolved if able Final data backup –Edited data are backed up to a directory containing edited data files

Data Processing Personnel Questionnaire administrators Data entry operators Secondary editors Data processing supervisor

Questionnaire Administrators Receive clusters from the field Check that all questionnaires are present Check that questionnaires are ready to enter Check that HH and individual questionnaires are in the proper order, and rearranges them if not Keep track of location of all clusters Should follow interviewer training

Data Entry Operators Enter main data Enter verification data Resolve differences between files Must follow interviewer training Must be familiar with the questionnaires

Secondary Editors Investigate complex inconsistencies Tell supervisor if and how to resolve inconsistencies Review editing guidelines Must be present during interviewer training Need excellent understanding of questionnaire and goals of survey

Data Processing Supervisor Resolves data entry problems Maintains programs Oversees entire data processing system Must be present during interviewer training Must have excellent grasp of questionnaire Must have programming skills in SPSS and CSPro

Questionnaire Administrator Training Review list of checks in data processing chapter Give QA several clusters and check work Establish questionnaire storage procedures

Data Entry Training Begin when you have one cluster for each data entry operator Allows you to –Train data entry operators –Debug programs Practice verification at the same time When you have finished –Fix entry programs –Delete data files

Secondary Editor Training Wait until you have 3-4 clusters double-entered Give secondary editors –Copy of editing guidelines –A cluster’s error listing –The cluster’s questionnaires Review work with secondary editor Try to schedule a day when data entry operators aren’t working

Data Processing Equipment Data entry machines –Windows XP, 2000, Vista,7 or 8 –Supervisor’s machine –Windows XP, 2000, Vista,7 or 8 –Available disk drive space for all software and MICS data files Uninterrupted power supplies (UPS) Network Surge Protectors

Data Processing Equipment A printer Paper Toner cartridges/printer ribbons Flash drives (if network is not established) Green pens

Data Processing Rooms Data Entry –Desk space for the monitor, keyboard, and stack of keyed and unkeyed questionnaires –Area to store “in-progress” clusters Editing –Quiet space for editors to work Questionnaires –Must contain means (e.g., shelving) to organize questionnaires by cluster

Server and Keyer’s Directory Structure MICS5 CSPRO DATA contains data from main entry ENTRY contains data entry programs VERI contains data from verification entry

Supervisor’s Directory Structure MICS5 CSPRO BACKUP backup of verified data DICTS dictionaries used in all programs (not just entry) ENTRY contains data entry programs EXPORT programs to transfer data from CSPro to SPSS FINAL a copy of all edited data GPS GPS data entry program (if applicable) RAW contains all unedited data (just after differences resolved) SUPER all programs not related to data entry phase SPSS contains all SPSS programs used for export and tabulation WEIGHTS contains weights spreadsheet and data files

Data Entry Cluster reaches the office Cluster acceptance is done. All the questionnaires are counted and recorded according to their type and result code to the cluster tracking form (CTF). Cluster is sent to Supervisor MAIN DATA ENTRY First keyer enters the whole of the questionnaires of the cluster. Supervisor runs the structure check control. If all questionnaires are entered, gives to second keyer for verification and records this information to system and CTF. VERIFICATION Second keyer enters the whole of the questionnaires of the cluster. Supervisor verifies the main and verification data by the supervisor menu. If there is typing errors, lets the keyers correct them at their computers. If no problem, the raw data is backed up. Sytem automatically gives date, it should be written to CTF Supervisor records the information on CTF to the system. Cluster is ready for main data entry. Supervisor gives it to first keyer and records keyers information to system and to CTF.

After backing up raw data “Back up the final data” After backing up of raw data: Secondary editing is done by the supervisor menu. ERRORS NO ERROR Secondary Editor: By using the “Editing Manual “ the necessary changes at the data with “Modify the data” option at Supervisor menu is done. Supervisor follows the same procedures for every cluster If all procedures are completed for all clusters: Export the data to SPSS

NETWORK STRUCTURE Network Computer:\\MICS5\CSPRO\DATA \VERI \ENTRY\entry.enc \entry_menu.enc Keyer Computer: \\MICS5\CSPRO \DATA \VERI \ENTRY\entry.enc \entry_menu.enc \entry_menu.pff Supervisor Computer: \\MICS5\SPSS \\MICS5\CSPRO\BACKUP \DICTS \ENTRY \EXPORT \FINAL \GPS \RAW \SUPER

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system

Mobile data collection In the recent years we have seen development of many innovative data collection tools using handheld PocketPC personal digital assistants (PDAs), smartphones or tablet personal computers.

Tablet Personal Computer Tablet – sized computer with the key features of a full – size personal computer Various operating systems Computer vs. cell phone capability

Personal Digital Assistants Handheld computers Various operating systems –Windows mobile –Palm –Others (e.g. iPhone, Nokia) Cell phone capability –Plus: good communications –Minus: security

Tablet vs PDA Choosing a tablet over a PDA -Tablets have much larger screens, better resolutions, more space for on screen typing making things easier to see and more useful for data entry -Tablets and PDAs have similar battery life, though if PDAs are used for making and receiving calls battery will need more frequent charging -Better security

Hardware requirements for CSPro Tablets: Required configuration: Full Microsoft Windows 7 or 8 (NO windows RT tablets) PDAs: Required configuration: Windows Mobile versions 5 and 6 (note that UNICODE is not supported)

MICS CAPI System All applications to collect and administer data are written in CSPro Three systems: –Interviewers - data collection – Tablet/PDA –Supervisors – data monitoring and control – Tablet/PDA –Central Office - centralized data and monitor fieldwork - PC

Screen Reports Interviewer’s System Supervisor’s System Int 2 Int 5 Supervisor Central Office System System Updates Reports Central Office Repair Utility Update Utility

Training 2 weeks training (paper questionnaires) including 3 days field exercise + 1 week extra training (PDAs/Tablets) days extra field exercise High Level Technical Support

Directories on Interviewer's Tablet \My Documents\MICS5\ DataData from the questionnaires as collected during the interview DictsDictionaries describing the data structures Entry Questionnaire and Interviewer’s menu programs RefReference data files used by the programs Utility Utility programs used for Bluetooth transfer, compressing files, upgrading programs, etc. Work Working files used by the menu and programs \SD Card\ Backup Backup copy of the data files from Data created after the interviews

Directories on Supervisor’s Tablet \My Documents\MICS5\ ReceiveDirectory for data received from the interviewers TempTemporary directory for working files transferred from interviewers, prior to transferring to the Receive directory WorkWorking data files ReportsReport files created by supervisor applications

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files

Secondary Data Processing Exporting data from CSPro –Create SPSS data file and syntax file from CSPRO data file and dictionary Importing data into SPSS –Executing syntax file created by CSPro Recoding variables –Creating new variables and recoding old variables

Secondary Data Processing Adding sample weights –Sample weights are added from weights spreadsheet Adding wealth index –Wealth index calculated then added to files Adding GPS data –Geographic location data added to files Tabulation –Tables are generated from the analysis files

Content of the Presentation Overview of the MICS data processing system Data processing using paper questionnaires Main characteristic of the MICS CAPI system Creating analysis files Data archiving

Data archiving: rationale Collecting data is expensive. Data should be used beyond producing basic report. Survey microdata are valuable resources for government departments and academic researchers. Survey data constitute valuable and irreplaceable assets which should be managed in a way that encourages their widest possible use and re-use. At the same time, data collectors main focus should be protecting respondents while making microdata assessable.