MICS Data Processing Workshop Multiple Indicator Cluster Surveys Data Processing Workshop Data Archiving

Look familiar? Todays storage medium is no longer file cabinets and storage rooms...

What is archiving and why do it?

Data ArchivingWhat is it? The long-term storage of data (raw data) and its affiliated documentation. Storage of metadata - the descriptive information about the survey and the data; essentially, data about other data.

The MICS Archiving Goal! Accessible datasets that are free of errorsno spurious or out-of-range values etc. Datasets that are well-documented – all variables and response codes labeled etc. Information available on the survey methodology that explains to external data users how the survey was designed and conducted. The final survey reports and materials used to disseminate results.

Group Work Survey archiving brings huge benefits to both data users and data producers. In 3 groups discuss the following: Group 1: What are the benefits to data users? Group 2: What are the benefits to data producers? Group 3: What should be included in a MICS survey archive?

Archiving Benefits for Data Users Facilitates the usage of datawithout proper documentation on the survey methodology and analysis, data cannot be fully understood which may lead to lack or inappropriate use of data; Proper documentation decreases the time a user needs to spend understanding the data and requesting information from the agency; Increases efficiency for usersless time can be spent producing pre-determined tables, more time can be devoted to higher-level analysis.

Archiving Benefits for Data Producers I ncreases visibility and credibility for the agency; Allows for easier management of the data and survey related materials; Allows for easy and efficient sharing of information with external audience; Prevents loss of all survey related documents by individuals and facilitates institutional memory.

MICS Documents to Archive Survey plan and budget Questionnaires Pre-test report Interviewer & field manuals Sample selection, listing information –Excel files, other supporting documents –Explanations of any event(s) that impacted the sample selection, survey dates, etc Steering/technical committee ToRs and minutes Any other documentation that will help people understand what happened in the design and implementation of the survey

MICS Programmes to Archive Software Programmes & Files: CSPro dictionaries CSPro data entry & menu systems CSPro secondary editing & menu systems CSPro GPS Entry program CSPro export & SPSS recode SPSS tabulation programmes Any country-specific programmes

MICS Datasets to Archive SPSS Files: Household (HH) Household listing (HL) Insecticide-Treated Nets (TN, if included in survey) Women (WM) Birth History (BH if included in survey) FGM (FG, if included in survey) Maternal Mortality (MM, if included in survey) Children age 0-4 years (CH) Men (MN, if included in survey) CSPro Files: Final (merged, not individual clusters) HH data file GPS data (if included in survey)

MICS Reports and Dissemination Materials to Archive Key Findings Report Final Report Presentations Factsheets Press releases Videos And everything else!

When to Archive? Archiving is often the last step in data collection activities As a result, documentation is often of uneven or poor quality Start archiving when you start your MICS survey

Archives for internal and external audiences If some survey information is considered sensitive or for internal use only then at the end of the survey process it is advisable to create two separate versions of the MICS archive. Restricted MICS Archive (complete archive but only accessible internally) Public MICS Archive (with some documents and information deleted)

What does a survey archive look like? MICS Final Archive Examples…

How to Archive?

Preparing for archiving Save as much as possible electronically –Electronic documentation has a longer life and is more readily usable –Paper documentation requires permanent storage areas –Paper documentation disappears! Save documents –in original format –in HTML –as PDF files

Preparing for archiving Save datasets –in original format (e.g., CSPro raw+final files) –in analysis file format (e.g., SPSS.sav file) Always include descriptions with data provide labels within each file for: Variable labels (e.g., Result of Interview) Value labels (e.g., 1=Complete, 2=Not at home) Dont forget to document country-specific questions!

Creating the archive? Recommend to use an archiving tool: International Household Survey Networks (IHSN) Microdata Management Toolkit

International Household Survey Network (IHSN) Established September 2004 as a recommendation of the Marrakech Action Plan for Statistics (MAPS) Users: International organizations actively sponsoring household surveys + local NSOs Governing committee members from DFID, ILO, Paris 21/OECD, UNICEF, UNSD, WHO, World Bank Distributes Microdata Management Toolkit and National Data Archive (NADA)

IHSN Toolkit 4 Main Components Archive creation: Metadata Editor (also known as the Nesstar Publisher) –Document your survey data, documentation, and metadata in accordance with international metadata standards Dissemination: CD-ROM Builder –Facilitates the publication of survey data, documentation, and metadata onto CD-ROMs and the Internet using HTML format Nesstar Explorer –Using their free program, access metadata and data, and export to common statistical formats IHSN Report Center –Provides diagnostics on your metadata, dataset relations, external resources, etc., and creates a PDF of your study documentation (a summary report of all your documentation)

Step 1: The Metadata Editor All data and documents are first archived in this software. Template-driven specialized editor Imports and exports data to the most common statistical formats Integrated interface, multi-lingual support Compiles metadata and data in single file /ddi-metadata-editor

Installation Unzip IHSN_Toolkit_v1.1_full.zip into C:\MICS5\IHSN Double-click on the file IHSN Toolkit Installer.exe Select the first option: Metadata Editor

Getting Started Create a new study (ctrl+N) Save your project (ctrl+S)this gives your project a name within the tree Select the IHSN templates (MICS5 generic template) Begin by customizing the fields under the: –Document description –Study description Start saving key survey documents and files under the folders in: –External resources

Importing Datasets Once the datasets are final make sure all variables and values are properly coded and that the data has been anonymised in SPSS first. Delete the current datasets in (generic) archive and import the final datasets. Make sure that all variables are properly documented with manual/interviewer instructions and questions from questionnaire. – (This will be automatic for all standard MICS variables if using the MICS template but will not be done for any non-MICS questions and modules)

Tips and Good Practices One person – ideally the survey manager – should be the main custodian of the survey archive and responsible for archiving all documents throughout the survey implementation. The most time intensive part of archiving is documenting all the variables once the final data has been imported. It is good practice to keep a complete record of all the changes from the standard questionnaires (changes to questions, interviewer instructions, new questions etc.) so it will be easy to go through the archive and make changes from the generic and add the information on variables that will not be there.

How to generating a final survey archive for internal and public use?

Step 2: The CD-ROM Builder Publishes survey metadata, data, and documents on a CD- ROM or website Stores archive in an HTML-based language Users can customize the layout (branding) and content of the CD (single or multi-surveys) Integrates with IHSNs Metadata Editor

Generating the Final Archive(s) Only once all information is complete in the metadata editor should the CD-ROM builder be used. –Cannot make changes to the archive once generated by the CD ROM Builder. If problems are found at this stage, need to go back to metadata editor and make corrections there. If different versions of a survey archive (public and restricted) will be generated, two projects need to be created in the metadata editor tree. To make a public archive delete all the documents that are for internal use only and give this archive a new name. Only need to do this at the end of the survey archiving process before using the CD ROM Builder.