Module 10: Data Sources and Metadata Tools for Civil Society to Understand and Use Development Data: Improving MDG Policymaking and Monitoring

What you will be able to do by the end of this module Identify and use effectively the different types of data used to produce indicatorsIdentify and use effectively the different types of data used to produce indicators Understand the concept of metadata and the role it plays in using indicatorsUnderstand the concept of metadata and the role it plays in using indicators

Sources of Data Administrative or routine dataAdministrative or routine data Census dataCensus data Survey dataSurvey data Surveillance dataSurveillance data

Administrative (or Routine) Data Primary purpose is management of processesPrimary purpose is management of processes Summary and/or dissemination occurs later (but usually within one or two years)Summary and/or dissemination occurs later (but usually within one or two years) Example 1.Registration of birth –Record birth a child, to be used later for purposes of child health, education, etc. 2.Immunization against measles –Purpose is to manage the national immunization programme

Administrative Sources Vital registration (births, deaths, etc)Vital registration (births, deaths, etc) Moldova: the civil registration offices (birth, death, marriages and divorces). Information on migration is received from the Ministry of Information Development and the National Bureau for Migration Belarus: the data on vital events comes from the acts of civil status registered by the Offices of Civilian Registration (ZAGS), and data on migration of population is collected from the Ministry of Internal Affairs Health systems (immunization rates, mortality rates, maternal health data, etc.)Health systems (immunization rates, mortality rates, maternal health data, etc.)

Administrative Sources and Statistical authorities Education (Enrollment and completion data)Education (Enrollment and completion data) Belarus: NSC collects annual information on pre-school, higher, and special secondary education and the Ministry of Education –general secondary education and out-of- school education, on educational establishments for children with psychophysical traits, vocational technical education, children's homes, and tutelage authorities. student-teacher ratio, etc.) Moldova: For primary and secondary education, NBS collects information from private schools only. Information about other schools is received from the Ministry of Education; data are processed by the NBS. For other types of education, the data are collected by NBS itself, from colleges, professional schools and universities

Administrative Sources and Statistical authorities Employment (numbers employed, industry, level of participation)Employment (numbers employed, industry, level of participation) Moldova: NBS of Moldova is responsible for Labour Force Survey Belarus: There is no Labour Force Survey Business (Industry, sector, size)Business (Industry, sector, size) Moldova: NBS of Moldova is responsible for business register Belarus: NSC is responsible for keeping of a business register. Information comes from the registration authorities responsible for state registration/ cessation Belarus: NSC is responsible for keeping of a business register. Information comes from the registration authorities responsible for state registration/ cessation

Censuses Collect data from every unit in the populationCollect data from every unit in the population 100% coverage (in theory)100% coverage (in theory) ExpensiveExpensive Time consumingTime consuming

Population Census Identify each member of the populationIdentify each member of the population Collect certain basic data about themCollect certain basic data about them –age, gender, location, etc. (Moldova 2004, Belarus 2009) Modules to collect data on specific topics may be addedModules to collect data on specific topics may be added Normally about every 10 yearsNormally about every 10 years Modeling methods used to generate population estimates between censusesModeling methods used to generate population estimates between censuses Good for small-area poverty mappingGood for small-area poverty mapping

Population Census Data AdvantageAdvantage –Excellent coverage –Creates sampling frame for household surveys DisadvantageDisadvantage –Potential for some bias – for example, could miss nomadic groups or homeless –May be inaccurate due to infrequency –Limited data collected –Lag before data produced

Establishments Censuses Censuses of businesses, hospitals, other organizations Provide a frame for later surveysProvide a frame for later surveys Collect basic data, as for population censusCollect basic data, as for population census Problems:Problems: –Smaller or informal establishments often excluded –Establishments may change more frequently than households

Sample Surveys Vehicle for collecting data from a subset of the populationVehicle for collecting data from a subset of the population Advantages of focusing on a subset:Advantages of focusing on a subset: –Save money and resources –Reduce time to collect data –Reduce time to analyse data

Characteristics of Sample Surveys Primary purpose is to get dataPrimary purpose is to get data Some units will be omittedSome units will be omitted Well designed survey should be representative of the populationWell designed survey should be representative of the population There needs to be reliable sampling frame (e.g. generated from census data)There needs to be reliable sampling frame (e.g. generated from census data)

Sample Survey Data AdvantageAdvantage –Cheaper and quicker than census –More frequent than census, though usually only every 1 to 3 years –Can collect wider range of data than census and administrative systems –Reduced potential for bias than in census and administrative data DisadvantageDisadvantage –Sampling error since coverage < 100% –Requires more sophisticated design to ensure consistency and accuracy

Household Surveys Reporting usually takes about 1 year after completion of data collectionReporting usually takes about 1 year after completion of data collection Focus is on socio-economic and health issuesFocus is on socio-economic and health issues Governments should develop inter-census survey programsGovernments should develop inter-census survey programs

Household Surveys (2) Multi Indicator Cluster Surveys (MICS)Multi Indicator Cluster Surveys (MICS) Demographic and Health Surveys (DHS)Demographic and Health Surveys (DHS) Living Standards Measurement Study Surveys (LSMS)Living Standards Measurement Study Surveys (LSMS) World Health Surveys (WHS)World Health Surveys (WHS) Core Welfare Indicators Questionnaires (CWIQ)Core Welfare Indicators Questionnaires (CWIQ)

Household Surveys (3) Household budget surveysHousehold budget surveys Moldova: NBS conducts Household Budget Survey (HBS), the sample size is households, half of which remain in the sample for 4 years, while the other half is replaced after two years. Response rate is about 70% Household income and expenditure surveysHousehold income and expenditure surveys Belarus: Minstat conducts the Household Income and Expenditure Survey (HIES), which is held annually. The sample consists of 6,000 households. The households participated at the survey are rotated every year. Response rate is 91%

Demographic Surveillance Systems (DSS) Longitudinal monitoring of sentinel populations (60,000 to 100,000)Longitudinal monitoring of sentinel populations (60,000 to 100,000) –Follow same people every year through life of survey 100% event registration100% event registration 40 countries40 countries

Demographic Surveillance Systems (DSS) (2) AdvantagesAdvantages –Coverage of sentinel pop = 100% –Rapid data availability –Facilitates targeting and short term monitoring DisadvantagesDisadvantages –Few large clusters – potential for serious bias –Expensive to include many clusters

Data Sources Compared CharacteristicAdminSurveyDSSCensus Inclusion criterion All ‘noticed’ events Designated units All events in clusters All units Coverage Variable, depending upon system % coverage specified Coverage of clusters only ~100% coverage Bias May be biased Designed to minimize bias Urban/rural included, but may not be sufficiently representative Lack of coverage may lead to some bias

Data Sources Compared (2) CharacteristicAdminSurveyDSSCensus CostCheapMediumMediumExpensive Time Ongoing, +1-2 years for reporting 3-5 years + 1 year for reporting Ongoing, report in < 1 year 10 years + 2 years for reporting Potential for Policy/advocacytargeting V good, but limited info, and problem if coverage poor Good, but only for medium to long term V good, but only for clusters and similar populations Good for long term and as input with other data

Synergy Across Data Sources Potential to use different types of data together toPotential to use different types of data together to –Build a wider picture –Provide a means for cross-checking Population estimates from censuses needed to supplement other sourcesPopulation estimates from censuses needed to supplement other sources Need to ensure compatibility ofNeed to ensure compatibility of –Definitions –Time frames

Metadata Clearly vital to know, when using dataClearly vital to know, when using data –How they were collected –When they were collected –By whom –Potential for bias –How data items are defined –Methodology The whole collection of this type of data is called the metadataThe whole collection of this type of data is called the metadata

Major Purposes of Metadata 1.Confirm definitions 2.Facilitate decisions about compatibility of data from different sources 3.Guides and validates the interpretation of data and their indicators

Summary In this module we have discussed The main sources of data used in the production of indicatorsThe main sources of data used in the production of indicators The definition and importance of metadataThe definition and importance of metadata

Which of the surveys discussed in this module are carried out in your country? How frequently?Which of the surveys discussed in this module are carried out in your country? How frequently? Summarize the progress towards Goal 3, using the MDG report. Include in your assessment a discussion of the limitations of the data which contribute to the relevant indicators. Use the metadata to inform this discussionSummarize the progress towards Goal 3, using the MDG report. Include in your assessment a discussion of the limitations of the data which contribute to the relevant indicators. Use the metadata to inform this discussion Practical 10