1. Assessing Information Needs and Survey Alternatives
Kathleen Beegle, DECRG
Poverty and Inequality Course
Module 1: Multi-topic Household Surveys
January 23, 2008

2. The Demand for Data
Performance-based management has created pressures on developing countries to improve the quantity and quality of their macro and micro-data:
Is the public sector delivering good services?
Are country policies/poverty reduction strategies reducing poverty?
Is aid supporting poverty reduction?
In the World Bank: “Results-based” CASs, for example

3. The Demand for Data Performance-based management
Millennium Development Goals (MDGs) reflects the strong commitment on the part of the international development community and national governments to monitor and evaluate the results of their policies and programs (United Nations, 2000).

4. The Demand for Data HIPC, Poverty Reduction Strategies (PRSPs)
Performance-based management
MDGs
HIPC, Poverty Reduction Strategies (PRSPs)
The HPIC (debt reduction initiative for heavily-indebted, low-income countries) requires
a plan for poverty reduction
that can be measured and monitored
access to concessional lending from the World Bank and International Monetary Fund (IMF/IDA, 1999a and 1999b).

5. The Demand for Data
Much of the increased demand for data has been focused on household-level data.
It is not possible to monitor PRSPs or MDGS without solid household data
Measurement of welfare and other key social indicators. (Muñoz and Scott, 2005).
Decisions on appropriate policies to reach the MDGs or the targets in the PRSPs need household data.
Although they do not exclusively need/require only household-level data, although they often imply survey data

6. What these demands look like
PRSP
Measure welfare/poverty
Identify problems--magnitude, causes
Alternative policies
Cost/benefit
Monitor
Evaluate

7. What these demands look like
PRSP
MDGs
MDG 1: Eradicate extreme poverty and hunger
MDG 2: Achieve universal primary education
MDG 3: Promote gender awareness, empower women
MDG 4: Reduce child mortality
MDG 5: Improve maternal health
MDG 6: Combat HIV/AIDS, malaria and others
MDG 7: Ensure environmental sustainability
MDG 8: Develop a global partnership for development

8. What these demands look like
PRSP
MDGs
General Demand
Poverty and Inequality
Benefit Incidence Analysis
Public services
Determinants of observed outcomes
Assessment of alternative policies
Impact Evaluation
Inputs to Program Design

9. Recent Efforts to Increase Data: Quantity and Quality
Partnership in Statistics for Development in the 21st Century (PARIS21) was initiated to “…to act as a catalyst for promoting a culture of evidence-based policymaking and monitoring in all countries, and especially in developing countries.” (PARIS21 web site). 1999
Trust Fund for Statistical Capacity Building (TFSCB), managed by the World Bank, set up in 2000 to help build the capacity of statistical systems/ statistical plans

10. Recent Efforts to Increase Data: Quantity and Quality
World Bank line of credit for Statistical Capacity Building to support countries in the implementation of these statistical master plans.
The Program to Improve Surveys of Living Conditions in Latin America (MECOVI), (co-sponsored by the IADB, ECLAC, WB, regional program to improve quality and data
SPARC- Eastern Carribean Initative to improve surveys, UNDP, CDB, WB IADB, OECS, inter alia. (2004)
LSMS Phase IV: Methodological Research on measurement, field work and technological advances

11. Focus of Presentation Assessing Information Needs Sources of data
Need for multiple sources of data
Role of household surveys

12. Assessing Information Needs
Inputs 
Financial & physical indicators of inputs (monthly)
Internal 
Outputs
Achievement/performance indicators (annually)
Outcomes
Benefits/usage indicators (annually)
External
Impact
Indicators of improvements in living standards (~ 5 years)

13. Data Sources National accounts Current public expenditure statistics
Program of Price collection (cons./prod.)
Administrative Records (from line ministries)
Qualitative Work
Surveys:
Multi-topic w/ welfare focus (LSMS/IS)
Monitoring (CWIQ, PS)
Income and Expenditure (IES, HBS)
Single topic (Labor Force Surveys (LFS)
Demographic and Health (DHS))
Enterprise
Facilities

14. Surveys and Policy Analysis
Gov’t Programs
Social Goals
Conditional Cash Transfers
Day care centers
Public Health Campaign
Increase enrollment
Increase female LFP
Lower infant mortality
Households
Individuals
Firms

15. Surveys: Going Beyond Rates Understanding secondary school enrollments, year olds, Albania 2002
In almost all countries we have a single statistic: mean enrollment at the national level. In this case it is 61%.
This is interesting for monitoring purposes, but it doesn’t say much about poverty or other factors.
... A regional disaggregation would be useful
Average
Percent

16. In some countries we have regional breakdowns, with marked contrasts
Understanding secondary school enrollments, year olds, Albania 2002
In some countries we have regional breakdowns, with marked contrasts
The contrast between urban and rural rates emphasizes the disadvantages faced by rural communities.
Other breakdowns would be useful
Urban
Average
Percent
Rural

17. …possibly, official statistics can add the gender dimension
Understanding secondary school enrollments, year olds, Albania 2002
…possibly, official statistics can add the gender dimension
…the figures show that, in urban areas, there is no gender differential but a large gap in rural areas.
But we still don’t know much about who sends their children to school
Male
Urban
Female
Average
Percent
Male
Rural
Female

18. Understanding secondary school enrollments, 12-18 year olds, Albania 2002
Female, urban
Male, urban
Male, rural
Female, rural
Average
Percent
…With a survey we can show enrollment rates broken down by consumption level--and thus understand an additional dimension
Consumption quintile

19. Gathering information through surveys (household or other)
There is a range of options
They can be ordered along two main dimensions:
degree of representativeness
subjective/objective dimension
These are not stark, either-or choices.

20. Degree of Representativeness
Case
study
Purposive
selection
Quota
sampling
Small prob.
sample
Large prob.
sample
Census
Let’s start by looking at the degree of representativeness of data collection tools.

21. Subjective/Objective Dimension
Direct measurement
Questionnaire
(quantitative)
(Qualitative)
Structured
interview
Open
meetings
Subjective
assessments
Conversations
Case
study
Purposive
selection
Quota
sampling
Small prob.
sample
Large prob.
Census
Then look at the subjective/objective dimension

22. Wonderful World of Surveys
Direct measurement
Questionnaire
(quantitative)
(Qualitative)
Structured
interview
Open
meetings
Subjective
assessments
Conversations
Case
study
Purposive
selection
Quota
sampling
Small prob.
sample
Large prob.
Census
Census
Household Budget
Survey
LSMS/ IS
CWIQ/PS
Participatory Poverty
Assessments
Sentinel Site
Surveillance
Participant
observation
Beneficiary
Assessment
Then look at the subjective/objective dimension
Community
Surveys
Windscreen
Survey

23. Wonderful World of Surveys: “Statistical Surveys”
Direct measurement
Questionnaire
(quantitative)
(Qualitative)
Structured
interview
Open
meetings
Subjective
assessments
Conversations
Case
study
Purposive
selection
Quota
sampling
Small prob.
sample
Large prob.
Census
Census
Household Budget
Survey
LSMS/ IS
CWIQ/PS
Focus on the upper right hand quandrant, quantitative surveys with probability samples

24. Features of a Statistical Survey
Structured Questionnaire
Random/Probability Sample

25. Tradeoffs to Consider When Planning a Survey as Part of a System of Surveys
Overall scope
Single vs. Multi-topic
Probability vs. Purposive Sampling
Sampling vs. Non-Sampling Errors
Time vs. Cost
Data vs. Capacity Building
Surveys over time: repeated cross sections, panels, rotating

26. Summary
Surveys are one source of information among many (system of information)
Consider all the key elements of a National Poverty Monitoring System

27. Key Elements of a National Poverty Monitoring System
Timely Annual National Accounts
Current Public Expenditure Statistics
Program of Consumer and Producer Price Statistics
In-depth Welfare Survey (LSMS/IS, IES?)
‘Light’ Annual Monitoring Surveys (CWIQ, PS)
Longitudinal Studies
Qualitative work on key topics
Specific tools for project/program/policy monitoring and evaluation

28. Summary
Surveys are one source of information among many (system of information)
No one survey can meet all data needs: System of Household Surveys

29. System of Household Surveys
Goal: System able to respond to evolving needs: not produce data X or survey Y
Determine data needs before they are URGENT
Identify appropriate instruments,
Implement them properly, timely fashion,
Analyze the resulting data

30. Improving the SHS Linking Users and Producers
Providing adequate resources
Continuous Survey Program
Not necessarily permanent survey
Benefits
Avoid loss of capacity
Create greater levels of capacity (building on existing)
Economies of scale
Policy makers know when data will be available
Protects NSO from pressures for ad hoc surveys
Ongoing system actually allows more flexibility and responsiveness

31. Summary Expanding demand for timely, relevant data
Need to determine the range of data needs to begin to define a system of information
Surveys are one, important, source of information among many
No one survey can meet all data needs: System of Household Surveys

32. References
United Nations (2000). “United Nations Millennium Declaration.” United Nations’ General Assembly, Fifty-fifth Session, New York, New York.
International Monetary Fund and International Development Association (1999a). “Building Poverty Reduction Strategies in Developing Countries.” Report to the Board of Directors, International Monetary Fund and International Development Association, Washington, D.C.
International Monetary Fund and International Development Association (1999b). “Heavily Indebted Poor Countries (HIPC) Initiative: Strengthening the Link between Debt Relief and Poverty Reduction.” International Monetary Fund and International Development Association, Washington, D.C.
Muñoz, Juan and Kinnon Scott (2005). “Household Surveys and the Millennium Development Goals.” Paris21, processed.

33. References
DECRG, (2006) “LSMS IV: Research for Improving Survey Data”, processed
LSMS Web page:
ISLC/ MECOVI web site:
then search on MECOVI