Designing LSMS Questionnaires Kinnon Scott Gero Carletto DECRG

LSMS Questionnaires p Household p Community p Price p Facility

Purposes of LSMS Surveys p Measure Welfare

Welfare p Measure Levels, Distribution, Causes p Various measures  Consumption  Income  Wealth, Savings, Human Capital  Objective measures  Subjective measures p Need: Multi-topic Household Questionnaire Price Questionnaire

Purposes of LSMS Surveys p Measure Welfare p Analyze Policy

Analyze Policy p Who Benefits From Programs p Impact of Programs p Information Need:  Use of Public Services  Who receives subsidies or transfers  Cost of services (fees, distance, time)  Outcomes policies designed to affect p Need: Household Questionnaire

Analyze Policy p Availability of Services to Household p Prices Charged p Quality of Services p Need: Community Questionnaire Facility Questionnaire

Purposes of LSMS Surveys p Measure Welfare p Analyze Policy p Determinants

Determinants p Why observed social outcomes occur p Household behavior p Examples:  Parental Education and child nutrition  Health and Labor Market Status  Risk diversification and Poverty p Need: Household Questionnaire

p Purpose:  Collect data on households  Collect data on all aspects of living standards p Content: seldom the same

Content: Bosnia and Herzegovina p Roster p Housing p Education p Health p Labor p Credit p Vouchers/Certificates p Migration p Social Assistance SECOND ROUND l Food Expenditures and Consumption l Non-Agricultural Household Businesses l Agriculture

Household Questionnaire Level of Observation p Household p Individual  Intra-household allocations  Accuracy, completeness

Respondent Head Individual Best Informed T Roster T Housing n Education n Health n Labor n Vouchers/Certificates n Migration n Social Assistance SECOND ROUND l Food Expenditures and Consumption l Non-Agricultural Household Businesses l Agriculture

Community Questionnaire p Purpose:  Basic characteristics of “community”  Services available  Social Capital  Link to spatial/admin data p Content  Economic conditions  Public and private services:distance, quality  Other relevant: Migration, transport

Price Questionnaire p Purpose  Allow Cost of Living Adjustments to be made  Collect price data p Content  Prices of Main Food and Non-Food Items  Three sources of prices per community

Facility Questionnaire p Purpose  Determine services provided  Assess quality of services  Assess resources available p Content  Inventories (equipment, materials, staff)  Administrative data on services provided  Fees, schedules  Key issues/problems

Starting Point p Discussion with policy makers  What are key policies to measure  What are other key issues

Starting Point p Discussion with policy makers p Discussion with Statistical Office  Issues  Feasibility

Starting Point p Discussion with policy makers p Discussion with Statistical Office p Create Data Users’ Group

Starting Point p Discussion with policy makers p Discussion with Statistical Office p Create Data Users’ Group p Existing Questionnaires  Comparability  categories  variables  reference periods, age groups  Panel  how to track households, individuals

Starting Point p Discussion with policy makers p Discussion with Statistical Office p Create Data Users’ Group p Existing Questionnaires p Qualitative research p “Increasing Policy Relevance of LSMS”  Revision of experience  Identify issues that hhld survey data can address  Draft modules linked to policy issues

Starting Point p Discussion with policy makers p Discussion with Statistical Office p Create Data Users’ Group p Existing Questionnaires p Qualitative/ethnographic work p “Increasing Policy Relevance of LSMS” p Research p Content p Techniques

Designing for Quality p Multi-topic Questionnaire: Complexity  Missing values  Internal Consistencies  Inaccuracies  Omission of key issues by analysts

Quality Control Mechanisms p Explicit Questions

Explicit Questions p Example: 1. Education? p Language 1. What is the highest level and grade of education that [NAME] completed?

Quality Control Mechanisms p Explicit Questions p Pre-Coded

Pre-Coded Questions

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns

Explicit Skip Patterns

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants p Sensitive Issues Last  Fertility  Savings and debt  Income

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants p Sensitive Issues Last p Packaging

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants p Sensitive Issues Last p Packaging p Two-round format

Two Round Format  Breaks up interview  Allows entry and checking of first half data  Allows for corrections with respondents  Reference periods

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants p Sensitive Issues Last p Packaging p Two-round format p Small Sample

Quality Control Mechanisms p Explicit Questions p Pre-coded p Explicit Skip Patterns p Direct Informants p Sensitive Issues Last p Packaging p Two-round format p Small Sample p Training

Pilot or Field Testing of Questionnaires Ensure Questionnaires capable of collecting required information: p Is the full scope of information needed being collected?

Covering Credit?  Roster  Parents of Hhld members  Housing, Utilities  Education  Health  Labor and Other Income  Privatization  Migration  Fertility  Credit  Expenditures / Consumption  Agriculture  Non-Agr. Businesses  Other Income Hhld  Anthropometrics Individual loans: formal-informal, unmet demand Mortgages Durable goods Food Purchases Supplier credit

Pilot or Field Testing of Questionnaires Ensure Questionnaires capable of collecting required information: p Is the full scope of information needed being collected? p Does every section respond to policy needs? p Is the information collected in different sections internally consistent?

Specific Sections p Is the section all inclusive? p Ex. Does the questionnaire include all prevalent  activities  living arrangements  sources of income  consumption items

Individual Questions p Clear in all circumstances? p Lead to unambiguous responses? p Alternative interpretations? p All responses anticipated (pre-coded)? p “Other (specify) ________” p Are skip patterns accurate/complete?

Pilot Test: How to p Involve both field staff and analysts p Test all instruments p Test in all conceivable situations p Ensure enough responses to each section to test well p Document Process

Testing Employment Module p Self-employed farmers p Self-employed businesspersons p Employees p Unpaid family workers p Unemployed p Homemakers p Males, females, children

Testing Housing Module p Homeowners p Renters p Squatters p Multi-family Units

Testing Agricultural Module p Subsistence Farmers p Cash-crop Farmers p Landowners p Hhlds raising Livestock p Absentee owners

Revisions p Leave plenty of time to revise  affects questionnaire  training  data entry p Test new revisions pTake multiple versions to field p Still, don’t forget your whitener!

Examples  Adequate for all situations  Measuring Accurately