1 Surveys in Humanitarian Emergencies

2 Methods of Data Collection AssessmentSurveySurveillance Objective Rapid appraisal Medium-term appraisal Continuous appraisal Data Type Qualitative/ Cross sectional snapshot Quantitative/ Cross sectional snapshot Quantitative/ Longitudinal trends Method Observational / Secondary source Sample with survey instrument Periodic, standardized data collection

3 Overview  When to do a survey  The need for a standardized tool and methodology  Key activities in designing and implementing health and nutrition surveys  Survey interpretation

4 Surveys in Emergencies Why would you consider doing a survey? When would you do a survey?

5 Why Do a Survey in an Emergency  Estimate prevalence (example acute malnutrition)  Determine mortality  Prioritize interventions  Collect baseline data for program planning  Evaluate program success  Advocate for intervention

6 When To Do a Survey in an Emergency?  After the initial emergency phase has passed  After basic needs for survival been met  When major population movement has stopped  When you are in a position to take action  After preparatory information has been collected  When security allows

7 Difference Between Emergency and Development Surveys  Emergency surveys Must be done with action in mind Must be implemented and analyzed quickly Hard to find trained/trainable personnel Difficult to come up with sampling frame Multi sectoral, multi partner (UNICEF, UNFP, etc.) Security

8 The Problem: Non-standardization of Methods Ethiopia Among 67 nutrition and mortality surveys, only 6 (9%) met all 5 eligibility criteria to be valid and precise*  Probabilistic sampling  Sample size > 500 children aged 6-59 months  25 clusters  20 or more children per cluster  Weight for height used as anthropometric index Source: Spiegel P et al, JAMA 2004

9 Survey design and implementation: key tasks  Determine objectives of the survey  Determine broad questions to be answered outcomes to be measured  Estimate Budget needed and Identify key partners  Define the sampling frame & sampling design  Design the questionnaire, translate, and test  Conduct training, plan the logistics, equipment, and survey team needed  Data entry, analysis, and Interpretation of results  Preparation and dissemination of results  Take action

10 What Might Be Included in an Emergency Survey?  Demographics  Household Characteristics  Child Health  Child Nutrition  Reproductive health  Mortality  Food Sources and Consumption  Food Aid deliveries & usage  Health interventions  Water sources  Sanitation

11 Defining The Sampling Frame  What population do you want to extrapolate your results to?  What population is or will be the target of the program?  Is the population homogeneous?  Are there geography and access issues?  Security conditions?

12 Sampling Frame for the Darfur Survey

13 Sampling Frame for the Ethiopia Survey  Population not homogeneous Highlands versus lowlands  However, intervention targeted by Woreda

14 Decisions Concerning Sampling Design Probability sampling:  Simple random sampling  Systematic random sampling  Cluster sampling Non-probability sampling:  Key informants  Convenience sampling  Purposive sampling Sampling methodology: types

15  Each individual or sampling unit in the population has the same chance or probability of being selected  The selection of one individual should be independent of the selection of another Representative Sampling

16 Simple Random Sampling  Most basic type of sampling  Selection of units independent and random  Steps: Number each sampling unit Choose new random number for each selection

17 Systematic Random Sampling  Similar to simple random sampling  First sampling unit chosen randomly  Systematic selection of subsequent sampling units  Steps: Compute sampling interval (SI) (Number in population / Sample size) Select random start between 1 and SI

18 Simple and systematic random sampling What is required for both simple and systematic random sampling? Both require a complete list of all basic sampling units.

19 Calculating the sample size For A Random Sample or Systematic Sample  The sample size is calculated using the following formula: n = { t 2 x p x q } d 2 Where:n = sample size t = the risk of error (1.96 or 5% error) p = expected prevalence (fraction of 1) q = 1- p (expected non-prevalence) d = level of precision (fraction of 1) Sample size should by multiplied by design effect in cluster surveys

20 An example of calculating the sample size  The sample size is calculated using the following formula: 81 = { x 0.3 x 0.7 } Where:n = sample size t = the risk of error (1.96 or 5% error) p =expected prevalence (fraction of 1) q = 1- p (expected non-prevalence) d = level of precision (fraction of 1)

21 Cluster sampling Objective: To choose smaller geographic areas in which simple or systematic random sampling can be done

22 Simple Random Sampling Simple random Sampling: (300 households required) Sampling universe Selected households Non-selected households

23 Cluster sampling Cluster sampling : 30 Clusters of 10 households each But, because we are using cluster sampling, we must increase the sample size because we are not choosing randomly, but households are clustered together. May end up choosing 30 clusters of 15 households.

24 The sample size is calculated using the following formula: n = { t 2 x p x q } x design effect d 2 Calculating the sample size in cluster surveys

25 Which sampling method here?

26 Which sampling method here?

27 Common Problems With Sampling For surveys: - Was a random or probability sample taken using a recognized method? –Did everyone have an known probability of selection?  Who was left out? (selection bias) –What was the geographical coverage? Does it relate to catchment area of program? –Were case definitions clear and standardized and piloted or validated? –How was age determined? –What was the sample size? –Are confidence limits reported? –Watch out for denominator of rates

28 Sampling - Goal Regardless of the sampling method, if the sample is not representative, you can’t generalize the findings to the whole population. Remember this point!

29 Sampling – Sample size So how do we decide on the needed precision?  One-time results for advocacy alone does not need much precision (  0.10 good enough)  Results that you will need to compare against in the future need greater precision (  0.05 if program will have large impact)  Results you will monitor frequently (e.g. year by year) need even greater precision (  0.02)

30 Adjustments to Sample Size  Source of information about number of analysis units (e.g. adult males) per samping unit (household) Calculate from census statistics Use previous surveys to calculate  Finally, need to add margin for non-response Look at previous surveys May differ by region of country May be higher for some measures (e.g. blood)

31 Training & data analysis  Training needs to include: what data to collect, how to select households, data collection method, consent form, interview method  Data entry: advise to begin data entry simultaneously as data collection  Data analysis: select the analysis tool (Epi-info, SPSS), identify key personnel

32 Interpreting Results Measles coverage:  Measles immunization coverage 59.4% [CI: ] Major morbidity in last two weeks:  Watery diarrhea 17.2% [CI: ] Mortality rates: (over 7 month period)  Crude mortality rate: 3.2/10,000/day  <5 mortality rate: 9.8/10,000/day

33 Interpretation: Other Factors  Trends and change  Confidence Intervals (CI)  Seasonality  Aggravating factors or risks  Baseline or ‘normal’ prevalence  Prevalence of other types of malnutrition e.g. chronic malnutrition  Mortality levels

34 Mortality Trends in Selected Refugee Camps

35 Assessing the Quality Of Survey Implementation  Did the survey answer the question it was designed to?  Were objectives clear measurable and measured?  Was the sampling frame adequate?  Questionnaire Clear? In local language? Translated, back translated?  Sampling design Representative? Sample size adequate?

36 Assessing the Quality Of Survey Implementation  Logistics, equipment, and survey team Equipment standardized? Training adequate? Personnel?  What checks for data quality have been done?  Do you agree with the interpretation of results?  Have results been disseminated to all partners  What action is planned?

37 One Way to Evaluate Methods CharacteristicAssessmentPoints Sampling frameUnadjusted Adjusted 1212 Sampling methodConvenience Random 1212 Sample sizeSmall1 Good (justified)2 Geographical coverageSmall Appropriate 1212 Case definitionsNot defined Standardized 1212 Confidence limitsNot reported1 Appropriate2

38 Standard deviation of sample = 1.75 Assessing The Quality of the Survey

39 Take Action  Interpret and understand findings  Review and revise program objectives  Advocate for resources e.g. food pipeline or access to clean water  Address underlying causes of poor health or nutrition  Increase coverage of programs Vaccination  Close emergency nutrition programs  Use findings as part of wider country information system  Use findings as baseline data