WFP/UNHCR Joint Assessment Mission Training Session 3.2. Sampling in the Humanitarian Context TABLE OF RANDOM NUMBERS

WFP/UNHCR Joint Assessment Mission Training statistics…or “statistical inference”… concerned with: generalizations predictions

WFP/UNHCR Joint Assessment Mission Training first, a few key concepts… population: the set of people or things to which our assessment findings are to be generalized sampling frame: the set of people or things from which the sample is to be drawn sample: the subset of the population that we actually investigate or interview or assess

WFP/UNHCR Joint Assessment Mission Training a few more key concepts… randomnessbias all members of a population have an equal chance of being included in sample some members have greater chance of being included in sample than others if the members are “randomly” selected, the result will be closer to the population’s true value if these have higher/lower value than true value, then the result will be higher/ lower than the true value remember: “random” does not mean haphazard

WFP/UNHCR Joint Assessment Mission Training probability sampling…what is it? sampling that uses random selection to choose units to be examined or measured

WFP/UNHCR Joint Assessment Mission Training non-probability sampling… sampling that doesn’t use random selection to choose units to be examined or measured

WFP/UNHCR Joint Assessment Mission Training Probability or Non-Probability Sampling?  rapid assessment methods (key informant/community group interviews/focus group discussions) generally use non-probability sampling  household surveys generally use some form of probability sampling

WFP/UNHCR Joint Assessment Mission Training Why might we use sampling…?  assess who is most affected  determine nutritional status of children < 5 years  monitor changes in food and nutrition over time  assess underlying causes of malnutrition  determine numbers of children eligible for supplementary feeding

WFP/UNHCR Joint Assessment Mission Training simple random sampling  used when complete list of sites/households is available: the sample frame can be defined  it is possible to find, visit any of them (randomly selected from list)  no need to differentiate the different types of sites/households (to ensure all types are represented)  sites/households are randomly selected from list

WFP/UNHCR Joint Assessment Mission Training simple random sampling

WFP/UNHCR Joint Assessment Mission Training systematic random sampling  used when complete list or map of sites/households is available: sampling frame can be defined  it is possible to find and visit any of them  no need interest to differentiate different types of sites/households (to ensure all types are represented)  sampling interval (‘n’) is calculated; every ‘n’th site/household is selected from list or continuous line linking all sites/houses on map  possible in well-organized refugee/IDP camp, or urban neighbourhood

WFP/UNHCR Joint Assessment Mission Training systematic sampling 480 households – want 40 units. 480/40 = 12. Interval = 12

WFP/UNHCR Joint Assessment Mission Training two-stage cluster sampling  used when sampling frame is very large and there is no complete list/map to permit simple or systematic random sampling, or it is not feasible to visit all sites/households (time/resources)  1st stage: sites/households are clustered  2 nd stage: households randomly selected within clusters for interview or measures  30/30 cluster survey (used in most nutrition surveys): population divided into 30 clusters; 30 children weighed/measured in each cluster

WFP/UNHCR Joint Assessment Mission Training two-stage sampling Stage 1: clustering Stage 2: random household selection

WFP/UNHCR Joint Assessment Mission Training stratified sampling  used when complete list is available; possible to find, visit each one if selected from list  sites/households are heterogeneous: have been affected/categorized differently; essential to ensure each is properly represented for more accurate overall estimate  each category is a stratum; sites/households are selected in each stratum proportional to its size

WFP/UNHCR Joint Assessment Mission Training stratified sampling Several groups – Suppose we need to interview blues and whites

WFP/UNHCR Joint Assessment Mission Training stratified sampling

WFP/UNHCR Joint Assessment Mission Training random or biased sample? a survey of child malnutrition is conducted by measuring the children of women who were advised over the radio to bring their under-fives to the health clinic on tuesday morning

WFP/UNHCR Joint Assessment Mission Training biased…  people at the clinic may not constitute random sample of the population at large  only those with radios learn about the survey  only those free on Tuesday can attend  people at the clinic may not constitute random sample of the population at large  only those with radios learn about the survey  only those free on Tuesday can attend

WFP/UNHCR Joint Assessment Mission Training a survey on views about development in a residential area is conducted by knocking on doors of a random sample of homes in that area random or biased sample?

WFP/UNHCR Joint Assessment Mission Training biased …  some will refuse to answer questions – known as non-response bias; even if a truly random sample of people is polled, those who do respond may not constitute a random sample  those who do respond to questions may be those with strong views about development, or those with much time on their hands  some will refuse to answer questions – known as non-response bias; even if a truly random sample of people is polled, those who do respond may not constitute a random sample  those who do respond to questions may be those with strong views about development, or those with much time on their hands

WFP/UNHCR Joint Assessment Mission Training tests to determine the incidence of hepatitis in the population are conducted on a random sample of people waiting in line to see the doctor at a community health centre random or biased sample?

WFP/UNHCR Joint Assessment Mission Training biased… people attending community health centre may not constitute random sample of population  those who suspect they are sick go to clinic  classic example of an opportunity sample--a sample chosen simply because it is easy to obtain people attending community health centre may not constitute random sample of population  those who suspect they are sick go to clinic  classic example of an opportunity sample--a sample chosen simply because it is easy to obtain

WFP/UNHCR Joint Assessment Mission Training a political poll is conducted by calling numbers picked at random selected from a telephone directory random or biased sample?

WFP/UNHCR Joint Assessment Mission Training biased…  those with no telephone will not be included in sample; those with two telephone lines are twice as likely to be in sample as those with one line  those at home not necessarily a random sample of voters, particularly if calls are made during the day when many work  some refuse to answer questions; those who do answer may not constitute a random sample of the population (again: non-response bias)

WFP/UNHCR Joint Assessment Mission Training NGO website invites online visitors to participate in a poll seeking to understand views among humanitarian community on appropriate uses of food assistance in emergencies random or biased sample?

WFP/UNHCR Joint Assessment Mission Training biased…  those who visit NGO home page choose to do so; don't necessarily constitute random sample of humanitarian community at large  those with strong opinions about subject are more likely to participate  example of self-selection bias; if participants in a sample are there because they choose to be, sample is unlikely to be random  those who visit NGO home page choose to do so; don't necessarily constitute random sample of humanitarian community at large  those with strong opinions about subject are more likely to participate  example of self-selection bias; if participants in a sample are there because they choose to be, sample is unlikely to be random

WFP/UNHCR Joint Assessment Mission Training but… in an emergency? probability sampling concerns pre-existing sampling frame rare; population size likely unknown few easily identifiable clusters within population; their size likely unknown pre-existing info about means, standard deviations of key variables likely unknown households may be broken; HH concept may need to be modified

WFP/UNHCR Joint Assessment Mission Training so…why bother with probability sampling? minimizes possibility of assessor drawing biased sample - consciously or unconsciously assessment findings can be generalized to the population (not so with non-probability sampling)

WFP/UNHCR Joint Assessment Mission Training TABLE OF RANDOM NUMBERS

WFP/UNHCR Joint Assessment Mission Training South Town Kelenni Filani Sabani Nanani Duuruni Wooroni Woluni Seguini Tanba Tanni Duguni Nyakelen Nyafila Nyoduru Nyasaba Nyasani Malanyi Kanyi Nyikulu Nyokoko Nyadaba Daba Malama Manyi Kabano Kamani Maba Kundugu Masadugu Masaba Masani Sama Samani Kono Wuluni Dioro Nyodioro Bamani Jiri Jakuma Tigui Tan Jama Tese Amana Jugu Fato Kini Malo Bolo Kalan Juguba Dabani Badaba Kelenba Gono Gononi 50 Kms. Saba Togoni Fatoni Fatoba Baji Seguiba Kabadugu Kununi Konodugu Kononi Wulu Wuludugu Wuluba

WFP/UNHCR Joint Assessment Mission Training Random Site Selection Only visit villages > 25 kms from main road Stratify population for livelihood groups: 1.Highland farm villages 2.Lowland farm villages 3.Fishing villages Visit six villages each group

WFP/UNHCR Joint Assessment Mission Training South Town Kelenni Filani Sabani Nanani Duuruni Wooroni Woluni Seguini Tanba Tanni Duguni Nyakelen Nyafila Nyoduru Nyasaba Nyasani Malanyi Kanyi Nyikulu Nyokoko Nyadaba Daba Malama Manyi Kabano Kamani Maba Kundugu Masadugu Masaba Masani Sama Samani Kono Wuluni Dioro Nyodioro Bamani Jiri Jakuma Tigui Tan Jama Tese Amana Jugu Fato Kini Malo Bolo Kalan Juguba Dabani Badaba Kelenba Gono Gononi 50 Kms. Saba Togoni Fatoni Fatoba Baji Seguiba Kabadugu Kununi Konodugu Kononi Wulu Wuludugu Wuluba

WFP/UNHCR Joint Assessment Mission Training Fishing Villages Baji Duguni Duuruni Filani Tanba Nanani Nyakelen Saba Sabani Seguiba Seguini Tanni Woluni We decide to randomly select four villages to visit

WFP/UNHCR Joint Assessment Mission Training Fishing Villages 01Baji 02Duguni 03Duuruni 04Filani 05Tanba 06Nanani 07Nyakelen 08Saba 09Sabani 10Seguiba 11Seguini 12Tanni 13Woluni We consecutively number the villages… …and then use the random numbers table to identify the four villages to visit.

WFP/UNHCR Joint Assessment Mission Training TABLE OF RANDOM NUMBERS

WFP/UNHCR Joint Assessment Mission Training Fishing Villages 01Baji 02Duguni 03Duuruni 04Filani 05Tanba 06Nanani 07Nyakelen 08Saba 09Sabani 10Seguiba 11Seguini 12Tanni 13Woluni

WFP/UNHCR Joint Assessment Mission Training Using a Random Number Table If total sites < 10, then use first digit of table numbers If total sites < 100, then use first 2 digits of table numbers If total sites < 1000, then use first 3 digits of table numbers Close eyes, place fingertip on table to start Use pattern to move through table (left to right through rows or top to bottom

WFP/UNHCR Joint Assessment Mission Training

WFP/UNHCR Joint Assessment Mission Training Three types of bias What is the potential impact? Non-random selection Team biases Operating environment

WFP/UNHCR Joint Assessment Mission Training How do we minimize bias?  Coordination / joint or inter-agency assessment  Triangulation: varying techniques, indicators of the same event, key informants, info sources, team composition  Random selection

WFP/UNHCR Joint Assessment Mission Training in conclusion… in crisis, you may not have time, resources for in-depth efsa rapid assessment can strive to:  improve results through random site/household selection  minimise bias through triangulation, cross-checking