2B. Finding and identifying: Sampling

Sampling within an Identified Area Directly drawing a sample of OOS 15 year olds probably not very sensible because a ‘listing’ has to include two further sets of questions: (a) to confirm that the teenager really is 15 (b) to establish the 15 yo’s exact school status These would require presence of 15 year old which cannot be guaranteed Probably better to first identify ALL households reporting a 15 yo in the household Make an appointment to return when 15 yo will be asked to be there

Deriving a Random Probability Sample Divide country into administrative (provincial or regional) units, and usually also into urban and rural areas to create strata Within each stratum, census or enumeration areas (EAs) - or each selected segment of an EA if the EAs are too large - are selected with probability proportional to size, Households within each selected EA or segment of an EA are listed together with the ages of household members; Households reporting a 15 year old are the sample to be interviewed.

Problem with Cluster Sampling Classic sampling designs of DHS/MICS are based on cluster sampling points, the areas selected with probability proportional to size The selected areas are then relisted but the disparity in size do not seem to be taken in account when calculating sampling errors

Rarity of OOS A mock calculation suggests that, if a sample size of OOS 15 year olds is required to be proportional to the typical PISA in-school sample size of 5,000, then we need a target population of 2,100. Given the rarity of 15 year olds and that OOS are about a third of those, then we need an initial listing of 63,000 households. The costs would be prohibitive It helps that PISA requires a minimum of only 150 schools in the sample although that would mean only sampling 14 per point

Options for Reducing Costs Option 1: Starting from Strand A school areas Choosing fewer sampling points. The costs would decrease proportionately but the Design Effects would increase Choosing fewer respondents within each sampling point so decreasing the Design Effect but not substantially reducing the cost

Options for Reducing Costs Option 2: Targeting specific areas Purposefully target specific areas either through local knowledge about rates of enrolment, or with poverty maps, or specific targeting of vulnerable groups – 2A: Direct approaches: Households – 2B: Indirect approaches: Employers, Marginal groups in urban areas

Option 2: Targeting Specific Areas If targeting is adopted, selection of areas should be oriented towards the likelihood of finding OOS 15 year olds Given wide variation in country contexts, it is suggested that country teams include someone who can contribute to defining a poverty map Sample in rural areas

Direct Approaches Rural Areas In small towns, villages, possibly use local informants; if not re-list households a la DHS/ MICS, noting age of each household member Where there are substantial numbers of nomadic/pastoralists, there are recognised procedures for sampling Urban Areas List households in specified areas, possibly with citizen volunteers (ASER and UWEZO) Could be difficult where parents are concerned about security of children In slum areas, could be difficult to list households

Indirect Approach Search for working 15 year olds in formal employment In participating countries, most males report that they are working, whilst much wider variation for females; but percentages working for someone outside household are much lower; and, apart from females living in urban areas in Cambodia and Guatemala, always lower than 20%

Sampling from Employers First compile a list of all businesses working in the area together with their telephone numbers Second, ring up their personnel departments (if they have one) to find out their size and whether or not they have any teenagers working for them Third, if there are a sufficient number of such businesses, divide the reduced list of businesses into small, (medium), and large enterprises and sample from them proportional to size.

Marginal Groups in Urban Areas Non-household populations Methods have been developed to estimate numbers of street children in several places Internally Displaced Persons They will be reluctant to take part in any semi-official exercise

Questions for discussion How to survey out of school 15 year olds? Advantages and disadvantages of option 1 and 2? Are these options mutually exclusive or complementary? Can you think of other options? What are the costs implications? How about synergies with other Strands? What is an appropriate ambition for PISA-D?