2C/D Approach to Sampling/ Surveying Administering the assessment/survey

UNICEF/ UIS Initiative 25 countries (including Brazil, Cambodia and Zambia) engaged in initiative Nearly all country reports use administrative data and standard household surveys to report on numbers of children falling within each dimension Complementary surveys in Congo:  Children in facilities from defined lists  Street children usually interviewed at night

ASER and UWEZO Both employ citizen volunteers ASER survey is in rural areas; and they recognise that UWEZO has a cascade training model based on 14 master trainers with 7,620 volunteers

Recommended Approach (1) List, including ages of household members; then Interview all households reporting a 15 year old, after making an appointment for a revisit In most countries, about 3 times as many household will be revisited as ‘necessary’; but Alternative to ask precise age and status vis-a-vis school of teenager could be unreliable Revisit with a three part instrument: i.Filter to verify reported age and establish her/his precise status vis-à-vis school ii.Test iii.Contextual questionnaire

Recommended Approach (2) Crucial cross-checking reported age Be careful about tendency for responses to behaviour or situations that respondent believes to be disapproved to be biased For non-household populations, distinct samples should be drawn from each group None of other surveys has faced problem of focussing on a specific age group

Ensuring high Response Rates Many OOS 15 year olds may be uninterested in being tested or interviewed Caregivers could help members Quite large proportions are working for or with other family members Also, based on recent DHS surveys, most appear to be living in parents’ households (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia)

Questions for discussion How to administer the assessment/survey? What can we learn from other experiences (ASER/UWEZO/Other)? What do you think of the recommended approach? Any other alternatives to be considered?

Options for administering the test

Persuading OOS 15 year olds OOS 15 yos may have a negative attitude towards school—type activities (such as testing) either because they have recently left or because they are antagonistic to their peers still in school Possible ‘encouragements’ may come from other members of the household. Quite large proportions are living in their parents’ household, (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia); although this may be countered by equally large proportions who are working for / with a household member (who may not want to release the 15 yo for interview and test)

Possible Role of Incentives Incentives often used in developing countries Recognised that, in developing countries, not always seen as appropriate (LAMP) Nevertheless, where only OOS 15 yos are targeted, refusals will be problematic, so important to maximise response rate None of surveys in developing countries have used incentives and all appear to have obtained response rates of >95%, relying on peer pressure from citizen volunteers BUT, they were interviewing all households

Incentives are Problematic We do not know how they might affect responses, they may have unintended consequences, and could be quite expensive If incentives are considered, Singer and Ye (2012) recommend: Offer small prepaid incentives to all sample members; this will increase sample size and help satisfy fairness criterion Offer differential incentives to those who refuse (or a subsample) for bias-reduction reasons

Incentives: Recommendations In the context of this exercise, there are two other issues: Should the incentive be offered only to the 15 year old or to the caregiver/ employer or to both? If citizen volunteers are used, it will be difficult not to give them an incentive as well. In any case, incentives should be explored in the field trial

Interviewing and Administration Tests carried out by ASER, OOSCI and UWEZO all by citizen volunteers, who had been trained by professional trainers. Tests carried out in children’s households, with exception of complementary surveys in institutions and of street children LAMP assessment also household-based Demonstration that most 15 yos were working for a family member validates this approach Mobile phones have been used to collect high frequency panel data (Dillon,2012); and household survey data (Tomlinson et al 2009)

Suggested Approaches Place and Time of Testing: in most cases at their home and probably in the evening, although that may pose problems of security Recruitment, Selection, Training, Deployment of Interviewers: from locality, sensitive to teenage attitudes! and very flexible. They may have to work in pairs, especially after dark Mode of Testing: possibility of designing test so that it can be administered by cell phone should be explored by contractor

Cost and Logistical Implications Assuming a two-stage approach is adopted, and final sample of 100 OOS 15 yos is required in each of 10 geographically dispersed locations in each country, then: Recruitment, training and monitoring of citizen volunteers to identify c.3,000 households reporting a 15 yo Professional interviewing of 300 households to administer test to c.100 OOS 15 year olds Additional context-specific costs for sampling non-household groups

Questions for discussion How difficult will it be to ensure a high response rate? What can we learn from other experiences (ASER/UWEZO/Other)? What do you think of the suggested approaches? Any other alternatives to be considered?