2C/D Approach to Sampling/ SurveyingAdministering the assessment/survey

UNICEF/ UIS Initiative• 25 countries (including Brazil, Cambodia and

Zambia) engaged in initiative• Nearly all country reports use administrative

data and standard household surveys to report on numbers of children falling within each dimension

• Complementary surveys in Congo:Children in facilities from defined listsStreet children usually interviewed at night

ASER and UWEZO

• Both employ citizen volunteers• ASER survey is in rural areas; and they

recognise that• UWEZO has a cascade training model based

on 14 master trainers with 7,620 volunteers

Recommended Approach (1)• List, including ages of household members; then• Interview all households reporting a 15 year old,

after making an appointment for a revisit• In most countries, about 3 times as many

household will be revisited as ‘necessary’; but• Alternative to ask precise age and status vis-a-vis

school of teenager could be unreliable• Revisit with a three part instrument:i. Filter to verify reported age and establish her/his

precise status vis-à-vis schoolii. Testiii. Contextual questionnaire

Recommended Approach (2)• Crucial cross-checking reported age• Be careful about tendency for responses to

behaviour or situations that respondent believes to be disapproved to be biased

• For non-household populations, distinct samples should be drawn from each group

• None of other surveys has faced problem of focussing on a specific age group

Ensuring high Response Rates

• Many OOS 15 year olds may be uninterested in being tested or interviewed

• Caregivers could help members• Quite large proportions are working for or with

other family members• Also, based on recent DHS surveys, most

appear to be living in parents’ households (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia)

Questions for discussion

• How to administer the assessment/survey?• What can we learn from other experiences

(ASER/UWEZO/Other)?• What do you think of the recommended

approach?• Any other alternatives to be considered?

Options for administering the test

Persuading OOS 15 year olds• OOS 15 yos may have a negative attitude towards

school—type activities (such as testing) either because they have recently left or because they are antagonistic to their peers still in school

• Possible ‘encouragements’ may come from other members of the household.

• Quite large proportions are living in their parents’ household, (over 80% in Cambodia, two-thirds in Tanzania, about half in Senegal and Zambia); although this may be countered by equally large proportions who are working for / with a household member (who may not want to release the 15 yo for interview and test)

Possible Role of Incentives• Incentives often used in developing countries• Recognised that, in developing countries, not

always seen as appropriate (LAMP)• Nevertheless, where only OOS 15 yos are

targeted, refusals will be problematic, so important to maximise response rate

• None of surveys in developing countries have used incentives and all appear to have obtained response rates of >95%, relying on peer pressure from citizen volunteers

• BUT, they were interviewing all households

Incentives are Problematic

• We do not know how they might affect responses, they may have unintended consequences, and could be quite expensive

• If incentives are considered, Singer and Ye (2012) recommend:

• Offer small prepaid incentives to all sample members; this will increase sample size and help satisfy fairness criterion

• Offer differential incentives to those who refuse (or a subsample) for bias-reduction reasons

Incentives: RecommendationsIn the context of this exercise, there are two

other issues:• Should the incentive be offered only to the 15

year old or to the caregiver/ employer or to both?

• If citizen volunteers are used, it will be difficult not to give them an incentive as well.

• In any case, incentives should be explored in the field trial

Interviewing and Administration• Tests carried out by ASER, OOSCI and UWEZO all

by citizen volunteers, who had been trained by professional trainers.

• Tests carried out in children’s households, with exception of complementary surveys in institutions and of street children

• LAMP assessment also household-based• Demonstration that most 15 yos were working

for a family member validates this approach• Mobile phones have been used to collect high

frequency panel data (Dillon,2012); and household survey data (Tomlinson et al 2009)

Suggested Approaches• Place and Time of Testing: in most cases at

their home and probably in the evening, although that may pose problems of security

• Recruitment, Selection, Training, Deployment of Interviewers: from locality, sensitive to teenage attitudes! and very flexible. They may have to work in pairs, especially after dark

• Mode of Testing: possibility of designing test so that it can be administered by cell phone should be explored by contractor

Cost and Logistical Implications• Assuming a two-stage approach is adopted, and

final sample of 100 OOS 15 yos is required in each of 10 geographically dispersed locations in each country, then:

• Recruitment, training and monitoring of citizen volunteers to identify c.3,000 households reporting a 15 yo

• Professional interviewing of 300 households to administer test to c.100 OOS 15 year olds

• Additional context-specific costs for sampling non-household groups

Questions for discussion

• How difficult will it be to ensure a high response rate?

• What can we learn from other experiences (ASER/UWEZO/Other)?

• What do you think of the suggested approaches?

• Any other alternatives to be considered?