The Use of Random Geographic Cluster Sampling to Survey Pastoralists

WPS 6589

Kristen Himelein

Stephanie Eckman

Siobhan Murray

1

Background

• Livestock play integral role in livelihoods of vulnerable populations – Main source of food and transportation

– Store of wealth

– Coping mechanism in response to shocks

• Populations difficult to survey

–Often pastoralist or semi-pastoralist

• HH based samples may not be sufficient

2

Location: Afar, Ethiopia

• Afar, Ethiopia highly pastoralist

• More than 40 percent of respondents reported owning 10 or more cattle in 2009 AgSS

– Camels, goats

• Bounded by – national borders north & east

– mountains to the west

– ethnic differences

3

Research Objectives

• Use an alternative sampling approach to reach pastoralists

• Are we able to capture populations missed by a dwelling based sampling frame?

• How do our figures compare with other sources of information?

4

Random Geographic Cluster Sampling [RGCS]

• 1st stage: select random geographic points

• 2nd stage: survey all eligible respondents within given radius

• Similar designs used: – Agricultural statistics agencies (ex: USDA)

– Livestock studies in developing world (Cameron, 1997; Soumarea et al, 2007; von Hagen, 2002)

– Surveys of forests (Husch 1982; Roesch et al 1993)

5

Stratification

• Strata Inputs: land cover (towns and settled agriculture), distance to water, vegetation index

Strata

Definition Expected Radius

1 Towns High 0.1 km

2 Settled agri. areas, commercial farms Low 0.5 km

3 Within 2 km of major river or swamps High 1 km

4 Within 10 km of major river or swamps Medium 2 km

5 Remainder Low 5 km

7

Stratification

8

Stratum 3 – High

Stratum 5 – Low

Field Work

• Selected points pre-loaded on GPS

– Alarm indicated when interviewer inside radius

• Interview all eligible respondents within radius

– Only HHs with livestock eligible

– Livestock questions related to cattle, camels, goats

• Ownership, vaccination, theft, death, etc.

13

Weights

• Weights are complicated to calculate. • Inverse of probability of selection

14

i i

i

• Probability of selection is

1 – (prob. that none of these points selected)

𝜋𝑖 = 1 − 1 −𝜋𝑟2

𝑡𝑜𝑡𝑎𝑙 𝑎𝑟𝑒𝑎

𝑐

c is number of points selected

Stratum 1 Stratum 2

X r2

r1

Stratum 1 Stratum 2

X

Weight Adjustment

• Teams did not always visit entire circle

• Two measures

– Supervisor report

– Viewshed measures

15

Weight Adjustment

• Should we adjust weights?

– No if we think area not covered because flooded or too thickly covered in vegetation for animals

– Yes if we think that areas were not covered because of time limitations / laziness

• Adjustment: 𝑤𝑖 = 𝑏𝑤𝑖 ∗1

% 𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑

• 2 sets of weights: baseweight, adj weight

16

Implementation Challenges

• Field workers unaccustomed to technique

• Unexpected challenges

– Early start to rainy season

– Ethnic conflict / kidnapping

– Volcanoes

– River crossings

– Trouble with vehicles

17

Results of Data Collection (1)

• 102 circles canvassed

– 59% contained at least 1 HH with livestock

• 784 households with livestock interviewed (9 excluded for being outside radius)

• Total livestock found per circle represented on map

Results of Data Collection (2)

Description Points Visited

Circles HHs

Circles without

Livestock

1 Towns 10 10 69 4

2 Settled agri. areas, commercial farms

15 14 113 8

3 within 2 km of major river

60 49 229 24

4 within 10 km of major river

30 22 182 6

5 Remainder 10 7 191 1

Total 125 102 784 43

19

Results of Data Collection (3)

• 3,698 individuals living in households owning livestock

• 127 reported having no permanent dwelling, (approximately two percent weighted estimate of the livestock-holding population in Zones 1, 3, 4, and 5).

• All but five of the individuals without a permanent dwelling lived in households in which all members are completely nomadic.

20

Comparison Survey (1)

• Ethiopia Rural Sample Survey (ERSS) – conducted by the Ethiopia Central Statistics

Agency in two zones in Afar six months prior using traditional census-frame approach.

• Comparisons can be made between the two methods. – Retrospective questions used in RGCS for

comparability

– Analysis limited to zones in which both surveys were administered.

21

Comparison Survey (2)

Mean (SE)

RGCS

(unadj weights)

RGCS

(adj weights)

ERSS

Cattle 10.4

(1.5)

10.8

(1.8)

15.3

(3.3)

Camels 8.0

(1.4)

7.7

(1.4)

6.2

(1.9)

Goats 20.2

(3.1)

19.7

(3.0)

20.7

(3.1)

Estimated means are comparable…

22

Total (SE)

RGCS

(unadj weights)

RGCS

(adj weights)

ERSS

Cattle 153,505

(34,384)

186,164

(51,283)

1,092,752

(367,307)

Camels 92,009

(25,893)

139,608

(37,186)

237,568

(116,430)

Goats 566,139

(146,182)

815,310

(222,853)

2,095,876

(488,027)

Comparison Survey (3)

… but estimated totals are not.

23

Low Effort Hypothesis (1)

• RGCS interviewers did not make efforts to:

– reach all portions of the circles

– did not interview all households in the circles and all holders at those households.

• Reasons:

– Weather (extremely hot during this period)

– Flooding (requiring long detours on foot and/or forge swollen rivers)

– Ramadan period (limited access to local guides)

24

Low Effort Hypothesis (2)

• Logit – Logit for circles visited (n=125) – Logit for circles visited excluding those in which teams

were accompanied by survey coordinator (n=105) – Covariates include kilometers to main road, kilometers

to nearest locality, kilometers to river, ‘relief roughness,’ historical EVI value (greeness measure), supervisor fixed effects

• OLS – percent of area covered conditional on circles having

been visited (n=102) – All of the above plus total rainfall in week prior to

survey, current mean EVI value, accompanied by survey coordinator.

25

Low Effort Hypothesis (3)

• Being accompanied by survey coordinator perfectly predicts success in visiting circle

• Larger circles and those further from the main road are less likely to be visited

• Historically drier circles less likely to be visited in absence of survey coordinator

• Circle radius is inversely correlated with percentage observed.

• Strong and significant supervisor effects

26

ERSS Quality Hypothesis (1)

• Also possible that there is overestimation in the values produced by CSA. – ERSS is a decentralized survey and Afar is considered a

‘low capacity’ field office

– Limited infrastructure makes communications between headquarters and field office difficult, even more difficult for field office to contact field teams

– Minimal road network makes supervision visits difficult

– High incidence of missing / incomplete questionnaires compared to national ERSS averages

27

ERSS Quality Hypothesis (2)

• Very few independent data sources for comparison, best option is flyover survey of neighboring Ethiopian Somali region conducted in 2004.

• Both areas are pastoralist with similar topography

28

Livestock Density (n/km2)

Afar Regions (Zones 1 & 3) Somali Region (Aerial Survey 2004) RGCS

(unadjusted)

RGCS

(adjusted) ERSS Afder

Degeh

abur Fik Gode Korahe Shinile Warder

Camels 2.5 3.8 6.4 2.6 3.6 0.8 2.8 4.2 2.6 8.4

Cattle 4.2 5.1 29.7 3.0 1.4 0.5 4.1 0.7 5.3 0.8

Goats 15.5 22.3 56.9 13.2 19.6 4.2 24.3 19.5 21.7 31.6

Conclusions

• RGCS can be implemented in a low capacity environment with inexpensive hardware – though not without some difficulties.

• RGCS does in fact capture nomadic populations

• Necessary to incentivize interviewers to elicit a ‘high effort’ response.

• It is likely that RGCS has under-estimated the total livestock population in Afar, but this still may be more accurate than those produced by the census-frame ERSS survey.

29

30

Contact: Kristen Himelein khimelein@worldbank.org Stephanie Eckman stephanie.eckman@iab.de Siobhan Murray smurray@worldbank.org