2016 aapor (not presented) rahija mtui aapor

Report on Tablet Assisted Personal Interview (TAPI) Implementation by the Ministry of Livestock and Fisheries Development (MLFD) in Tanzania

Michael Rahija, Research Officer, GSARS Dr. Niwael Mtui, Principle Veterinarian, MLFD

1

Overview

• Background: Study and justification of using

TAPI

• Survey Implementation

• Monitoring reports and communication with

field staff

• Overview of costs

• Challenges encountered in the field

• Lessons Learned and Conclusion

2

Background

3

Background – Study Rationale

FAO, the Tanzania Ministry of Agriculture,

Livestock and Fisheries and the Tanzania National

Bureau of Statistics have been collaborating since

2011 to:

- Improve the livestock component of the

agricultural statistics system

- Facilitate the use of official statistics / statistics

methods for formulating / implementing

evidence-based policies

4

Background – Study Rationale

- Tanzania currently has one of the largest datasets

on livestock at household (farm) level throughout

Africa via the National Panel Survey)

- Ministry of Agriculture analysed the data and found

that many farmers relied on livestock, but did not

access extension services.

- Accordingly, a policy increasing access to

extension services would benefit farmers and

enhance food security.

5

Background – Study Rationale

MALF agreed to invest resources to improve the

system of livestock extension.

HOW?

6

Background – Study Rationale

MALF agreed to invest resources to improve the

system of livestock extension.

1. A survey of extension officers (change agents)

2. An experiment to test alternative options to

improve their on the job performance

7

Background – Study Rationale

1. A survey of extension officers (change agents)

2. An experiment to test alternative options to

improve their on the job performance (Jan 16)

TAPI

8

• Timelineness and cost

• Data Quality

• Capacity development

• Choice of Survey Solutions

Background – TAPI Rationale

9

Implementation

10

• Headquarters at MALF and PI from

London School of Economics (LSE)

–Survey design - Translation

–Survey management

• Enumerators and field staff

–Basic tablet operations

–Questionnaire training directly on tablet

–Use of quizzes

Implementation - training

11

• 5 teams of 3 enumerators and 1 supervisor

• Allocation of enumerators – Based on quiz performance

– Tech savviness

• Assignment of Field Supervisors

– Based on understanding of questionnaire

– Tech savviness

• Headquarters – MALF, Global Strategy, LSE

Implementation – Teams

12

• Target population: Village Ward livestock

officers, and District Livestock Officers

• Goal: complete enumeration in Morogoro,

Iringa, and Dodoma

• Final sample: 63 DLOs and 415 VLOs

Implementation – Sample

13

• Questionnaire (~150 questions for VLOs): – Personal info

– Livestock service provision and transport

– Farmer interaction

– Hierarchy

– Other income sources

– Personality measures

– Knowledge of rule, regulations, and policies

– Incentives

– The game

Implementation – Q

14

Implementation – Field work

Interview locations

15

Implementation – Field work

Progress by day

16

Implementation – Field work

Duration

17

Monitoring

& Communication

18

• Daily export of paradata

• Rmarkdown for automated tabulations – Interviews by team and enumerator

– Duration of interviews

– Average interviews per enumerator

– Assignments

• Daily export of microdata and automated

checks

Monitoring

19

• After report generation, mgmt at the MALF

called Supervisors

• HQ called tablet of enumerators re:

specific interviews

• Comments inside of questionnaire used to

alert enumerators of various problems

Communication

20

• Information management:

– Email, 2 sim cards, IDs through CAPI software

Communication

internal.id devicename email interviewname enumerator AIRTEL card tigo MinistryLF1 MinistryLF1 MinistryLF1@gmail.com lss_inter1 ### 0687333122 0675343970 MinistryLF2 MinistryLF2 MinistryLF2@gmail.com lss_inter2 ### 0687332727 0675343971 MinistryLF3 MinistryLF3 MinistryLF3@gmail.com lss_inter3 ### 0687327582 0675433985

21

Data analysis

• September 2015 data

cleaning (limited)

• October 2015: MALF

report available

• Nov 9th 2015: report

presented to the

Permanent Secretary

22

Overview

of Costs

23

Overview of costs of using TAPI

Item Units Total (TZS) Total (USD 17/9/15)

Samsung Galaxy 4 22 20,592,000 9,950.00

Battery chargers 22 50,000 512.00

Tablet covers 22 20,000 204.00

1 month data Tigo 22 25,000 256

1 month data Airtel 22 25,000 256

Server (Amazon – cloud) 0 0 0

Programming cost 0 0 0

Total - 23,232,000 10,820

24

Challenges encountered

in the field

25

• In some remote areas, it was difficult to

find a signal.

• Solution 1: Enumerators waited to

synchronize until a signal was found.

• Solution 2: Back-up paper questionnares

were provided.

Challenge - Connectivity

26

• Seemingly at random, SIM cards would

become de-configured and the tablets

wouldn’t synchronize.

• Solution 1: The second SIM card was

used.

• Solution 2: Field Supervisors were trained

and provided instructions for manual re-

configuration.

Challenge – De-configuration

27

• During the first week of data collection,

enumerators recommended some

improvements to questionnaire.

• Solution 1: HQ team made changes and

conducted trouble-shooting at night. Then

field supervisors were instructed to tell

enumerators to re-synchronize. Not

Recommended!

Challenge – Questionnaire

28

• The initial delivery of tablets was missing 6

tablets.

• Solution 1: Anyway, four tablets were

meant to be back-ups.

• Solution 2: Paper questionnaires provided

to some enumerators

• Solution 3: NBS loaned the project 2

tablets.

Challenge – Procurement

29

Lessons Learned,

Software recommendations,

and Conclusion

30

• Cost – justified in terms of data quality,

time b/t collection and analysis, and cost.

• Monitoring – Daily monitoring reports were

valuable to enhance team performance.

• Connectivity – Giving enumerators 2 SIM

cards minimized these problems.

• Training – Supervisors should be trained

on basic tablet maintenance.

Lessons Learned

31

• Training – One should train enumerators

on the questionnaire directly in the tablet.

• Training – Survey Solutions can be used

by people with limited or no programming

experience.

Lessons Learned

32

• Offline survey designer

• Feature to toggle between languages

• Automated generation of PAPI

questionnaire needs to be greatly

improved

• More documentation on enabling and

validation conditions.

Recommendations

33

• MALF consider implementation of CAPI to

be a success.

• Preliminary results were made available

and circulated within 1.5 months.

• The expected benefits of using Survey

Solutions were realized.

• Experience to be replicated / improved

through other FAO’s policy-assistance

initiatives

Conclusion

34

Thank You • For more questions,

email:

• Michael Rahija

• michaelaustin.rahija@fao.org

• Dr. Niwael Mtui

• niwaelanna@yahoo.co.uk

35