Management Survey Tool Changing the way we collect data in health surveys

www.ifrc.orgSaving lives, changing minds.

Management Survey Tool

Changing the way we collect data in health surveys

Presented by Jenny Cervinskas

and Bong Duke

On behalf of the MST team for the Cross River State, Nigeria rollout

July 6, 2011


Partners

Roll Back Malaria (RBM)/MOH

Nigerian Red Cross National Society

IFRC (International Federation of Red Cross Red Crescent Societies)

WHO


Outline

① MST• Purpose and key features• How it works• How much it costs• Benefits of the tool

② MST rollout in Cross River State

③ What’s next?


Purpose of the Management Survey Tool (MST)

Provide a survey methodology in which Red Cross societies, government and other partners can: conduct health surveys at reduced cost with limited external technical assistance achieve high standards of survey design and quality

MST aims to simplify and improve the timeliness of the entire data collection cycle

To use mobile phones and a web-based, freely accessible software domain as a data collection technique to conduct health surveys


Rapid spread & technology of cellphones opens up new possibilities

What can you do with a phone?The same things you can do with any other computer.

CountTalk

Reward

Photograph

Assist

Teach

Monitor

Analyse


Traditional Paper and Pencil Questionnaire

The time and costs of data collection can be substantially reduced if mobile phone data collection is used in place of the traditional paper and pencil method that has been the best practice in health surveys for decades


Key Features of the MST

Has an accompanying training manual, technical manual, and tools adaptable to local settings

Allows for: web based questionnaire design using EpiSurveyor questionnaire forms to be uploaded to standard mobile phones data collection using low cost, familiar and widely available mobile

phones (e.g. Nokia, Samsung) Data can be exported to Microsoft Excel, as a text file, and in

Mdb format Enables rapid analysis and reporting of survey results


With the MST you can transform a standard mobile phone into an innovative evaluation or research tool

Conduct surveys and capture data

froma standard mobile

phone

Manage surveys, people and data from your web-based server


How does it work?

Data is entered and saved in cellphone, sent to the server…

There is 2G/GPRS coverage in almost all villages in CRS

Survey Bulletin within 24 hoursSurvey Report within 72 hours


Web Based Server

Create a free account using Datadyne’s EpiSurveyor software

Access your web based server from a web browser anywhere in the world

Design your questionnaire with embedded logic and in multiple languages

Monitor, manage and communicate with your team

Export data and analyze results in real-time.

http://www.episurveyor.org/user/index

http://www.episurveyor.org/user/index


Why use mobile phones to collect data? Real-time data entry on cell phones Daily upload of data from cell phone over 2G cell network to

internet database Real-time data monitoring and data quality checks Real-time data cleaning Real-time data analysis Rapid production of survey results within hours or days of last

interview


Stakeholder benefits

Decision Makers• No software licensing or subscriptions • Reduces environmental impact• Scalable solution for teams and studies of varying sizes• Get data and information rapidly for decision-making and planning

Evaluators/Researchers• Incorporate a multitude of question types with custom logic and validation• Manage and upload surveys in multiple languages• Monitor staff work rate, productivity and quality• Export data for custom analysis with your favourite statistical analysis package

Fieldworkers• Conduct surveys anywhere, even in areas with no network coverage• Use standard and familiar mobile phone• No more paper to collect, transport or return• Automated submission of data when network reception is available


MST Rollout in Cross River State, Nigeria:Focus on Malaria


Site identification

Cross River State, Nigeria 10 of 18 LGAs in CRS LLIN distribution took place January-June 2011 MST malaria survey to provide a post-campaign evaluation of

the NRC door-to-door LLIN distribution and hang-up campaign Mobile network coverage

Data collected on key indicators

MST survey focus: Malaria

A rainy season survey


Standard survey methodology used in an MST survey

Multi-stage cluster sampling 1st stage: standard probability-proportional-to-

estimated-size (PPES) selection of clusters 2nd stage: selection of households- simple random

sampling (SRS) to choose 10 households/cluster

Survey methods: Sampling

30 clusters, 10 households per cluster

300 households


Modeled after the standardized MIS questionnaires Household questionnaire Person roster/Treatment and diagnosis of fever in

under-5S Net roster

Questionnaire administered in English

Survey questionnaires

Red Cross volunteers carried out the interviews

Six teams: Two interviewers per team

Six team leaders

+Survey Supervisory Support Team

Training (June 20-24)

Adapted the MST curriculum and guide

Content Cellphone basics Questionnaires Informed consent Interview techniques Field procedures Field logistics/reporting Supervisor training

Methods

Fieldwork (June 27-July1)


Locating the chosen cluster, preparing the

sketch map, segmentation and selecting the

households to be interviewed

Conduct household interviews

Team leader/interviewers send data to server

Evening briefing (“quality round”)

Data cleaning, editing and analysis

A day’s schedule

Real-time data editing and cleaning

Daily data monitoring, editing and cleaning


Preliminary Results


Results: key indicators

Access: % HH ≥1 ITN

Access: % pop. with access to

ITN**

Access: % HH with sufficient

ITNs***

ITN use, all persons

ITN use, chil-dren <5 years

ITN use in chil-dren <5y, given at least 1 ITN in

HH

% of ITNs used last night

Treated with ACT, children

with fever

Treated with ACT within 24

hr, children with fever

Blood taken (testing), chil-

dren with fever

0

10

20

30

40

50

60

70

80

90

100

87

66

5660

71

80

84

60

24

16

Percentage


Other indicators

Key indicators

Target population 1.699.246

Persons per net 1.94

Estimated LLINs in households in all 10 LGAs

627.887

Estimated LLINs in all 10 LGAs from 2011 campaign

540.068


Results: High percentage of ITNs were used last night. High ITN use, especially in children

Key indicators Point estimate

% ITNs that were slept under last night 84%

% ITNs that were hung last night 86%

ITN use, all ages 60%

ITN use, <5 yo 71%

* 99% of nets were ITNs & 99% of ITNs were LLINs


ITN use by age group by gender

<1y 1

2-4

5-9

10-1

4

15-2

4

24-4

4

45-5

9

60+0

10

20

30

40

50

60

70

80

90

100

Male Female Both genders

Age groups (years)


Age in months %

<12 months 85

12-23 months 1

24-36 months 5

36+ months 9

Age of ITNs


Number of persons sleeping under ITN last night %, nets

1 person 33

2 persons 45

3 persons 18

4 persons 3

Number of persons under an ITN last night


Treatment & diagnosis, <5 yo

Key indicators %

Treated ACT 60

Treated ACT within 24 hours 24

Received finger/heal stick for blood 16

- Denominator for all indicators was % of children <5y with fever in the previous two weeks


So, does the MST “work”?

Field methods followed Daily data monitoring and cleaning accomplished Preliminary survey results bulletin finished within

24 hours Preliminary report finished within 72 hours Provided excellent management information on

the key indicators


Cost component USD

Training 9,053 (26%)

Survey operations: per diem, accommodation, technical assistance

13,604 (38%)

Transportation 9,698 (27%)

Cellphones and equipment 2,357 (7%)

Other 728 (2%)

Total 35,440

Local survey costs: CRS, Nigeria


Cellphone-based Surveys: Summary Points

REAL-TIME DATA AVAILABILITY AND ANALYSIS Via your web-based server, responses may be viewed, monitored and exported

instantly

IMPROVED DATA INTEGRITY The removal of paper from the research process reduces the number of points at

which error can be introduced

FIELDWORKER MONITORING/MANAGEMENT Monitor the productivity and quality of work conducted by field staff (GPS, time and

date stamp)

ENHANCED MOBILITY Do not need network coverage to conduct surveys, responses are stored securely on

the mobile phone, thus can reach even the most remote communities

OPTIMISED RESOURCE USAGE Save on survey printing, distribution and collection costs


What’s next? For IFRC…

Extract lessons learned from Kenya, Namibia and Nigeria and share for application in other surveys

Finalize and disseminate the IFRC MST technical manual and the training manual

Continue developing strategies for technical support in order to gradually reduce external support

Test the MST in other sectors and disciplines if appropriate

Continue searching for innovative ways to collect data in a timely fashion in order to better serve the communities we work in

Thank-you for your attention


What’s next?

Documents

Management Survey Tool Changing the way we collect data in health surveys