Improving OVC programs with mobile M&E:

Operations research results

Adele ClarkCORE Spring Meeting

11 May 2011

Project Background

• 2004-2010, through PEPFAR Track 1 program, CRS served 37,749 OVC in TZ

• CRS’ approach is to work with Parish Coordinating Committees and Most Vulnerable Children Committees

• CRS implemented the

OVC project through

8 local partners

M&E Background

• Geographical scope: project operates in 5 regions

• Poor communication infrastructure

• Delays in data exchange (2-3 months)

• Multiple stakeholders

M&E Background

• 36 questions, Likert scale

• Self-report measure

• Used with children ages 13-18

• 10 domains of wellbeing

• OWT is available in multiple languages and has been used in more than 20 countries

http://www.crsprogramquality.org/ovcwt/

Pre-Operations Research Phases

The prototype of 4 forms for mobile phones (OVC

wellbeing tool, follow-up , exit form and referral) were

programmed and downloaded on telephones.

The phone prototype was field tested, by 6 volunteers.

Based on the lessons learned, the team refined the

prototype. A training manual was created.

Operations Research

Arusha, Tanzania

April-June 2010

Operations Research Objectives

(1) Identifying the CRS technical support and structures that will most enhance effective usage of the technology

(2) Identifying the profile of the user who will most efficiently use mobile technology

(3) Developing guidance for the administration and management of the telephones

Methodology

• Pre- and Post- Tests

• OVC Activity Questionnaire

• Field observation

• Focus Group Discussions

• Participant workshop, post-field test

• OVC interview simulations

Participant Demographics

• Among the 17 regular volunteers , 71% men and 29% women, mean age of 48

• High Achievers : 6 lead volunteers, 83% men and 17% woman, mean age of 42

• Dropouts : 4 volunteers, 100% women, mean age 51

• Geography: Arusha with urban, peri urban and rural mix

• Education: All had completed primary school (standard 7), with 5 having completed secondary school

• Cell phone experience: All had 2+ years of previous SMS use , all at least weekly texting and most daily; 90% of volunteers already own their own phones

Results- Objective #1

• Network failures and delays were identified as problematic (5 out of 29)

• Battery charging is an ongoing challenge (14 out of 29)

• Deleting entire application from phone requiring D-tree to re-install (3 out of 29)

Results- Objective #2

• No demographic differences in success (or failures) in using mobile technology

• 100% of volunteers favored submitting data forms and receiving reminder messages and technical support from supervisors via cell phones

• 90% reported that the mobile application improved their communication with OVC

Results- Objective #3

• No telephone theft; however, battery theft (swapped for older ones) occurred at recharging stations

• Need to establish payment policy for extra minutes for personal cell use

• Need to raise awareness about why volunteers have cell phones

Advantages

Mobile method

• Eliminates skipped questions and multiple answers

• Reduces lag time in reporting

• Less bulky/heavy for volunteers

• Increased objectivity (potential)

Pen and Paper method

• Ability to skip questions , insert multiple answers, or add subjective notes when appropriate

• Creating a community record

Recommendations- Technical Support

• 2 batteries provided to each volunteer

• Retooling of mobile application to correct common errors

• Continue relationship with mobile application provider, such as D-Tree, for maintenance

Recommendations- Management

• Phones remain property of the project with guidelines on voucher procurement, damage liability and replacement costs

• User contracts must be established

Conclusion

• Dual format method (i.e., mobile and paper) is recommended

• Electronic feedback mechanism should be created to maintain community ownership of data

• When you think you’ve done enough training, do it again

Thank you!

www.crs.org

http://www.crsprogramquality.org/