VALUE CHAIN MONITORING AND EVALUATION GUIDE

MODULE 5

Select Data Collection Tools

April 13, 2023

2

GETTING STARTED

OBJECTIVE OF MODULE 5Define the tools you will use to

collect data around the indicators you have identified, both routine

and observed

April 13, 2023

3

GETTING STARTED

This module will:• Lay out a simple process for deciding the types of

data collection tools that will best meet your needs• Share information on some common tools used in

CARE market engagement projects that can readily be adapted to meet the needs of other CARE projects

• Refer you to other resources that index additional tools and data collection methods worth considering including

April 13, 2023

4

GETTING STARTED

Core theme of this module:

The tools we consider must be the simplest way of

meeting the data needs of our priority M&E clients within our resource and capability constraints.

April 13, 2023

5

GETTING STARTED

WARNING!

Just because you can write it down does not mean you can make it happen!

April 13, 2023

6

Market Engagement Indicator Information Sheet

Measurement tools and processes used by other CARE projects in the CO and/or partner organizations

Measurement tools and processes used by CARE programs in the CO

Information on any donor-required tools

Secondary information sources for related indicators

MATERIALS / INPUTS RECOMMENDED

April 13, 2023

7

STEP-BY-STEP GUIDE

1

•SEPARATE ROUTINE OBSERVATION INDICATORS FROM THOSE FOR ROUTINE MEASUREMENT

2

•IDENTIFY TOOLS FOR ROUTINE OBSERVATION INDICATORS

3

•IDENTIFY TOOLS FOR ROUTINE MEASUREMENT INDICATORS

4

•ENSURE TOOLS ALIGN WITH THE NEEDS OF PRIORITY CLIENTS AND RESOURCES / CAPACITY

5

•FINALIZE THE MARKET ENGAGEMENT INDICATOR INFORMATION SHEET

April 13, 2023

8

STEP-BY-STEP GUIDE: STEP 1

Purpose•Simplifies the selection process

•Different actors will use different tools

April 13, 2023

9

STEP-BY-STEP GUIDE: STEP 1

Two options to complete Step1:1. Filter your Market Engagement Indicator

Information Sheet by indicator type (measurement, observation)

2. Create a separate table like the one below:

Expected Change

Observation Measurement

Indicator Data Collection Tool Indicator Data Collection Tool

April 13, 2023

10

STEP-BY-STEP GUIDE

1

•SEPARATE ROUTINE OBSERVATION INDICATORS FROM THOSE FOR ROUTINE MEASUREMENT

2

•IDENTIFY TOOLS FOR ROUTINE OBSERVATION INDICATORS

3

•IDENTIFY TOOLS FOR ROUTINE MEASUREMENT INDICATORS

4

•ENSURE TOOLS ALIGN WITH THE NEEDS OF PRIORITY CLIENTS AND RESOURCES / CAPACITY

5

•FINALIZE THE MARKET ENGAGEMENT INDICATOR INFORMATION SHEET

April 13, 2023

11

STEP-BY-STEP GUIDE: STEP 2This process will require

input from field staff, project managers, implementing partners as well as other key M&E clients that will actively use this information.

April 13, 2023

12

STEP-BY-STEP GUIDE: STEP 2

COMPLETING STEP 2• Have a dedicated meeting to agree on

criteria for the tools. Consider the following:• Tools or practices from other initiatives• Set the expectation that field staff and

implementing partners will be helping to co-design the data collection tools.

• Develop a participatory approach to develop field staff and implementing partner ownership.

• Facilitate the discussions, capture the outputs and define the process for finalizing the tools – which will likely require several meetings!

April 13, 2023

13

STEP-BY-STEP GUIDE

1 •SEPARATE ROUTINE OBSERVATION INDICATORS FROM THOSE FOR ROUTINE MEASUREMENT

2 •IDENTIFY TOOLS FOR ROUTINE OBSERVATION INDICATORS

3 •IDENTIFY TOOLS FOR ROUTINE MEASUREMENT INDICATORS

4 •ENSURE TOOLS ALIGN WITH THE NEEDS OF PRIORITY CLIENTS AND RESOURCES / CAPACITY

5 •FINALIZE THE MARKET ENGAGEMENT INDICATOR INFORMATION SHEET

April 13, 2023

14

STEP-BY-STEP GUIDE: STEP 3

•This step will feel much more familiar to M&E system designers•Refer to the following:•CARE DME guidelines•Donor standards or guidelines

•CARE Program M&E System tools

April 13, 2023

15

STEP-BY-STEP GUIDE

1 •SEPARATE ROUTINE OBSERVATION INDICATORS FROM THOSE FOR ROUTINE MEASUREMENT

2 •IDENTIFY TOOLS FOR ROUTINE OBSERVATION INDICATORS

3 •IDENTIFY TOOLS FOR ROUTINE MEASUREMENT INDICATORS

4 •ENSURE TOOLS ALIGN WITH THE NEEDS OF PRIORITY CLIENTS AND RESOURCES / CAPACITY

5 •FINALIZE THE MARKET ENGAGEMENT INDICATOR INFORMATION SHEET

April 13, 2023

16

STEP-BY-STEP GUIDE: STEP 4

Take a step back and ask the following questions:  1. Will the combination of tools we have

identified allow us to meet the needs of the priority clients of the M&E system?

2. Based on our resources and capabilities assessment, can we apply and manage all of these tools effectively?

April 13, 2023

17

STEP-BY-STEP GUIDE

1

•SEPARATE ROUTINE OBSERVATION INDICATORS FROM THOSE FOR ROUTINE MEASUREMENT

2

•IDENTIFY TOOLS FOR ROUTINE OBSERVATION INDICATORS

3

•IDENTIFY TOOLS FOR ROUTINE MEASUREMENT INDICATORS

4

•ENSURE TOOLS ALIGN WITH THE NEEDS OF PRIORITY CLIENTS AND RESOURCES / CAPACITY

5

•FINALIZE THE MARKET ENGAGEMENT INDICATOR INFORMATION SHEET

April 13, 2023

18

Capture the outcomes of your tool selection process in your Indicator Information Sheet

STEP-BY-STEP GUIDE: STEP 5

Result Level Domain (Women's Emp, Enterprise, Sector)

Key Change (Y/N)

Indicator - Measurement

Indicator - Observation

Target Data Users (Clients)

Data Source Collection Method / Tool

Collection Frequency

Responsible for Collection

Impact

Lagging outcome

Leading outcome

Output

April 13, 2023

19

COMMON PITFALLS

•M&E system designers lose sight of the diverse needs of M&E clients leading to tools that only meet the needs of the most obvious or powerful interests – typically donors.

•Projects focus disproportionately on traditional information gathering methods (periodic, formal, quantitative methods applied to large samples) and fail to utilize ad hoc, less formal, qualitative, and rapid assessment methods.

•M&E system designers focus on proving that our project caused a particular change (attribution) instead of considering whether an M&E system designed to demonstrate how we contributed to change might be more appropriate given our needs and our resources.

April 13, 2023

20

TEMPLATES AND SUPPORTING MATERIALS

Additional resources for Routine Measurement and Observation:

•CARE Design, Monitoring and Evaluation Guide

•CARE International Evaluation Policy

•ODI Tools for Knowledge Management and Learning: A Guide for Development and Humanitarian Organizations

•DCED Results Measurement Standard

April 13, 2023

21

QUESTIONS?

COMMENTS?

April 13, 2023

22

Multiple ways to continue the discussion and continue learning:• Initiate a monthly session on the M&E guide

and case studies from across CARE. Contact cpennotti@care.org

• Join the Market Engagement Community of Practice on LinkedIn.

• Join a task force to review and refine the universal indicators. Contact nardi@careinternational.org

Want to Learn More?