Designing an M&E System for Market Engagement at CARE

Christian Pennotti

Technical Advisor, Learning & Impact

Economic Development Unit

cpennotti@care.org

CARE’s M&E Approach – Questions for Consideration

How can we provide guidance that ensures M&E becomes an integral part of regular project processes?

How can we provide guidance that enable teams to design M&E that effectively responds to value chain dynamics?

How can we combine an increased desire to be held directly accountable to our impact group members while maintaining reasonable cost structures for M&E?

Core Principles

1. Base M&E on a clear and well-documented causal model

2. Support project management and accountability to target beneficiaries

3. Emphasize leading outcomes in monitoring and lagging outcomes in evaluation

4. Incorporate learning loops that leverage both tacit and explicit knowledge from multiple stakeholders

5. Be flexible and able to adapt to project evolving project priorities

6. Be feasible to implement with resources and capacity

Steps to Designing the M&E System

Core Principles

1. Base M&E on a clear and well-documented causal model

2. Support project management and accountability to target beneficiaries

3. Emphasize leading outcomes in monitoring and lagging outcomes in

evaluation

4. Incorporate learning loops that leverage both tacit and explicit knowledge from

multiple stakeholders

5. Be flexible and able to adapt to project evolving project priorities

6. Be feasible to implement with resources and capacity

Step 1M&E System Client

Mapping

Step 10Implement M&E

System, Review and Adjust

Step 9Provide Training to All

Personnel Responsible for M&E

Step 8Develop Data Management

Protocols

Step 7Ensure Incentive

Alignment

Step 6Design Data Analysis

and Use Plan

Step 5Identify Data

Collection Tools

Step 4Identify Indicators

Step 3Assess M&E

Resources & Capacity

Step 2Develop / Refine

Causal Model

1. M&E System Client Assessment

Power / Relevance Mapping

Relevance

Application • Participatory exercise

• Group brainstorm

• Small groups discuss and place stakeholders on the matrix

• Facilitated discussion about variance until final set of clients to be “Managed Closely” defined.

• Leads to discussion on client needs and expectations

2. Value Chain Initiative Causal Model

text

Enterprise

Results

Women’s Empowerment

Results

Sector

Results

Direct / Enterprise

Results

Women’s Empowerment

Results

Indirect / Sector

Results

Sustainable Household Livelihoods & Women’s Empowerment

Impacts Impacts

Activities & Outputs

Activities & Outputs

Leading Outcomes:Knowledge, Attitudes, Practices

Lagging Outcomes: Countable,

Tangible Chnages

Assumptions

Application • Participatory exercise

• Break the causal model into pieces

• Small groups discuss and place results on the matrix

• Facilitated discussion about variance until final agreement reached on the project’s fundamental logic

• Leads to clarity on ‘killer assumptions’ as well as indicator needs

2. Example from Peru

Women’s Emp. Enterprise Sector

Leading

Lagging

Activities

Impacts

4. Indicator Selection

Two types of indicators

Those tracked through routine measurement – traditional M&E

Those tracked through routine observation – leveraging staff observation

4. Indicator Selection – Identifying Killer Assumptions

Application • Participatory exercise

• Pull out the ‘lines’ connecting causal pathways in the causal model

• Blind ranking of assumptions with the greatest risk and greatest potential reward

• Definition of ‘indicators’ to monitor validity of prioritized ‘killer assumptions’ via observation

Step 5: Select Tools

Indicators to be tracked through routine measurement Traditional M&E tools (FGDs, surveys, etc)

Indicators to be tracked through routine observation KM-oriented tools (After Action Reviews,

routine meetings, common learning agenda, observation cards, etc)

Step 6: Designing Data Analysis & Use

Indicators for Routine Measurement Highly related to traditional practices but

with greater focus on leveraging data for multiple M&E System Clients

Indicators for Routine Observation Focus on KM tools, reflective practice and

increased opportunities for simple ranking and benchmarking performance across the project

Step 7: Ensure Incentive Alignment

To be developed

Focus on ensuring data quality and ability to improve decision making

Leverage examples raised in the e-consultation like HR integration, monetary incentives, peer pressure

Need to define limitations – not everything can be solved by the M&E system!

Illustrations of Routine Observation Integration

Zambia ADAPT Project Staff-defined monthly learning themes, consolidated

summaries Observation-based agro-dealer categorization tool

Bangladesh SDVC Project Routine, focused meetings at all levels of the project Participant-reporting categorization tool Regional performance scorecards for field managers

Ethiopia PSNP Plus Initiating geographically rotating technical working group

meetings

Moving Ahead

Developing data analysis / use guidelines

Developing incentives module

Target guide release – Fall, 2011

Ongoing application, revision and building use case library

Collaboration via GROOVE & MaFI to refine, apply, advance