Impact Monitoring in Value Chain Kenya

8/7/2019 Impact Monitoring in Value Chain Kenya

1/54

Concepts and Application

Deutsche Gesellschaft frTechnische Zusammenarbeit (GTZ) GmbH

Impact Monitoringin Value Chain Promotion

Heike Hffler, GTZ Kenya

First sequence of thoughts, prepared for the Training Course

Promoting Promoting Value Chains for Agribusiness

Development in Africa

held in Nairobi, 4th - 7thApril 2005


2/54

2

Impact Monitoring: Conceptsand Application

Measuring Impacts in Value Chain Promotion1. Impact Hypotheses chain of changes

2. Scope of influence

3. Benchmark Data, Milestones and Indicators

4. Methods of Measuring Impact5. Using Monitoring Information

2

A Concept for Impact Monitoring

1. What to monitor? What are impacts?

2. GTZ Impact Model

3. Theory: Six steps to set up a monitoring system

1

Practice: PSDA Impact Monitoring System3

Heike Hffler Kenya


3/54

3






2



2. GTZ Impact Model


1


Heike Hffler Kenya


4/54

4

Monitoring is an ongoing activity!

Projects and Programmes need to follow-up theirprogress

Successes and failures need to be looked atwhile projects and programmes are running

Monitoring is an internalised process of teamcommunication, continuously undertaken whileimplementing,

whereas

Evaluation is an act of stopping implementationto reflect past activities (but drawing frominformation from monitoring).

1

Heike Hffler Kenya


5/54

5

The World of Impact Monitoring

Impact Chain

Impact Model

Impact Indicators

Outputs

Outcome

Outreach

Benefits

Performance Monitoring

Input Monitoring

Activity Monitoring

Output Monitoring

Implementation Monitoring

Impact Monitoring

Impact Assessment

Process Monitoring

Result-based MonitoringEvaluation

Heike Hffler Kenya


6/54

6


Impact Chain

Impact Model

Impact Indicators

Outputs

Outcome

Outreach

Benefits

Performance Monitoring

Input Monitoring

Activity Monitoring

Output Monitoring


Impact Monitoring

Impact Assessment

Process Monitoring

Result-based MonitoringEvaluation

Heike Hffler Kenya


7/54

7


Performance Monitoring Impact Monitoring Impact Assessment

Input Monitoring

Activity Monitoring


Process Monitoring

Output Monitoring

Impact Chain

Impact Model

Impact Indicators

Result-based Monitoring

Outputs ImpactsBenefits

Outreach

Evaluation

Outcome

Heike Hffler Kenya


8/54

8

What to monitor in projects

Performance & Impact Monitoring1 Performance

Observing outputsagainst plannedactivities

Providinginformation forproject management

Day-to-day activity

To trigger short-termadjustments in

operation

Impact

Focussing on effectof outputs: impacts!

Observing the directbenefit of outputs

Strategic steering ofimplementation

To self-evaluatewhether activitiescontribute to

objectives

Heike Hffler Kenya


9/54

9

Impacts are ...

... changes that have a causal - or at least aplausible - link to a project/programme

... a change of circumstances as a consequence of

an intervention, it can be intended orunintended, positive or negative.

... there: from the first moment of intervention andthey continue to occur all the time.

... rather the result of social interaction than astraight-forward interventions

... the result of complex interactions and thus, acomplex matter to deal with!

1

Heike Hffler Kenya


10/54

10

Why concentrating on Impacts?

Broad international discussion late 90ies

Criticism about the efficiency of development

cooperation

Criticism about the Monitoring and Evaluationsystem of GTZ

1

BMZ is a contractor, GTZ is an agent

New structure of the project documents

Changes in political dialogue

Changes in Report obligations

AURA

Heike Hffler Kenya


11/54

11

1

The GTZ Impact Model (I)

Impact Orientation has become a principle of

GTZs corporate development.

quality at entry quality at exit

what has been done what has changed

Heike Hffler Kenya


12/54

ProjectUse of OutputDirect BenefitIndirect Benefit

Engel, P., The Social Organization of Innovation, 1997

Model of Interaction

1

Heike Hffler Kenya


13/54

Use ofOutputs

PovertyAlleviation

Direct

Benefit

(Goal)

Outputs

Indirect

Benefit

Activities

The GTZ Impact Model (II)

1

Partner

Heike Hffler Kenya


14/54

14

Illustration: which of the 100

Bricks did we donate?


15/54

Use ofOutputs

PovertyAlleviation

Direct

Benefit

(Goal)

Outputs

Indirect

Benefit

Activities

The GTZ Impact Model (II)

1

Partner

diff.ImpactLevels

Heike Hffler Kenya


16/54

16

Increase in rural

employment & income

Indirect Benefits

Direct Benefit (Goal)

Use of Output

Output

Activities

Inputs

Producers can access international markets

Market information is used to change production standards

Presentation of Research to stakeholders and publishing

Market research for a strategic agricultural product

Advisory Services in the Ministry of Agriculture

The GTZ Impact Model (III)

1

Attribution Gap

Pro-poor Rural Economomic Growth

Heike Hffler Kenya


17/54

17

Impact Models form the essential methodological part in

project design!

Project reports are reduced to the information relevant for

political intervention, i.e.

Changes in risk

Changes in assumptions

and: IMPACTS! i.e.

Which changes can be observed?

Which impacts can be plausibly attributed to the project?

Are there unintended impacts?

Consequences for GTZ Projects

1

Changes in Monitoring!

Heike Hffler Kenya


18/54

18


19/54

19

Theory: Six Steps (GTZ)

Step 1: Identify the System Boundaries

Step 2: Agree on Purpose and Procedures for

Results-based Monitoring

Step 3: Agree on Results Hypotheses

Step 4: Review Indicators and Define Milestones

Step 5: Conduct Data SurveyStep 6: Using Monitoring Results

1

Heike Hffler Kenya

GTZ 2004: Result-based Monitoring: Guidelines for Technical

Cooperation Projects and Programmes


20/54

20

Theory: Six Steps (Herweg/Steiner)

Step 1: Involvement of Stakeholders and

Information Management

Step 2: Review of Problem Analysis

Step 3: Formulation of Impact Hypotheses

Step 4: Selection of impact Indicators

Step 5: Development and Application of ImpactMonitoring Methods

Step 6: Impact Assessment & Follow-up

1

Heike Hffler Kenya

Herweg, K & Steiner, K. 2002: Impact Monitoring and Assessment,

Vol. I & II


21/54

21

Theory: Seven Steps (MAPP, GDI)

Step 1: Preparing a Life Line

Step 2: Preparing a Trend Analysis

Step 3: Cross-checking with other sources

Step 4: Compiling an Intervention List

Step 5: Developing the Influence Matrix

(connecting trends and interventions)Step 6: Developing the Impact Profile

Step 7: Attribution of Impacts to MDGs

1

Heike Hffler Kenya

DIE 2004: Briefing Paper: Impact Analysis of Development

Cooperation is Feasible


22/54

22






2



2. GTZ Impact Model


1


Heike Hffler Kenya


23/54

23

Impact Models: A chain of changes

What do we expect to happen in value chain promotion?

2

SpecificInput

providers

Primaryproducers Traders

Final Con-

sumers

Logisticscentres,

Industry

Specific

Inputs

Trans-

formationTrade ConsumptionProduction

C H A N G E !

Intervention;

i.e. extension

Trans-

formationTrade Consumption

Specific

Inputs

Heike Hffler Kenya


24/54

24

Specific

Input

providers

Primary

producers TradersFinal Con-

sumers

Logistics

centres,

Industry

Specific

Inputs

Trans-

formationTrade2

Defining the Impact Hypothesis (I)

Consumption

Use of Output Farmer group gets EuroGap certified

Output

Activities Training on EuropGap Compliance and Certification

Inputs Advisory Services for Extension Providers

Production

Direct Benefit (Goal) Small-scale producers access international markets

Farmer Groups are setting up an Internal Control System

Heike Hffler Kenya


25/54

25

Specific

Input

providers

Primary


sumers

Logistics

centres,

Industry

Specific

Inputs

Trans-

formationTrade2

Defining the Impact Hypothesis (II)

Consumption

Use of Output

Output

Activities

Inputs

Production


Heike Hffler Kenya


26/54

26

Specific

Input

providers

Primary


sumers

Logistics

centres,

Industry

Specific

Inputs

Trans-

formationTrade2

Defining the Impact Hypothesis (III)

Consumption

Use of Output

Output

Activities

Inputs

Production


Heike Hffler Kenya


27/54

27

Specific

Input

providers

Primary


sumers

Logistics

centres,

Industry

Specific

Inputs

Trans-

formationTrade2

Defining the Impact Hypothesis (IV)

Consumption

Use of Output

Output

Activities

Inputs

Production


Heike Hffler Kenya


28/54


29/54

29

Impact Hypotheses Food for Thought:

2

Pre-formulation of impact

chains can lead to mono-causal

conclusions.

Linear following of impact

chains can fade out the

complex interaction of

interventions and impacts.

The higher the impact level, themore a context oriented

approach is needed, taking into

account development trends.

Heike Hffler Kenya


30/54

30

Defining the Scope of Influence (I)

Specific single interventions

2

SpecificInput

providers


Final Con-sumers

Logisticscentres,

Industry

Specific

Inputs

Trans-


Heike Hffler Kenya


31/54

31

Defining the Scope of Influence (I)

Specific single interventions

2

SpecificInput

providers


Final Con-sumers

Logisticscentres,

Industry

Specific

Inputs

Trans-


Heike Hffler Kenya


32/54

32

Defining the Scope of Influence (II)

Chain promotion at all stages

2

SpecificInput

providers


Final Con-sumers

Logisticscentres,

Industry

Specific

Inputs

Trans-


Heike Hffler Kenya


33/54

33

Defining the Scope of Influence (III)

Full chain promotion, all stages, all levels

2

Specific

Inputproviders

Primary

producers Traders

Final Con-

sumers

Logistics

centres,Industry

Specific

Inputs

Trans-


Subsector-specific Technical Agencies

Subsector-specific BDS providers

Associations

Local Government, Providers of Utilities / Infrastructure

National Government (Line Ministries) & Public Agencies

Heike Hffler Kenya


34/54

34

Impact Hypothesis - Conclusion

2

Specific

Inputproviders

Primary

producers Traders

Final Con-

sumers

Logistics

centres,Industry

Specific

Inputs

Trans-


Subsector-specific Technical Agencies

Subsector-specific BDS providers

Associations

Local Government, Providers of Utilities / Infrastructure

National Government (Line Ministries) & Public Agencies

Heike Hffler Kenya

2


35/54

35

2

Benchmarks, Milestones & Indicators

Benchmark Data & Information

The state of the art at given point in time (i.e.

programme start)

Milestones

Desired output at a given point in time for (programme)

achievements

Indicators

Determinants for checking whether activities led to the

expected output (performance) or expected

implications (impact).

Heike Hffler Kenya


36/54

36

2

Benchmarks, Milestones & Indicators (I)

Indicators

Heike Hffler Kenya

0t

Benchmark

Data

Programme level

Value chain level

MilestonesProgress

Reports

Evaluation


37/54

37

Specific

Inputs

Trans-

formationTrade

2

Benchmarks in Value Chains

ConsumptionProduction

Usage of

InputsUsage of

services

Outreach:

How manyare

involved?

Employment

Gross

margins

Regionaldistribution

Number of

processorsDegree of

vertical

concentra-

tion and

integration

Market

Concentra-tion

Entry

Barriers

Price

Volatility

Seasonality

Trade

margins

Quality

Food Safety

Product

Diversity

Heike Hffler Kenya


38/54

38

Specific

Inputs

Trans-

formationTrade

From Benchmarks to Milestones


Output

Activities

Inputs

3 processing

firms

Low process

quality

High costs of

processing

In 18 Month:

2 more firms

established

At least 2 certified

under ISO xxx

Cost-effectiveness

of at least 2 firms

improved by x %

Milestones serve as process indicators.

They are directly derived from programme indicators

and the impact hypothesis.

2

Heike Hffler Kenya


39/54

39

Specific

Inputs

Trans-

formationTrade

Benchmarks & Milestones in Value Chains


Try to build these couples per stage in a chain that

you are targeting with your project.

Not all Benchmarks need to be quantitative data;

qualitative description also serves the purpose.

Market Analyses need to take the need for benchmarks

into account (into the TORs!)

We might not be the first to undertake this keep

looking for partners in gathering data!

2

Heike Hffler Kenya


40/54

40

2

From Milestones to Indicators

or was it the other way round?

Output

Activities

Inputs

Use of Output


Production

Heike Hffler Kenya

Several Milestones in a chain

Single Milestone

Programme Indicator(s)

Value chain Indicator(s)


41/54

41

Specific

Inputs

Trans-

formationTrade

Benchmarks, Milestones & Indicators (II)


Output

Activities

Inputs

3 processing

firms

Low process

quality

High costs of

procesing

In 18 Months:

2 more firms

established

At least 2 certified

under ISO xxx

Cost-effectiveness

of at least 2 firms

improved by x %

...

2

Heike Hffler Kenya

Use of Output


Impact Indicator,

Component 3,

Phase 2

The number of

profitably operating

private processors

doubles (2002: 6)


42/54

42


43/54

43

Example: Employment

Customers

Market 2

Customers

Market 1

(ResearchInstitute)

Retailers A

Retailers B

Industrial

SMEs

Importers

Large processing companies

3

2

1Primary

Producers 1

SM Primary

Producers 2

n = 400 n = 450n = 20

n = 6000

n = 40

n = 2400

n = 20

n = 200

3

Heike Hffler Kenya

The number of

primary producers

doubles.


44/54

44

100,00

Specific

InputsProduction Trans-

formationTrade Con-

sumption

Example: Value Added

Specific

Input

providers

Primary


sumers

Logistics

centres,

Industry

Sales Prices / unit 60,0025,00

Cost / unit

Income / unit

Distribution in %

- of value-added

- of income

23,00

2,00

27,00

8,00

30,00

10,00

25 %10 %

35 %40 %

40 %50 %

3

Heike Hffler Kenya

The valueadded at

producer

level

increases

from 25%

to 35%.


45/54

45

Example: Better Business Relations

Heike Hffler Kenya

Specific


formationTrade Con-

sumption

Specific

Input

providers

Primary


sumers

Key informant interviewsshow that wholesale

traders are more satisfied

with the

quality/quanity/reliability

of produce delivered by

small scale farmers.

Interviews show

that small calefarmers have

improved

relationship to

traders and trust

them.


46/54

46

2

Indicators in Value Chains

Some examples:

Customer satisfaction

Repeated customers

Number of enterprises demanding a service

Satisfaction with last service purchased

Percentage of women-owned enterprises

Heike Hffler Kenya

There is increasing interest in, and use of, private

sector tools for performance and impact measurement;

i.e. consumer market research tools for measuring

changes in markets and private sector business tools

for measuring service provider performance.


47/54

47

Methods of Measuring Impacts

2

Heike Hffler Kenya

Market Studies

Key Informant Interviews

Point of Leverage

comparisons (before after) Participatory Monitoring

Time-series

etc...

Specific


formationTrade Con-

sumption

Where to measure in

the chain?

What data qualitative

or quantitative?

Which degree of

participation?


48/54

48

Example: Spider Web Diagrammes

0

5

10

oduc ua

onsume a s ac on

Fa m ncome

o h n xpo s

o e

se o e ces

eg nng ng

a e 3 ea s

a e 10 ea s

2

Heike Hffler Kenya


49/54

49

E-Val

2

Heike Hffler Kenya

GTZ tool to catch different perceptions of projects and

programmes.

There are only subjective opinions to changes and

impacts

Triangulation of opinions:

GTZ,partners, and

target group


50/54

50

Using Impact Monitoring Systems

2

Heike Hffler Kenya

The objective of

Impact Monitoring

was ....

Focussing on effectof outputs: impacts!

Observing the directbenefit of outputs

Strategic steering ofimplementation

To self-evaluatewhether activitiescontribute toobjectives


51/54

51


2

Heike Hffler Kenya

SteeringObserve direct Benefit Self-evaluation

Steering

Committee

Reports

Annual

ProgressReports

Participation!

Discussion with

Stakeholders

Finetuning

Communication

Team

Meetings

Partner

Meetings

Other

donors

Private

Sector


52/54

52


2

Heike Hffler Kenya

More ideas lets be creative!

Replanning

Communication of

Results

Newsletter

Business Fora

Publications

Go- No-go- Decisions

Accountability


53/54

53

Its better to be approximatelyright

than precisely wrong!

( John Maynard Keynes )

Please email your comments & questions to [email protected]

Heike Hffler Kenya

H ik Hffl K


54/54

54






2



2. GTZ Impact Model


1


Heike Hffler Kenya

Documents

Impact Monitoring in Value Chain Kenya