Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities

Screen 1 of 27

TargetingMonitoring and Evaluation of Targeting

LEARNING OBJECTIVES

Understand the basic measures to monitor and evaluate targeting activities.

Use qualitative information to integrate measures of targeting accuracy.

Screen 2 of 27


INTRODUCTION

Good feedback information can be very important in fine-tuning or correcting the targeting during an ongoing operation.

Which types of information and analysis are useful in monitoring and evaluating how well

targeting works in practice?

Food security information systems can make an important contribution.

Screen 3 of 27


INTRODUCTION

KEY QUESTIONS:

• Has the targeting system identified the right people?

• How many of the target group have been assisted?

• Has the targeting system succeeded in concentrating resources on the intended beneficiaries?

• Have any of the resources intended for the target group been diverted elsewhere?

Screen 4 of 27


CAUSES OF TARGETING “ERROR”

In the real world, no targeting system is perfect.The imperfections are called targeting “errors”.

Some potential causes of targeting error:

Technical problems

• Inaccurate needs or vulnerability assessment

• Gaps in data or information

• Poor design

• Flawed implementation

Governance problems

• Malpractice, especially diversion or misuse of resources• Weak administrative capacity• Poor accountability • Poor analysis of power structures and interest groups • Clash of values • Cultural issues

Screen 5 of 27


THREE APPROACHES TO MEASURING ACCURACY

1. How many people were correctly or incorrectly targeted?

2. How much aid reached the right people?

3. Who are the beneficiaries (and the non-beneficiaries)?

Key questions:

Screen 6 of 27


COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?

The accuracy is often measured in terms of how many people are correctly or incorrectly receiving benefits.

The diagram shows how the target group and the beneficiaries may not exactly

overlap

Target population

Beneficiaries (population receiving assistance)

TOTAL POPULATION

CORRECT TARGETING

EXCLUSION ERROR

INCLUSION ERROR

1. How many people?

Screen 7 of 27


Correct targeting is usually measured in terms of coverage.


Coverage is the percentage of the target group who receive benefits.

Coverage and exclusion error are complementary.

TARGET GROUP

Exclusion Error

Coverage

Example – Coverage and exclusion error

Screen 8 of 27



Example – Perfect targeting

Community of 100 people.

20 meet the targeting criteria.

Perfect targeting

Beneficiaries Non-beneficiaries

Screen 9 of 27


Example – Perfect targeting

total Beneficiaries Non-beneficiaries

total 100 20 80

Target group

20 0Exclusion error = 0/20

= 0

Non-target group

80 0Inclusion error = 0/20

= 0 %

80

20Coverage = 20/20

= 100%


Screen 10 of 27



Example – Imperfect targeting (with adequate resources)

Imperfect targeting (with adequate resources)


Total Beneficiaries Non-beneficiaries

Total 100 20 80

Target group

20 5Exclusion error = 5/20

= 25%

Non-target group

80 5Inclusion error = 5/20

= 25%

75

15Coverage = 15/20

= 75%

Screen 11 of 27



Example – Imperfect targeting (with under-resources)

Imperfect targeting B (with under-resourcing)


Total


Total 100 20 80

Target group

30 11

Exclusion error (or under-coverage) = __%

Non-target group

70 1

Inclusion error =__%

69

19

Coverage =__%

Screen 12 of 27



Example – Imperfect targeting (with under-resources)

Imperfect targeting B (with under-resourcing)


Total


Total 100 20 80

Target group

30 1111 / 30

Exclusion error (or under-coverage) = 37 %

Non-target group

70 11 /20

Inclusion error = 5 %

69

1919 /30

Coverage = 63 %

Screen 13 of 27


2. How much aid?

Another approach to measuring the success of targeting is by assessing the quantity or proportion of aid actually received by the

target group.

EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?

Screen 14 of 27



Beneficiaries

Beneficiaries (20)

Target group (15) Non-target group (5)

(25% inclusion error)

Target group Non-target group

Example – Imperfect targeting (with adequate resources)

Screen 15 of 27



Targeting Effectiveness = the percentage of transfer resources actually delivered to the intended target group.


targeting effectiveness

= 75%

Effectiveness Leakage

Effectiveness and leakage are complementary.

Screen 16 of 27


IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?

The 20 sacks have been distributed as follows:

• 18 to the correctly targeted households; and • only 2 to the non-target group.

Example – Imperfect targeting (with adequate resources) - 2


Effectiveness = ?

Leakage = ?

Screen 17 of 27


IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?

The 20 sacks have been distributed as follows:• 18 to the correctly targeted households; and • only 2 to the non-target group.



Effectiveness = 18/20 = 90%

Leakage = 2/20 (or 100-90)

= 10%

…of course, the opposite can also happen...

Screen 18 of 27





?? Missing resources

Effectiveness = ?

Leakage = ?

Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.

Screen 19 of 27





?? Missing resources

Effectiveness = 12/20 = 60%

Leakage = 100-60 = 40%

Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.

Screen 20 of 27



TOTAL POPULATION

TARGET GROUP

Targeting Performance = % of resources received by the target group / % of target group in population.

Targeting effectiveness can be compared with the results of an untargeted distribution.

The accuracy of the targeting can be measured in terms of how much more the target group receive from the targeted programme.

Screen 21 of 27



It is worse than no targeting.

PERFORMANCE < 1

PERFORMANCE = 1

PERFORMANCE > 1

The targeting has had no effect .

Targeting has improved the outcome for the target group.

Screen 22 of 27



Dilution = the amount of aid received by each targeted beneficiary, as a percentage of the planned amount.

Dilution is the opposite of concentration (the aim of targeting).

It is a common targeting problem in which aid is shared among a larger number of people than the programme was designed for.

Screen 23 of 27


BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?

3. Who are the beneficiaries?

The third group of targeting indicators falls under the question of who is included and who is

excluded by the targeting system.

It is very useful to compare the characteristics of the beneficiary and non-beneficiary populations.

If there is a clear difference between them it means that the targeting has been effective.

Screen 24 of 27


BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT?

It is also useful to look at the profiles of people in the exclusion and inclusion errors.

Exclusion Inclusion

Do the excluded target group members have any factor in common that might suggest systematic marginalization?

Who are the people wrongly included among the beneficiaries? Why and how the error is happening?

Screen 25 of 27


THE BOTTOM LINE

Standard of accuracy and targeting priorities are different in different contexts.

Targeting is more accurate in countries with higher incomes, good administrative capacity, and high levels of inequality.

In emergency operations, the priority is often to maximize coverage , while in other situations, the priority may be to eliminate inclusion error and leakage.

Screen 26 of 27


THE BOTTOM LINE

Contextual information and qualitative assessment of how the targeting system is working help to interpret the error measurements in a meaningful way.

• Why are problems or errors happening? What can be changed in order to reduce them?

• What do beneficiaries and their community think about the targeting?

• How are targeting decisions actually made, and by whom?

• How transparent and accountable is the process?

• Does the targeting match the programme’s objectives?

• Could the targeting design have been better?

Screen 27 of 27


SUMMARY

No targeting system is perfect. Usually some degree of inaccuracy must be accepted to make the programme workable and affordable. A judgement has to be made about which kinds of error, and what degree of error, are acceptable given the programme’s objectives and context.

The most useful indicators for measuring the accuracy of targeting fall under 3 key questions:

1. How many people were correctly or incorrectly identified as beneficiaries?

• Coverage = % of target group receiving benefits

• Exclusion = % of target group not receiving benefits

• Inclusion = % of beneficiaries who are not target group members

2. How much of the resources for distribution reached the right people?

• Effectiveness = % of transfer resources received by the target group

• Leakage = % of transfer resources not received by the target group

• Performance = % of transfer resources received by the target group, divided by the % they would have received from an untargeted distribution

3. Who is included, who is excluded?

Documents

Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities