1 of 16 Evaluating an Information Project Building Consensus © FAO 2005 IMARK Investing in Information for Development Evaluating an Information Project

1 of 16Evaluating an Information ProjectBuilding Consensus

© FAO 2005

IMARK Investing in Information for Development

Evaluating an Information Project

Building Consensus


Learning ObjectivesLearning Objectives

At the end of this lesson you should be able to:

identify a framework for analysis of subjects to be included in the evaluation;

be aware of how to carry out a preliminary analysis of risks to the evaluation.


IntroductionIntroduction

In order to consider the issues involved in the planning of an evaluation, we are following a scenario, in which Dr Kumar, the Director of publications at SMAU University, must evaluate a newsletter.

So far, Dr Kumar and his colleagues have:

defined the focus of the evaluation;

developed a list of possible Evaluation Management Committee

members; and

called the first Evaluation Management Committee meeting.


Reaching agreements on the subjectsReaching agreements on the subjects

Here are the questions for the agenda of the meeting:

What will be the main objective in an evaluation of RICE NEWS?

What will be the main subjects to be included?

What is going to be measured?

What is likely to go "right" and what could possibly go "wrong"?


Reaching agreements on the subjectsReaching agreements on the subjects

There are many stakeholders invited to the meeting.

The goal will be to encourage everyone to agree on the basis of the evaluation.

Jon suggests to use a “card technique”...


The NBAO analysisThe NBAO analysis

Let’s divide the RICE

NEWS project into three-four

smaller topics...

Should we use inputs,

activities, outputs and

impacts?


I would rather use simpler categories, like “NBAO”... NEEDS

BENEFICIARIES

ACTIVITIES

OUTCOMES



Here is how the technique works:

• give each member a small pile of cards.

• start with “N” (needs).

• ask everyone to write down the needs RICE NEWS was supposed to address, one need for card.



• then, gather up the cards.

• pin or tape them on a large board, so that everyone can see.

• Dr Kumar can lead a discussion in which people try to group them, and reach an agreement on the main needs, beneficiaries, activities, and outcomes.




Once the meeting has started, Dr. Kumar is ready to use the “card technique”. He explains to the members of the EMC its goal:

This technique will help us:

• to reach an agreement on the subjects we want to include in the evaluation;

• later, as a basis to develop specific questions.



At the completion of the “card technique", here's what the EMC has come up with:



Now, it is important to be aware of what could go wrong with the evaluation before the whole process starts. Dr. Kumar says:

…we should also do a “risk analysis” right at the beginning, to think of things that might go wrong, and

prevent potential problems...


...I propose a SWOT analysis, to think

together about the Strengths,

Weaknesses, Opportunities and

Threats of the evaluation we are

proposing...

The SWOT analysisThe SWOT analysis



What are likely to be the biggest STRENGTHS of our evaluation?

What are likely to be its biggest WEAKNESSES?

If the evaluation goes well, what are likely to be the biggest OPPORTUNITIES that it will bring to us?

And what are the biggest THREATS to it not going well?



After the completion of the SWOT analysis, Dr. Kumar concludes the first EMC meeting.

He and his colleagues have both an evaluation committee in place and a consensus on major issues.

The next step will be to become more specific, and think about what to measure and how to do it.


Summary

•When planning an evaluation it is important to find an agreement on the content of the evaluation.

•During the Evaluation Management Committee meeting, a good starting point can be to examine: Needs, Beneficiaries, Activities, Outcomes.

•To reach an agreement, it is useful to introduce in the meeting a “card technique”.

•The next step will be to use the same technique to do a SWOT analysis.

•After reaching consensus on these major issues, the Evaluation Committee can move on to the details of “what” needs to be measured.

Documents

1 of 16 Evaluating an Information Project Building Consensus © FAO 2005 IMARK Investing in Information for Development Evaluating an Information Project