PM&E toolkit_IT Power

8/7/2019 PM&E toolkit_IT Power

http://slidepdf.com/reader/full/pme-toolkitit-power 1/60

MONITORINGONITORING AND ANDEVALU ATIONVALU ATIONOFOF THETHE

IMPACT MPACT OFOF

RENEWABLEENEWABLE

ENERGYNERGYPROGR AMMESROGR AMMES

AT T OOLKITOLKITFOROR APPLYINGPLYING

PARTICIPATORYRTICIPATORY

APPROACHESPROACHES



2005

Thisprojectwas part f undedby R EEEP



MONITORINGNITORINGANDND EVALUATIONALUATIONOFF THEHE IMPACTPACT OFF RENEWABLENEWABLE

ENERGYERGY

PROGRAMMESOGRAMMES

A T T OOLKITOLKITFOROR APPLYINGPLYING

PARTICIPATORYRTICIPATORYAPPROACHESPROACHES

Written by

Kavita R ai, IT Power

Acknowledgementsto:

JohnGreen,T eresaMarston, Jayantha Gunasekara,Gisela Vogt, Wendy Annecke, and Mila Jude.

Designedand edited by Naomi Davidge

Sketchesby

Emanuele

Scanziani









i

Evaluation An evaluation is the systematic collection of information about activities,characteristics and outcomes of the energy programme or project that may lead to inform and improve the effectiveness of the programme.

Impact Impact is the long-term effect on the stakeholders, institutions andenvironment caused by the energy programme or project intervention. Animpact can be positive, negative, intended or unintended.

Indicator An indicator is an easily measurable criterion that provides information ontrends or changes after the renewable energy programme or projectintervention.

Mapping Mapping is a participatory rural appraisal exercise that providesinformation on the physical or socio economic characteristics of acommunity through maps drawn based upon their own perception.

Monitoring Monitoring is the systematic process of observation and collection of information at the individual household or community level to ascertain

progress or impact of a project or programme.Outcome Outcomes are specific changes in attitude and behaviour, increase in

knowledge and skills of stakeholders resulting from activities generated by the project or programme intervention. Outcomes are often cumulativeresults of an output.

Output The outputs are the direct tangible measurable results or productsdelivered to the intended beneficiaries of an energy intervention.

ParticipatoryRural

Appraisal

Participatory Rural Appraisal (PRA) is a family of approaches, methods,and behaviours that enable people to express and analyse the realities of their lives and to plan, monitor and evaluate their actions accordingly.

SocialEquity/approach

In this toolkit, 'social equity' means keeping in mind 'who' is using andbenefiting from the renewable energy technology programme/project. Allbeneficiaries who contribute towards the implementation of the projectshould be able to access and gain benefits from the programme or project

without any discrimination during or after the process of intervention.

Stakeholder A stakeholder is the person or organisation that has an influence orinterest in the project or programme.

Timeline Timeline is a PRA exercise that provides information on an individual or agroup activity pattern during the course of a day, season or years.

Transect Walk

A transect walk is a PRA exercise where the facilitator/evaluator takes anobservational walk with a small group of local people through a transect(line that is defined by the group or according to interest of the observer)showing appropriate resources, environmental features, peoples' activitiesand interests.

Triangulation Triangulation is a way to confirm datasets through different methods orsources, often conducted to check differing or similar views or changes onthe same indicator.

WealthRanking

Wealth ranking is a PRA exercise used to categorise communities or a

group of people according to their economic status.

GLOSSARY



ii

ACRONYMS

FGD Focus Group Discussion

M&E Monitoring and Evaluation

MDG Millennium Development Goals

NGO Non Governmental Organisations

O&M Operation and Maintenance

PM&E Participatory Monitoring and Evaluation

PRA Participatory Rural Appraisal

RET Renewable Energy Technology

SWOT Strengths Weaknesses Opportunities Threats



1

Why do we need a toolkit?Renewable energy technologies providereliable and affordable energy suppliesto millions of people in developing countries. Timely monitoring andevaluation (M&E) is one way to enableorganisations and communities who

install and implement energy technology systems and programmes tocheck the efficiency and effectivenessof their work. This information canfeed back to governments and donorsto improve the effectiveness of theirfuture programmes.

The common M&E approach used by practitioners of renewable energy programmes is to conduct conventionalrapid assessments or an evaluation atthe end of the programme. Also, many renewable energy programmes areimplemented with an emphasis on thetechnical and financial delivery mechanisms. In the process, there is aneglect of the users’ perspective andactual needs. This neglect results in

minimising the positive impact of aprogramme or project, particularly afterthe withdrawal of institutional support,as many renewable energy technologiesare maintained and managed by theusers or communities. However, thereis a growing recognition that theeffective involvement of people orusers is required to gain positive impact

through a sustained maintenance and

use of the renewable energy systems. This toolkit provides tools to bridge thegap between programme developersand users or stakeholders.

T he content of the toolkit This toolkit describes how to listen tousers, take account of their opinionsand make them active players in theinstallation and delivery of renewableenergy systems. The approachmentioned in the toolkit intends to helppractitioners and donors alike to forgestrong 'partnerships' with users ratherthan let them remain only as'recipients'.

The content of the toolkit is nottechnology specific. The toolkitprovides a clarification of participationin monitoring and evaluation processes,design of an M&E system, and theusage of participatory tools and

methods. The toolkit addresses, inparticular, how social equity can beincluded in the M&E process.

The participatory approach isqualitative, so this toolkit does notcover quantitative methodologies thatmay also be useful as part of the M&Eprocess.

Who are the potential usersof this toolkit? The toolkit is designed to be used by both government and non-governmental agencies andcommunities involved in theimplementation of energy projects. Itcan help field technicians and social

mobilisers in the field to competently

SECTION1ECTION1INTRODUCTIONTRODUCTION



2

monitor the project or programmeactivities and build up a wide range of data. Different users have differentinterests that require differentinformation from the M&E process.Individuals or organisations can pick ideas from this toolkit to plan an M&Eprocess tailored to their own individualcountries and regions. Alternatively,some of the tools can be combined

with quantitative methods, such as thesurvey, to enable the triangulation of data or information.

A thorough use of the participatory methodology in an M&E system willenable the practitioners and users aliketo review and overcome any hurdlesduring the implementation, rather thanface it all in the evaluation phase.

R enewable energy projectsconsidered in this toolkit

The focus of this toolkit is on thefollowing types of renewable energy technologies:

Isolated off grid systems (microhydro, wind, solar).

Other renewable energy technology options/systems based

on biomass (biogas, improvedcook stoves), wind and solar for water pumping.

Box 1: Monitoring technicalcomponents of the project/

programme

It is necessary to monitor andevaluate both the technicaland socio-economic aspects of a project interventionsimultaneously. TechnicalM&E needs specifictechnology related indicatorsand technical experts.

To use a participatory approach in the technicalM&E process, a list of actionsand plan can be made for thelocal operators or householdowners to generate data onthe technical components of project installation/s. Groupdiscussions and stakeholderanalysis can be conducted

with the owners, technicaloperators or managementcommittees.



3

Approach used in the toolkit The toolkit uses a participatory approach to M&E. It also focuses on

the issue of social equity and itsintegration in the monitoring andevaluation process. Social equity inessence looks at the progress orimpacts of the renewable energy programme intervention based on race,ethnicity, gender or economic groupdifferences. It is an approach to findout whether justice is done to all groupsof people, and that the benefits are notreaching only one section of community and not the other.

The combination of using participatory methods and social equity will provide

an impetus to gain in-depthunderstanding of the social relations inthe communities and help programmeimplementers to focus on where thebenefits should go or are going. Using this approach will confirm that theproject impact focuses on differentgroups, especially the poor and

vulnerable members of the community who may otherwise be left out if theprogramme or projects are technical orfinancial in nature.

Section 1 and 2 are introductory andsection 3 focuses on understanding theparticipatory and social equity approachto monitoring and evaluation. Thefollowing three sections deal withmethods, indicators and assessment of results.



4

Why is monitoring andevaluation impor tant?Before defining monitoring andevaluation, let us briefly look at theimportance of the monitoring andevaluation process. Just as a good chef carefully puts together a variety of ingredients for different courses of ameal, a good project planner has tocarefully plan activities for the differentphases of the project or programme.Monitoring and evaluation is oftenconsidered a 'side dish' and is notplaced in the main menu in theplanning of renewable energy

programmes that are technology driven. However, one does not oftenrealise that these same ingredientsshould in fact come from the core of the M&E process.

Each phase of a programme willpresent an opportunity to fulfildifferent needs and expectations. Eachphase must also link in to the next andcumulatively produce a range of outcomes/impacts specific to the RETproject or programme. To carefully select who will participate and benefit,

what the needs and fulfilment of various stakeholders are, how theproject will be carried out, or when thedifferent activities will take place, are allessential 'ingredients' needed for aproject to be carefully planned,

monitored and evaluated. Theseingredients are also the 'indicators' thatconstitute the monitoring andevaluation of each renewable energy project. Before we go on to indicators,let us look briefly at the meaning andpurpose of monitoring and evaluation.

MonitoringProject planners and implementing agencies should differentiatemonitoring and evaluation into twodistinct components in the planning process. Monitoring is different fromevaluation. Monitoring should befrequently and regularly conducted and,in fact, can be referred as an 'on going'evaluation.

Monitoring is a systematic process of observation and collection of information by implementing agenciesat the individual household or

community level to ascertain progressor impact of a project or programme. Itis also a process for critical reflection.

The information generated frommonitoring activities will provide

valuable clues on the occurrence of problems, the success or failure of some aspects of intervention, and theareas of improvement. Theimplementing or donor agency should

SECTION2ECTION2MONITORINGNITORING&&EVALUATIONALUATION

It is extremely important that professionals trained in renewableenergy issues conduct in-depth baseline studies. Thoroughbaseline information will provide the best indicators to measure progress and change.



5

know through the monitoring processif the aims and objectives of theproject are being met and changesmade accordingly. Monitoring is alsoparticularly important if there is nospecific baseline information orassessment. A regular monitoring process will enable an evaluation to beconducted with accuracy and in arelatively short time.

FIGURE 1: The Project Cycle

Evaluation An evaluation focuses on theeffectiveness and relevance of therenewable energy project orprogramme to the communities or

household. An evaluation is thesystematic collection of informationabout activities, characteristics andoutcomes of the energy programme/project that may lead toinform and improve the effectivenessof the programme. Evaluation is oftennot a continuing process.

It is better to conduct two evaluationsduring a programme or projectimplementation. The first one could bea mid-term evaluation, conductedideally a year or two after the project

construction, to assess the possibleimpact. The most important andrigorous process of evaluation wouldbe the second 'impact assessment' thatshould be conducted a few years afterthe project implementation. An

evaluation buildsupon the monitoring data and the two can

be seen as mutually dependent. Without aproper monitoring system, it is oftendifficult to conduct acomplete evaluation.

A consistent set of monitoring data canlead to an effectivesummarisation of theprogress made. Afteran evaluation, theproject implementersmay decide to take the

project further, replicate effortselsewhere or stop the project. Both themonitoring and evaluation need toinclude the main stakeholders,particularly the users of the technology and the implementing agencies.

Budgets for M&E processes arenot integrated properly in project planning.Make sure that both the processes are adequatelybudgeted.

BASELINE

Late Evaluation/Impact Assessment

Planning / Setindicators

Mid-termEvaluation

ProjectImplementation/Monitoring

Review/Plan for next phaseCompletion /Exit





7

and effects. It provides a platform forusers and project implementers to deal

with the negative or unanticipatedevents and strengthen the positiveaspects.

Box 2: Basic ground rules for M&E

Completeness: Create goals and objectives, set indicators, set time plans, andinteract regularly with stakeholders, particularly beneficiaries, from beginning toend of the programme or project.

Command: Establish basic command and knowledge of the geographical area,stakeholders and the aspects you need to monitor and evaluate.

Commitment: Commit to the process by setting personnel and funds aside forM&E in your planning, implementation and exit strategies.

Consistency: Consistency and more consistency: Be consistent in your approachat all stages of M&E. Data needs to be derived using the same units of analysis,and preferably with the same stakeholders. You need to follow this rule even forqualitative exercises. “Less”, consistent data, is better than “more”, inconsistentdata. It is probably the most important ground rule.



8

Understanding par ticipationin develo pment & renewableenergy inter ventionParticipation is not just a fancy word in

development but if used well can proveto be a powerful tool. It means bringing the knowledge and real needs of peoplein to planning the intervention of specific projects or programmes. Itgives importance to the perspective of the 'user' and not just the 'provider'.

Participation does not entail the soleinvolvement of users in constructing ahydro project or installing a home solarsystem. It is not a 'one time activity' buta continuous one. Most importantly, theproduct of the participation processshould lead to the empowerment of allstakeholders, particularly thebeneficiaries during and after theprocess of intervention.

The following are the commonprocesses of participation found in theimplementation of renewable energy projects:

1. Communities identify resources andconduct initial project planning anddesign. This is a rare practice, but itis used in Sri Lanka wherecommunities who want to install a

micro hydro scheme in their villages

identify first the water resource,measure water flow and head, thendesign the distribution based on theirknowledge about the villagegeography and locations of houses.

2. Full or partial labour contribution by the users or beneficiaries during theinstallation of the renewable energy systems.

3. Share of cost and maintenance by thehouseholds or community membersinvolved in the installation of therenewable energy system/s.

4. Project management committeesformed by local community members' who organise themselvesformally and regulate the functioning of the system. Alternatively, in moreadvanced projects, all the localcommunity members, both men and

women, take a pro-active role.

A par ticipatory approach tomonitoring and evaluation

A Participatory Monitoring and

Evaluation (PM&E) approach involves

SECTION3ECTION3PARTICIPATORYRTICIPATORY

MONITORINGNITORING&&EVALUATIONALUATION



9

the stakeholders expressing their viewsand knowledge to support programmeimplementation. The purpose of thePM&E process can be two fold. First,to involve people in project planning and implementation, rather than just be'recipients' of the technology. Second,assess the level of outcomes andimpacts of the project or programme inassociation with users and stakeholders.

Using a participatory approach, thecommunities and project implementersshould be able to continuously improve

their management styles, upgradefacilities and provide feedbacks to eachother or to other stakeholders such asdonors or manufacturers. Therefore, aparticipatory M&E becomes successfulif the people who are involved in theenergy project or programme own theprocess.

Participatory processes can include various types and degrees of involvement, control over, anddecision making during the energy intervention process. It can lead notonly to an effective implementationof an energy project but to a degreeof empowerment and socialtransformation.

Basic elements of Par ticipatory Monitoringand Evaluation

A participatory approach is not a 'single'approach. There are two ways to take itfurther. In the first, the owners andusers need to take ownership of theprocess by developing their own actionsand create a favourable decision making

process. In the second, institutions thatare implementing renewable energy systems in association withcommunities or entrepreneurs useparticipatory methods extensively toinvolve communities in the variousstages of PM&E. The first approach isan intensive process that may take mucheffort, dedication and learning from theusers and stakeholders. The secondapproach is widely used and can besubjective. For example, many practitioners consider a participatory approach to be 'conducting a PRAassessment'. However, a PRA is only one of the various methods to gaininformation.

This toolkit focuses on the secondapproach. It will include how monitoring and evaluation can beparticipatory and consider the users'perspective. Although participatory methods are purely qualitative, theM&E process does need indicators to

measure progress. The indicators put

Five general functions of PM&EProject management and planning

Impact assessmentOrganisationalstrengthening or institutional learning

Understanding andnegotiating stakeholder perspectivesPublic accountability

Source: Estrella and Gaventa(1998)





11

Using an equity oriented par ticipatory approach This toolkit differs from other manualsand toolkits in that it uses a social equity oriented approach. If your goal is to

achieve high positive impact on poverty

reduction, the inclusion of social equity as a central concept within theparticipatory process is necessary.Renewable energy projects need tofocus on 'people' because if the users donot understand the implications of itsusage, it might create the loss of financial investments and failure of thetechnology intervention. Often,technicians who perform feasibility assessments can get into the trap that'communities' are all homogeneous.However, there are many variations thatare specific to country, regional orlocal areas.

Box 3: Principles of Participatory Monitoring and Evaluation

All stakeholders participate in the M&E processes. Under anequity-oriented approach, participation should include all groups,men and women without any discrimination.

All stakeholders work to develop indicators, clarify expectationsand priorities, and build ownership of outcomes.

All stakeholders need to inform each other, particularly if thereare problems or issues. It will encourage an increase intransparency and accountability.

All stakeholders, particularly the users, negotiate on the process,collection and analysis of data and the actions that will follow the evaluation.

Stakeholders, including users, learn from the PM&E process toown the process and have an influence on improving actions.

Flexibility is essential as the process entails constant lessonlearning, negotiation and ongoing change.

Participation

Stakeholderinvolvement

Informationsharing

Negotiation

Learning

Flexibility

Source: Modified from Blackburn et al, 2000

A participatory evaluation needs toinvolve stakeholders, especially thebeneficiaries in the evaluation process. It is also important that the project or programme evaluators play the role of a facilitator.



12

Unlike other traditional processes, afocus on 'social equity' means keeping inmind 'who' is using and benefiting fromthe renewable energy programme orproject and how the intervention caninfluence results that provide benefits tothose who need it. All beneficiaries whocontribute towards theimplementation of the project shouldbe able to access and gain benefitsfrom the project without any discrimination during and after theprocess of intervention. Social equity needs to be considered in renewableenergy projects because mostexperiences show that projects oftenfavour the richer sections of thecommunity and remain within thedomain of the men.

There is a common misinterpretationthat using participatory processes willautomatically generate 'equitable'

benefits. A participatory approach oftenmobilises the stronger members of thecommunity. The interests of the richerpowerful sections of the beneficiariesmay dominate meetings. In an equity-

oriented approach, participation of allmembers of the community or a widerrange of stakeholders needsencouragement. The inclusion of

women in decision making, forexample, is an important indicator of using a social equity approach.

Planning and designing thePM&E A participatory process can lead tobetter negotiation processes betweenusers and suppliers. Therefore, if you

want the PM&E process to besuccessful, you have to take intoconsideration time and careful planning,preferably with the communities, for theprocess to be owned by the users.

The planning stage is crucial in any PM&E process. In a participatory approach, all personnel involved in theproject or programme needs to play anactive role. The planning process shouldinclude discussion between stakeholdersand programme planners on setting

possible impact indicators as well as a



13

timeline for different activities. Allproject or programme personnel andusers/stakeholders need to be aware of their roles and responsibilities in thePM&E process. The whole idea of aPM&E system is to get local peoplecompletely involved in the process sothat the intervening actors and agenciesbegin to phase out their involvement by the end of the project. It is alsoadvisable that an organisation has adedicated person for M&E activities. It

will allow for a clear strategy and follow up of the problems and issues withinthe project.

Deciding on content of PM&E:ProjectactivitiesFor a PM&E to be successful, it isimportant that project implementersdiscuss M&E activities and plan withusers at the beginning of the project.Designing an M&E needs a carefulanalysis beforehand of what type of information is required from theprocess. Partners and local people needinformation about potential activities.In a normal M&E process, the 'end' ismade clear to 'derive the means'.However, in participatory processes, itis often difficult to determine the 'end'by itself.

It is extremely important to keep inmind that there could be a'participation overload' for thecommunity members. Therefore, plan and take consent from users/communities or beneficiaries of the programme.

To include social equity in amonitoring system, it is essential to generate disaggregated data from the project initiation phase.The results will allow the project or programme implementers to actupon any inequalities that arise

between men and women or between rich and poor. There is aneed to create social equityindicators to effectively monitor progress and benefits from therenewable energy intervention for all groups.



14

P a r t i c ipa t o r y E x e r c i s e1 -a r t i c ipa t o r y E x e r c i s e1 -P l a n n i n g at t r i b u t e s f o r M& El a n n i n g at t r i b u t e s f o r M& E

Points for discussion on management and maintenance issues during initial M&E planning with community members.

Ensuring that the community is participating and providing result is an important aspect of the PM&E process. To do this in a participatory context, make small slips of paper with the attributes as shown above and give them to small groups of 6-7 persons. Each group should sort out the top three or four attributes that they feel

are most important to them. Then, they should be able to explain in the larger groupthe reasons for doing so. Once the different groups have explained their choices, there can be another session to take this forward and develop measures to improve on them or take action for the next step. Women and the poor should be included in the

participatory processes by first forming different groups depending on the cultural elements of the communities you are working in.Similar exercises can be carried out in different focus groups using different topics.

An example can be the access and use of energy resources by male and female members. Remember that each exercise should be based on providing feedback to the

M&E process by discussing indicators, obstacles, problems and lessons.

MANAGEMENT MAINTENANCE

Leadership

Sense of responsibility

Planning skills

Ability to participate inconstructive group discussions

Capacity to decide onmaintenance of renewable energysystems

Ability to work well onManagement Committees

Women's role in decision makingregarding meetings, technologyassessments

Land and water ownership relatedto implementation of therenewable energy schemes

Willingness to provide labour

Technical know how

Willingness to pay for basicservices

Repair parts

Constraints and ability toovercome maintenanceproblems

Distance to repaircentre/markets

Users participation inmeetings

Continuing role/benefits fromenergy services to women

Possible conflicts andresolution mechanisms

Time constraints



15

Decidingon time,personnel,skilland r esourcesIf the programme developers want tolearn lessons and target for achievablepositive impacts, PM&E activities fordifferent stages of the interventionneed to be budgeted at the initial designphase of the project or programme.Often, the budget for M&E is non-existent, under funded or only meantfor a final evaluation. A proper budgetallows timely M&E activities to takeplace and assign clear responsibilitiesfor individual project or programmemembers.

Participatory exercises can consume alot of time of the community members.

Therefore, one should avoid highly productive seasons (such as harvest orplanting time if it is an agrariancommunity) or busy period during theday. Make sure that participants areaware that you will use a certain 'numberof hours' for the exercise and set aconvenient meeting point.

Project managers should ideally beresponsible for the activities if there isno specific person responsible forPM&E activities. However,participatory approaches may require

the expertise of a trained person. It isimportant that the resource person/sconducting the PM&E exercises beequally well versed in renewable energy interventions or the data that isrequired for the assessment. All teammembers need training in participatory approaches to conduct exercises withthe community. Teams need to rehearse

the exercises to give communities a

sense that 'you know what you aredoing'. For larger energy projects, it ispreferable to train social mobilisersfrom the local area in using participatory tools so that they caninteract and help thecommunities/users to monitor theirprogress in terms of implementationand in the final evaluation.

Team members need an allocation of duties on 'who should conduct what'.Observation and documentation of theprocess is essential for information and

analysis. Make sure that the teamprepares beforehand all items for theparticipatory exercises. The teamshould also be prepared to conduct the

various components of analysing energy issues and be able to provideanswers to various queries that thecommunity might have.

Determining indicators for PM&E The most important step in an M&Eprocess is determining the indicatorsfor the process as well as theoutput/outcome. An indicator is aneasily measurable criterion thatprovides information on trends orchanges after the renewable energy

programme or project intervention.Indicators should also be able to clearly link to the overall programme orproject goals and users aspirations.

Therefore, the key role of an indicatorin the PM&E process is to providespecific pointers in gathering the typeof data that will meet the objectives of the programme/donors and that of the

beneficiaries and stakeholders.



16

Discuss the difference in the situationthat you want to change or achieve overthe number of project or programmeyears. These will be the impact indicators

(see Section 4).Discuss how you can achieve theproposed impact. Such indicators willhelp you develop process indicators .Examples can be number of localpeople trained in the renewable energy technology intervention, number of participatory meetings and number of male and females participating inmeetings.

If it is a whole programme you want tomonitor and evaluate, indicators for efficiency targets can be helpful. Examplesinclude training conducted by month/year using 40 hours of staff time, electricity to be provided to usersby month/time and so on.

This toolkit will focus on the first twoindicators. Process and efficiency indicators will vary from one project orprogramme to another and needsdiscussion amongst stakeholders,particularly the implementing organisation.

Out put vs. Outcome/Impact The outputs are the direct tangiblemeasurable results or productsdelivered to the intended beneficiariesof an energy intervention. They couldbe the electrical units or householdsbenefiting from the projectintervention, number of peopleemployed, and so on. While some

outputs may be direct, such as numbers

If you use the PM&E approach,indicators are set or discussed with theusers in the planning process. Anindicator is useful if it is measurable,

precise and consistent. Both the usersand the project implementers should beable to understand it in the same way.

The lack of indicators will limit themonitoring and evaluation process.

Following are some basic steps todevelop indicators with differentstakeholders:

Identification of the social, financial orinstitutional situation that you want tochange after the renewable energy intervention. Examples includeincrease in income, reduction in healthexpenditures and increase inparticipation of women. This will helpyou develop baseline indicators (seeSection 4). Baseline indicators are the

measures taken before the programmeintervention and is used as a starting point to track changes and trends.





18

Content OutputIndicators

(examples)

Outcomeindicators

(examples)

Methods 1 Informationsources

EnergyServices

- Access/ usageof RE services(electricity,cooking,heating)

- Access tohealth/schools

where REprovide services

- Changes anddifferences in theaccess to energy services

- Economic benefitsof fuel savings(differences inshare of household expenseresulting from fuelswitching)

- Changes to healthafter fuel switching

- Resourcemapping

- FocusGroupDiscussions

- Stakeholderanalysis

- Household

interviews

- Users of theenergy technology

- Stakeholders- Management

committee- Local

technicians

Effectivenessof themanagement

- Number of meetings held

- Number of male/femalemembers incommittee

- Number of problems solvedby committee

- Consistency rates of managementmeetings

- Timeliness of electricity/ energy provision

- Frequency of tariff collection

- Frequency of rulesand regulationsusage

- Changes inmanagement withincorporation of

poor and women


- Householdinterviews

- SWOTanalysis

- ManagementCommitteemembers/owner/s

- Users

Effectivenessof training

- Number of trainedpersonnel

- Number of technicalproblems solved

- Increase in numberof quality servicesprovided by thetechnician


- Interviews

- Technicians(local)

- ManagementCommittee orowner/s

TABLE 4: Indicators in relation to output and impact/outcome

Note: These are only examples. If you were to use a participatory approach fully, you have to set indicators with the users/stakeholders.1 See details in Section 4



19

How do you approach acommunity?Step 1: Planning

Approaching any community needsplanning. Be cautious when you firstcontact any community or user groups.Before you visit, learn about thecommunity and make sure that you willget an audience by choosing the righttime to conduct participatory exercises.It is important to familiarise yourself

with the activities of otherorganisations working in the sameproject or programme area. They couldprovide important information.Contact the local leaders andgovernment officials if you needpermission for the visit. This isespecially important in countries thathave high security risks. Discuss overallissues and get a feeling for what you

would need to expect in your dialogue with the communities.

Be aware that participatorymethods:

Can lead to repetitive pre-defined answers byrespondents if participatoryexercises are conducted more than once. Are subjective, for both therespondent and the evaluator.Will generate highlyqualitative data that are sitespecific and cannot begeneralised to a larger contextor area.Can only be a 'tool' and willnot bring you instant results.Interpretations may need to be triangulated with other conventional methods such asa survey or pre-existing data.

Participatory exercises or researchmay have been carried out byothers in the same area before you.So, make sure that you find outabout this and adapt accordingly. Be prepared to tackle antagonismand conflict.It is important that there are atleast two people carrying out the participatory or interviewexercises. In this way, there can bea continuous exchange of information without distraction.

A series of participatory techniques ormethods can be used in a participatory assessment. It is important that thereare trained personnel, especially withregard to interactions withcommunities, for participatory methodsto be successful. You should rememberthat these methods help to organise andextract information from the people.

Figure 2 on page 20 shows a step by step process to using the participatory approach.

SECTION4ECTION4PARTICIPATORYRTICIPATORY

METHODSTHODS



20

FIGURE 2: Step by step process to use the participatory approach in theM&E process

- Review literature, contact people- Familiarisation visits- Conduct stakeholder analysis- Prepare materials and plans for participatory

exercises

1. T ransectWalk Initial rough mapping exercise, familiarisation walk

2. Mapping,Wealthr anking with differentgroups

3. SeasonalMappingsand T imelines4. FocusGroupDiscussions

Problem ranking, technology interest / issuesranking, discussion of indicators

5. ConductPersonalinterviews(can draw personal time lines)

6. Call groupmeetings: Discuss plans andactions

________________________________________ 7. During M&E, review the above, especially the

indicators. If no baseline exists, start similar process and spend more time drawing up andreviewing the indicators.

- Record all the data- Triangulate results- Prepare initial reports

Monitoring- Review with users/stakeholders- Formulate action plan incorporating changes

with users/stakeholders- Write reports/ Implement changesEvaluation- Review and analyse results of impact and

strategies- Formulate an exit strategy with users/

stakeholders

Step 1Step 1PreparationPreparation

Step 3Step 3Repor tingRepor ting

Step 4Step 4 Action plan Action plan

or exitor exitstrategystrategy

Step 2Step 2Field workField work

usingusing par ticipatory par ticipatory

methodsmethods



21

Step 2: Assess local resourcesLook to leaders or others who willpotentially help you in yourassessments or activities. A quick walk through the area will allow you to gainan overall understanding of the localinfrastructure, resources and thecommunity members. Get their initialideas and set a time for meeting that iscomfortable for all members, men as

well as women, to participate. You canstart the meetings by mentioning someof your initial impressions, and then

relate it to the energy assessment.

Step 3: Hold meetings to provideinformationDepending on the different stages of the project intervention, explain themonitoring and evaluation processes tothe users/stakeholders and find outhow and when they can participate. If you are approaching a community forthe first time, one of the icebreakers isto take some demonstrativeinformation materials with you. Anexcellent example is to takedemonstration toolkits of thetechnology (for example solar or low

wattage light bulbs), large posters, videos demonstrating the technology

(can show with a solar powered system)and leaflets. With direct and visibleinformation, the communities will gainknowledge about the technology. Thecommunities/users can discusspractical needs and their own roles inthe installation of the technology andthe M&E process after deriving information from the demonstrative

materials.

When you hold participatory meetings with the community, introduce thetechnology, and link it to their ownneeds. In the participatory exercisetables 2 and 3, you will find a possible

Provide appropriate information on efficient use oenergy to avoid this situation!



22

list of discussion points on energy access, usage and needs. You can alsoassess the capabilities and the ways of

working in a community by discussing the management of other resourcessuch as water for irrigation or drinking purposes. The project team membersshould discuss any conflict issues orpotential ones amongst members of the community at the initial projectstage. Once the community or users arecomfortable with the information, youcan make an action plan with them todetermine the various stages of theM&E process. If you are approaching acommunity midway or at the finalstages of the project, you will still haveto go through the same process.Remember to keep notes of all themeetings and participatory exercises touse in the next phase or meetings.

Step 4: Record all information for further planningOnce you have information of energy access and usage, make it a point torecord all the information. This will beyour baseline figure. Crosscheck any inconsistencies while you are still in thefield. Use the same data again the nexttime you hold a meeting during themonitoring process. Once you have setup the project, some of the following can be the discussion points andsubsequently the data for monitoring

the input of different forms of renewable energy use. If thecommunity management find theprocess helpful, they can conductsimilar exercises within their ownmeetings.



23

Part icipatory E xercise2 -art icipatory E xercise2 -Possiblepoint t o d iscuss r egardingossiblepoint t o d iscuss r egardin g

energy access,supply and d emand i n t henergy access,supply and d emand i n t hearea/householdbeforepro j ect i nterv entionrea/householdbeforepro j ect i nterv ention

TYPE OFFUELS

DISCUSSION POINTS

Kerosene orParaffinDiesel

LPGCoalCandles

Nearest available supply centreServices of supply: regular or irregularCost of the commodity

Approximate usage (in units/month)Constraints (if any)

Batteries Type of appliances using batteriesUsage (in numbers and hours) and costs per monthNearest available centre

Fuel Wood Types of wood

Usage in bundles/month (preferably in weightcategories)Distance for collectionRate of fuel wood if bought per bundle/weightConstraintsResponsibility of collectionSeasons for collectionDifficulties over the past 2 years

Crop residue Types (maize, rice, millet, barley, others)Usage in bundles/month (preferably in weight

categories)Distance for collection/responsibilityConstraints over the past 2 years

Animaldung/charcoal

Produced by households or suppliedAverage number of livestock per householdUsage of the charcoal/dung (preferably in weightcategories where appropriate)Costs of the livestock/charcoal if bought or soldConstraints, if any

F o s s

i l f u e

l s

R e n e w a

b l e

E n e r g y



24

Part icipatory E xercise3 -art icipatory E xercise3 -Possiblepoints t o d iscuss w i th r egard t oossiblepoints t o d iscuss w ith r egard t o

specific t echnology f or baseline and M&Epecific t echnology f or basel ine and M&ETECH-NOLOGY

DISCUSSION POINTS FOR BASELINE

DISCUSSION POINTSIMPORTANT FOR M&E

Solar Sunlight hours (seasonal charts)Nearest available supply centrefor energy/technologyCost of the energy services:Willingness to payPotential usage and benefits:households, communal battery

charging, health centresConstraints, if any

Approximate usage (inunits/month)Running costsMaintenance and repairConstraints or problemsBenefits

Wind Days with sufficient wind(seasonal calendars)Potential usage andbenefits/constraints

Approximate usage (inunits/month)Maintenance and repairConstraints of benefits

Biogas Number of livestockAvailability of animal waste andwaterCost of the commodity/willingness to payNearest available supply centreConstraints and possible benefits

Usage of the biogas(users/time)Cost of the installationAvailability of animal wasteand waterAvailability of maintenanceand repairsConstraints and benefits

Improvedcook stoves

Availability of woodUsageAffordabilityConstraints and foreseen benefits

Usage (type/weight)Fuel wood savingsConstraints and benefits(such as smoke reduction)Maintenance and repairfacilities

Hydro Water resources (distance,

ownership, ownerships, seasonalflow variations)Potential usage andbenefits/constraintsMaintenance and repairManagement

Usage (numbers and hours)

and costs per monthManagement of the plantMaintenance and repairConstraints and benefits

Gridservices

Distance from nearest gridHousehold needs for electricityCost and willingness to payConstraints and possible benefits

Usage for householdsUsage for productiveenterprisesContinuing affordabilityConstraints and benefits



25

Stakeholder meeting andanalysisParticipation of stakeholders is the key to the success of any project orprogramme. A stakeholder meeting provides a good opportunity fordifferent 'voices' to be heard, and todiscuss the issues that arise before,during and after projectimplementation.

Who is a stakeholder? A stakeholder is the person ororganisation with an influence orinterest in the project or programme. Astakeholder is also the one affectedpositively or negatively by projectactivities. Stakeholders can be membersof organisations involved in the projectsuch as donors, NGOs, government(national, regional or local), installersand manufacturers, energy utilities,distributors, political parties or any suchperson or institution that is involved inthe project. Representatives from thecommunities, male and female, poorand rich or leaders involved in the

Individuals or groups affected directly by the project intervention

Actors involved as intermediaries in the project such as members of NGO’s, utilities, installers etc

Actors not formally involved in a project but whose actions anddecisions may create an impact on the decisions and activities of theproject or programme. Donors or governments (if they are not theimplementers) may fall into these categories

Primary

Secondary

External

Box 4: Types of stakeholders

project are all different types of stakeholders. Remember that eachstakeholder has a different role andfunction in the renewable energy intervention.

When do you conduct astakeholder analysis?

A stakeholder analysis can be conductedat all stages of the project orprogramme (see figure 3). In theplanning/baseline phase, it will helpdetermine the interest and influence of the main stakeholders in theprogramme. While interest refers to thestakeholders' particular concerns andstake in the project itself, influencerefers to the way in which stakeholderscan take control or have influence indecision-making processes. If differentstakeholders are involved in aprogramme, it will be useful to monitortheir actions intermittently to sum up aneffective lesson learning evaluation of the project or programme. One doesnot have to bring all stakeholderstogether for a stakeholder analysis.



26

Stakeholder consultation process The stakeholder analysis needs an

expert facilitator and other helpers tohelp him or her with the process of taking notes and reporting. Stakeholder

workshops should ideally be a one-day session and need to be prepared wellfor it to be successful. Cards and boardsfor participatory exercises areimportant in stakeholder discussion formembers to come up with differentideas and explanations. Issues relating to equity and gender needs should bediscussed specifically in a stakeholdermeeting. This will ensure institutionalparticipation and support for theproject. The session can also help insensitising them towards these issues. Itis important that there is a presence of

female members of the community as well as representatives from differentclans or ethnic groups and poorer

members.In a stakeholder meeting, community members can provide feedback on theinstitutional support they received. Thisis the main reason why community members need to be participants in the

Strengths, Weaknesses, Opportunitiesand Threats (SWOT) for eachstakeholderFind out the varying interest andpotential impact of the project ontheir interestsFind out the varying influences suchas power, control over resources and

relationshipsSetting and achieving indicators setfor programme/projectExpectations and needs from theenergy projectInvolvement/feedback in differentstages of the M&E process

Action Plans (inputs, participation,resources)

Box 5: Points to discuss in astakeholder meeting/analysis

Stakeholders’Interest/Influence Action Follow-up Result

BASELINESELINE

MONITORINGNITORING

EVALUATIONALUATION

FIGURE 3: Stakeholders involvement at different stages of project or programme



27

stakeholder meeting. It is very important that participatory stakeholderassessments be conducted in the locallanguage or dialect. Alternatively,

translators can be sought if there are afew members who do not understandeach other. Group discussions withinsuch meetings need to be planned tofacilitate the presence of appropriatemembers in each meeting.

Strengths, weaknesses,oppor tunities, and threats(SWOT) analysis

A SWOT analysis is an effective andsimple planning, monitoring andevaluation technique that addressesaspects of the strategic planning andimplementation processes. A SWOTanalysis can help in two ways. First, thisanalysis helps in identifying what thestrengths and weaknesses of the projectare for different stakeholders (may include project partners, communitiesincluding men and women,entrepreneurs, manufacturers and soon) and what kind of opportunities andthreats exist for each group of people.

Par ticipatory R ural AppraisalsPractitioners often think thatconducting participatory ruralappraisals (PRA) are the means andends of a participatory approach.However, a PRA is only a family of approaches, methods, and behavioursthat enable people to express andanalyse the realities of their lives andconditions, to plan themselves whataction to take, and to monitor andevaluate the results (Chambers andBlackburn 1996). In the appraisalprocess, the knowledge and views of local people are provided high priority.PRAs are semi structured and are bestcarried out by an interdisciplinary team.It is also often used as a quick, cheapand flexible way of gathering information. One of the mainobjectives of PRA is that users may

learn from the process and carry outsimilar exercises on their own. Some of the common tools used within a PRAare games, diagrams, maps, calendarsand small group interviews.

If different stakeholders holddifferent views and interests, youhave to analyse all theinformation and negotiate.Community needs and interestsshould always be given first preference as they are the primarystakeholders.



28

Part icipatory E xercise4 - W orksheetf orart icipatory E xercise4 - W orksheetf orS W O T analysis f or plannin g aW O T analysis f or plannin g a

renewablee nergy progr amme o r pro j ectenewablee nergy progr amme o r pro j ect

Please note that you should discuss specific points with different stakeholders to find out their perceptions. You can carry out an exercise through group discussions tothink about the various issues. One approach is to use four different coloured sheets of paper cut into small squares with each participant given a few cards to write their own viewpoints. Individual cards can then be collated on a large sheet of paper or board. Similar ideas need regrouping into one section. Discuss the various points in detail and ask the participants to list out the most important ones (could be three to

five). A facilitator is required to complete this exercise. He/she should be aware that in a SWOT analysis, a threat could also be an opportunity.

* Mention who will be responsible for the action

Strengths Actions* Weaknesses Actions*

Finance available Budget allactivities well No finance Look for loans

Create funds

Willingness toprovide labour bycommunities

ListresponsibilitiesMake an actionplan

No technical ormanagementskills

Assess possibletraining facilitiesfor staff/users

Opportunities Actions* Threats Actions*

Good localmanufacturers Link them tocommunities

Possibility of gridelectrificationthrough politicalpressures

Find out detailsfrom ElectrificationDept

Funding throughgovernmentsubsidies or otheragencies

Approachfundingagencies withplan

Conflicts overdistribution of energy and useof resource

Active participationand consensusfrom allusers/community

Materials required for par ticipatory exercisesPaper- large sized (A stand to put up the paper) and Marker pens of different coloursStones, seeds, coloured beads, pebbles, leaves, or different coloured paper squares, etc.



29

Wealth Ranking Wealth ranking is used to categorisecommunities according to theireconomic status, the most commonbeing high, middle and low-incomegroups. However, differentcommunities may have their own

methods of categorising whether ahousehold is poor or rich. To find thisout is the main purpose of wealthranking. Wealth ranking exercises may be difficult to conduct accurately inlarge villages or settlements. The wealthranking exercise should ideally becarried out as one of the initialparticipatory exercises. It is useful to

determine the relations amongst socialgroups and importantly the assets andstrategies needed to exit from poverty.

If you use an equity approach, the firststep before a wealth ranking should bethe differentiation of social groupsbased on their ethnic or linguisticorigin. The social group differentiationprocess should be carried out with thecommunity. However, you have to be

cautious in taking this step because, insome communities, it may providereasons for hostility to break out if different social groups do not get on

well with each other. One has to specify the reason for undertaking this task.

Once you have identified the different

social groups, proceed to make surethat there is sufficient if not an equalrepresentation in the participatory meetings. Proceed to conduct the

wealth ranking exercise. For step 3,allow the community members to listcriteria such as income, livestock, andland distribution for the personconsidered poor or rich and so on. Inthis process, women headedhouseholds or single parent householdsshould also be included. Facilitatorsshould have the materials (papers,charts, pens) ready before the process.

After the completion of the wealthranking exercise, you can link it todiscussions on indicators such as accessand control over resources,involvement in various community activities, and decision-making.

* Discussed in detail in the text

PRA Methods* Baseline Monitoring Evaluation

Ranking: wealth ranking, preferenceof problem ranking x x x x

Mapping: groups or personal x x x x x x Trend analysis: seasonal calendars,individual charts x x x x

Interviews: Focus Group discussions& in some cases, individual x x x x x x

TABLE 5: Seasonal calendar and time availability of different members of thecommunity



30

Advantages and Disadvantages: Theadvantage is that illiterate people,especially the poor, can effectively participate in the process. Thedisadvantage is that some membersmay dominate the process. The

communities may also not be ready tocategorise themselves according toeconomic categories, as it is often asensitive issue.

Implication of results: Time planning with regard to participation inmeetings, technical installation, andother such activities need detaileddiscussions according to the availability of the members of different groups.

Dialogue: Focus GroupDiscussions and InterviewsFocus group discussions (FGDs) helpin the identification of interests, accessand usage of various resources fromspecific groups. FGDs are extremely important in a participatory processbecause they provide channels toexchange information and ideasbetween the communities and theservice providers or projectimplementers. Participants of each

focus group may be selected afterdiscussion with community membersor key informants. Depending on theobjective or different informationsought, different focus groups can beformed outside the community,grouping similar like-mindedorganisations such as renewable energy suppliers, NGOs and donors,

government representation of energy planning, and so on. The moderatorshould facilitate and guide thediscussions and not put in his or heropinion.

During the implementation of community projects, it is important todifferentiate groups according to socialcategories (rich or poor), gender(women and men), and so on. A focus

Remember that participants can provide either exaggerated or underrated information. You have to cross check data relating to particular indicators usingdifferent sources and techniques to formulate a reliable informationsystem.

Process1. Create a household list of all

users/ beneficiaries inconjunction with participants.

2. Write the name of the householdhead on small cards.

3. Ask participants to list the types and characteristics of different wealth groups.

4. Participants can divide the setof initial cards into each wealthcategory (and according to

different social groups).5. Reconfirm and crosscheck the process. Count the piles andrecord.

6. Use the ranking information asyou perform and analyseindividual time lines, seasonalvariations, and focus groupdiscussions.

7. Document the results and process.



31

group may be formed to discussparticular issues or to find solutions toexisting problems within these groupcategories. Specific points should beprepared for discussion and if there istime, members can be provided withsome discussion points before themeeting. In this way, the reactions of different groups to the same issues canbe determined and compared. Forexample, it may be important to know

which impacts can be/are derived froma particular technology by differentsocial groups.

FGDs are useful to gain informationon indicators that are related to energy access, service and use of differentgroups and particularly the changes thatoccur before/during or after an energy project intervention. Detaileddiscussions can take place on the prosand cons of participation of variousgroups.

Advantage: It is possible to attain in-depth targeted information aboutattitudes and behaviours that may notbecome clear using other techniques. If

the participants are committed, they

Process1.Identify the topic/issues you want to discuss in a focus group. Create a

checklist.2. Identify the target group who will be able to provide feedback on this topic.

Remember to consider men and women/rich and poor or have separateinterest groups.

3. Identify a common place to meet in discussion with community members.4. Identify group members through discussions with leaders or contact

persons in the communities. An FGD should not exceed 10 to 15 members,otherwise it will be difficult for each member to contribute.

5. When you start the meeting, explain the process, duration for the meeting,and the possible outputs you envisage. Ask their opinions and if they have

issues to discuss. Do not drag the meeting on for more than two hoursunless people in the group want to discuss it further.6. Use charts and pictorial representations if required. Involve the people in

the discussion. 7.Record all the points made in the meeting for further discussions during

M&E.8. If possible, arrange for some snacks and drinks for the participants to

show appreciation for the time provided by them.



32

often suggest solutions to problems, which would be acceptable to serviceproviders and users. Focus groups helpto clarify differences especially whenthere are divisions in the societies interms of gender, ethnicity, economicgroups and so on. Focus groups canalso be formed to bring two dividedcommunities to solve one particularproblem. Importantly, focus groupdiscussions help to bring the voices of all people to contribute and not just afew powerful individuals.

Disadvantage: Prominent andpowerful members may dominate thediscussion. Women and poor may beleft behind in discussion if it is mixedgroup.

Results: If group discussions show conflict in the community, these issuesneed to be solved before the project isimplemented. Project implementersmay stop partnerships or even mediatebetween groups and individuals if thereare severe conflicts.



33

Process: In a study to assess theusefulness and acceptability of anelectricity prepayment programme inArgentina, the research teamconducted different focus groupdiscussions. One of the groupdiscussions was based on the users'awareness of energy efficiency. Itwas expected that the discussionwould indicate the potential of usersto reduce electricity consumption and

engage in energy efficient practices.In one exercise, each member of thegroup was given a set of pictures of selected appliances (see below) andthe group was asked 'If all theseappliances are in use for half an hour,which would use the most electricity'?

Results: The group ranked theselected appliances from 1-10according to their perceived energyconsumption. The group recognised afew of the high-energy consumption

appliances, such as the iron, theelectric shower and the electricheater, but participants were not sureof the correct order of these. In mostcases, it was hard for participants todistinguish between the electricityuse over a short and longer period. Itwas clear that the group had nocommon reference point.

Source: Annecke, W. et al, 2004

Group discussions also led to theunderstanding that both males andfemales experimented differentlyabout the energy consumption of each electrical appliance. While themen were more knowledgeable ontechnical issues, the women hadacquired valid knowledge throughpractical experience. For example,one woman turned off all the lights,and started testing each appliance by

checking on the prepayment meterscreen for an individual consumption.After the increase in knowledge aboutthe electricity consumption, thegroup members expressed that theywere cautious about using appliancessuch as iron, electric heaters andwater heaters. In all, only one of theparticipants in the group usedelectricity inefficiently, the otherstried to save electricity.

Lessons: The focus group providedconsiderable depth to the teamsunderstanding of how customersthink about electricity consumptionand appliances, and how, because of a lack of information from the utility,each household has to deduce theelectricity consumption of eachappliance, and make their owndecisions on what to use and for how

long on an ongoing basis. Therefore,the main lesson learnt was that thetrial and error learning initiative couldbe assisted by appropriate input fromthe utility. It was also found that ontheir own, individual ranking of appliances was not very accurate, butwhen the group discussed energyefficiency and re-ranked theappliances, a marked improvementwas shown.

CASE STUDY:Focus group discussions with poor in Argentina -

deter mining knowledge of users regarding energy ef f iciency



34

Mapping: social, resource or mobility mapping, transect walksMapping through transect walks anddrawings by the communities can beeffective, particularly in relation togathering information on access andcontrol over resources, such as forestand fuel wood, water, land, farm mapsand other services. Monitoring andevaluation processes can investigateenergy specific access and control as

well as compare the old and new situation.

When developing baseline information,mapping of location and usage of watertaps and wells can lead to an idea abouthow locals use their resources.Discussions can be based on themaintenance and management of theseor other resources by the communities.Such information can provideimportant indicators for the renewableenergy intervention.

After mapping, transect walks can beconducted whereby team members walk

with the local community members and

Process

1. Have an idea of the features you want to map before the exercise.

2. Select an accessible venue to carry out the mapping exercise.

3. Select community members to assist in the mapping exercise and discuss the details before starting the exercise. Clarify the idea further and discusswhich particular features are important for the local community in order toinstall a renewable energy system.

4. Use large sheets of paper and place them on the ground. Ask one person ata time to draw the project area. Avoid drawing large areas. Use differentgroups if the area covered is large.

5. Ask members to draw or use different materials (such as stones, pebbles,coloured beads) to show different features of the area such as houses, roads,water source, schools, health clinics, and so on.

6. Depending on the information you want or the information they want to tell you, ask them how these features have been used in the project andwhat the impacts have been to different households (in the mapped outarea), and on men and women.

7.If it is a monitoring or evaluation exercise, use the map or draw thedifferent features of the technology (power house, solar installations,access to resources) and discuss accordingly. If you have a map madebefore, use the same map for further discussions and trace the changes.



35

improve on the map already drawn.Discussions can be carried outsimultaneously.

Maps can be drawn with eitherindividuals or groups. Personal maps

will show individual perspectives andmay draw attention to differentiating between gender and social groups.

Example1: Participatorymappingof mill locationsin India

In a watermills study conducted in India, participatory mapping provided anunderstanding of mill location and approximate distances to fields, fodder and fuel

wood collection areas, diesel mills, market towns and other significant landmarks. The study team conducted transect walks along the river valleys where the mills were located. The transect walks were also used in all study areas to enhance orreconfirm information collected through other techniques.

Source: IT Power, Socio-Economic Impact Assessment of Improved Watermills, 2003



36

Advantage/Disadvantage: Whiletransect walks and mapping are goodfor gathering data on access andcontrol in smaller areas, largegeographical areas may be difficult tocapture in the mapping exercise.

Results: Mapping can lead to decisionson the location of electric poles, solarpanels, powerhouses, mills and biogasdigesters. It can provide details on thespatial area to install the renewableenergy technologies, particularly interms of access to settlements where

the poor or disadvantaged beneficiariesmay live. Monitoring and evaluationexercises can draw data on where thefrequent problems lie and the solutionsthat communities come up with.

Diagrams: Time lines andseasonal calendars

Time lines and seasonal calendars areextremely important tools that can helpdraw a time plan for projectimplementation and monitoring.

Timelines can be drawn with individual persons (forone day during differentseasons) or in groups.Individual timelines canhelp you understand the

daily activity patterns orfinancial flow of households. Timelines of different householdmembers of active age canbe an importantdeterminant for planning users' inputs in projects orprogrammes.

You can chart time lines with groups todetermine a variety of informationsuch as seasonal variations of theiractivities and different work processes.In terms of gathering technical data,timelines can be drawn to show changes in rainfall, sunlight, river flows,

wind and other such parameters thatmay be needed for technicalassessments.

Advantage: Timelines can be drawnindividually or with members from afew households. It does not require the

presence of all the community members.

Results: Timelines are perfect forplanning activities. They can help in theconstruction of projects if thebeneficiaries are involved. If you draw timelines according to different socialor economic groups or men and

women, it will provide you with theperfect basis for gaining results for anequitable outcome.

Season Activities Which season are they free?

Men Women Group 1 Group 2

Jan-Mar Cropplanting x x x x

April-June Harvesting x x x x

July-Sept Charcoalmaking x - - x

Oct-Dec Collect wood, rest - x x -

TABLE 6: Example of a seasonal calendar and timeavailability of different members of one particularcommunity





38

advisable that you do not carry out thesame assessments repeatedly. Besensitive to the needs of therespondents and ensure these needs are

met. Designing a survey questionnaireneeds to be targeted specifically to thegoals and targets set by the community or the project developers.

Despite the useful information provided by the village map and seasonal matrix during the first mission, feedback from the women implied they do not wish to attend groupmeetings in which they could see no clear benefit and which disrupts their work. It wasalso found that in larger groups the more vocal women tended to lead the discussion,

with the others agreeing passively or not participating at all. As a result, it was agreedthat a preferred approach to gain an understanding of women's time use and livelihoodissues would be to spend time with them in the villages over a few days.

Although it was planned to shadow women at their work, initial observations showedthat this approach alone would not provide the necessary details about women's timespent on grinding grain and husking rice because these activities happen only a few times a month. In order to elicit a range of views and to minimise disruption it wasthen decided that a door-to-door survey should be used for collecting informationabout household milling habits and their perceived impact of upgraded watermills. Tothis end, a door-to-door interview with 5-8 households in each village was felt to be amore effective way of collecting the desired information. Women were approached

when in their homes and less likely to be disturbed in their work. This approach alsoseemed to be welcomed by the women, many saying they were glad for the opportunity to talk about their experiences. However, when women said they were too busy theinterview team moved on. The other benefit of an individual (door-to-door) approach

was to avoid the tendency of group answers, making individual variation and opiniondifficult to draw-out.

Such experiences are common, especially if you want to conduct in-depth monitoring or impact research. Therefore, it is important to use a combination of methods andapproaches.

Source: IT Power, Socio-Economic Impact Assessment of Improved Watermills, 2003

Box 6: Feedback on using participatory approaches from the field:extract from a watermills study in India



39

SECTION5 ECTION5 ASSESSMENTSESSMENT

INDICATORSDICATORS This section will focus on defining specific M&E indicators to determinethe impact of renewable energy projects or programmes. It is difficultto set accurate measurable impactindicators using participatory approaches because the indicators need

to be derived from discussions and theresults are qualitative. Basic indicatorsare required nonetheless to indicateprogress and impact. Accurate data onsome of the indicators below may notbe directly available using participatory methods but these indicators aredesigned to guide discussions withusers and stakeholders. A triangulation

of data derived from participatory methods, surveys and backgroundinformation would provide the idealinformation for a strong M&E output.Participatory methods can be usedtherefore, as one of several methodsfor M&E.

In any monitoring and evaluationprocess, it is important in the first step

to set a basic overall structure of theproject or programme area and itsinhabitants. Below are a set of indicators that can be used irrespectiveof the type of renewable energy intervention.

The general indicators in table 7 areimportant to provide not only anoverall impression of the area andhouseholds' socio economic

characteristics but also to determinethe results in the analysis and reporting of the M&E process. For example,household characteristics will enablepractitioners to distinguish rich andpoor households. Data to fulfil thedescription of most measurableindicators are best derived from asurvey. You have to design yourapproach in a way that allows you todiscuss the indicators with the users orbeneficiaries during group discussionsor whilst undertaking a mapping exercise.

Monitoring and evaluation will need toconsider specifically the impacts thatresult from the project or programmeintervention. Tables 7 and 8 show a setof indicators that you can use fordiscussion to generate information forthe M&E component of yourprogramme.

Remember that indicators for anM&E process will depend on:

The needs and priorities of users, stakeholders and programme implementers

The specific project or programme area

The types, size, quantity of renewable energy technologyschemes installed



40

Variables Indicators

G e n e r a

l a r e a a n

d h o u s e - h o

l d c

h a r a c t e r i s t i c s

Dem ogra phi cs - Proportion of

households inregion/area

- Proportion of male/female/children

- Literacy rates in area

- Number/ percentage of households served/not served by project/programme

- Number of male/female/children in area- Percentage of beneficiaries' literate/illiterate

Bas ic i n frast r u ctur e - Transport- Electricity - Water- Health- Markets- Communication

Number/percent of households served by/ access to: - Nearest transport connections: roads (type), vehicles

- Distance from nearest electricity provisioncentre

- Different sources of water, Distance to source- Nearest health centres or hospitals- Distance from nearest markets- Distance from nearest communication facilities

Household cha ract er ist i cs - Types/ numbers of

rooms- Household member

characteristics- Income and assets- Expenditure

- Number/ types of houses owned /no. of rooms/ type of roofing, flooring of house

- Number of household members according tosex, age, education, occupation

- Income access, type, scale- Type/quantity of assets (land, livestock)- Expenditure: according to commodities

Soci a l net works - Leadership- Local organisations- Local management

systems

- Type of leadership- Types/ numbers of local organisations- Number of male/females involved in different

organisational capacities, and decision-making

TABLE 7: Overall area and household characteristic indicators



41

TABLE 8: Output and impact indicators for the project/programme

General Indicators

Energy accessand usage forinstitutions

- Number and type of institutions (health, school, smallindustries) served by the renewable energy project orprogramme

- Number of households/percentage of population(male/female) benefiting from institutions served by theproject or programme

- Type of appliances/usage from the renewable energy servicesin the institutions

- Quality and reliability of energy services

Energy accessand usage for

productiveenterprises

- Number and types of productive enterprise/s started after therenewable energy project or programme

- Number of productive enterprises started- Years of operation of each productive enterprise- Electricity required/used by the enterprise- Investment costs and returns- Type/quality of management of the enterprise

- Number of households/families benefiting from theproductive enterprise

- Awareness levels of users of tariffs/customer satisfaction

Access toinformation/communication

- Number/type of communication technologies afterinstallation of the renewable energy scheme

- Increase in access to information of: income generationactivities, news, health and safety and others



42

Types of Groups Classification Methods

Economicstatus

- Income groupsaccording to income &expenditure/ or asindicated by users

- Classify thecommunity/ies

- Official statistics- Wealth ranking /

FGDs- Mapping - Survey

Ethnicity - Mixed group- Single group(depending on specific

location)

- Number of households in eachgroup

- Survey andinterviews/ FGDs

- Official statistics- Ranking - Mappings

Gender- Male/female - Number of malesand females in the

household (in eachgroup)

- Official statistics- Surveys- FGDs

Family - Extended, nuclear, joint,others

- Type/ number of members inhouseholds

- Household surveys/interviews

- FGDs

Socialnetworks

- Members of groups - Number of memberships in

other groups

- FGDs- Interviews

If you use the social equity approach,the first exercise you have to conduct inall your participatory exercises is todetermine differentiated data withregard to different social (ethnic, gender,clan) and economic groups in theproject area (see table 9). If you want tofollow this approach, this exercise willform the core for differentiating the

various impacts according to gender,social groups, rich and poor and so on.

The overall aim of your project orprogramme should lead you to decidehow to differentiate the groups. Forexample, if you want to determine theimpact on gender, it is important thatyou carry out different participatory exercises with men and women to find

out their roles, responsibilities, interestsand influences amongst others specificto the renewable energy technology intervention.

After disaggregating the specific datarelating to different social and economicgroups, and gaining an overallunderstanding of the project orprogramme area, you should start todefine definite indicators that would leadto monitor the access and usage of energy. Remember that indicatorsshould be chosen specific to the project

or programme site/s. Table 10 providessome indicators that you could start todiscuss with the users and stakeholders.

TABLE 9: Disaggregating groups for the social equity approach



43

General Variables IndicatorsEnergyaccess andusage

- Equitable accessand usage of electricity andother benefits

- Number of households/percentage of populationswitching fuel from traditional to renewable energy services

- Number of households/percentage of population with access to the energy services (electricity/ biogas)

- Number of households/percentage of populationusing the energy services (in units/ months)

- Number of households/percentage of population with access/ usage of productive enterprises/equipments and appliances

Financial - Affordability of the renewable

energy services- Division of costsand benefits

- Cost/percentage of household income andexpenditure spent on:

- the installation, connection and tariffs- costs of appliances- repair and maintenance costs- Changes in economic levels of users

(income/expenditure) resulting from fuel switching ornew employment opportunities.

- Repayment schedules for the renewable energy system

Ownershipandmanagement

- Control over theownership,maintenance andmanagement of the renewableenergy technology andits benefits

- Participation inmeetings,increase in self esteem,bargaining power

- Number of households (groups, male/female)access/control to use, maintain and manage the energy

- Number of new employment opportunities- Number of trainings- Number of male/female in Management Committees- Decision making positions- Levels of management skills generated- Users perception of management- Levels of transparency - Frequency of meetings- Increase in the quality of services

Maintenance - Access to

maintenanceservices

- Distance to local service centres in area

- Costs of maintenance- Users access to maintenance operators/mechanics- Levels of local maintenance skill creation, number of

trainings

Environment - Rate of deforestation

- CO2 reductionrate

- Forest area affected- Biomass utilisation- Savings of CO2 emission through fuel switching

Otherbenefits

- Leisure- Social networks

- Increase in number of hours for relaxation- Increase in listening to radio/watching TV - Increase in social activities- Increase in prestige

TABLE 10: Renewable energy output and impact indicators for households



44

Be careful while gathering data from aparticipatory exercise. Take an exampleof energy source and usage to show theparticipants. Make sure that you discussenergy access and usage using the sameunits during baseline, monitoring andevaluation. Be consistent. Differentbeneficiaries will have differentperspectives and usages. Therefore,such information needs to be taken atan overall level but, for definite outputsand impacts, individual beneficiariesshould be interviewed. Gathering information on energy expenditure isoften very tricky. One way to do this isto get accurate information on quantity of the energy source and multiply it by the cost per unit of the energy usage.

This needs to be done immediately and,

if discrepancies arise, it can bediscussed further with the participants.

R enewable energy projects

and the MillenniumDevelo pment GoalsRenewable energy schemes may contribute directly or indirectly tofulfilment of the MDGs. Below is a listof indicators that you can discuss withlocal users and beneficiaries. The M&Eprocess will be effective if you use thesame indicators to carry out

participatory exercises at differentstages of the project cycle. If you wantto use the equity approach, it isimportant that you differentiatebetween the various social groups.

TABLE 11: Renewable energy output and impact indicators for householdsMDGs Common indicators to measure outputs and impactsEradicate

poverty andhunger

- Number of households benefiting from projects according toincome/gender

- Number of poor households with increase in productive activities after theinstallation of the renewable energy project/programme

- Number of people employed by the project- Differences in income and expenditure resulting from use of renewable

energy - Availability of finance for poor households to access energy from the

projects





46

FIGURE 5: Differentiating socio-economic groups

Socialgroup(ethnic/clan)

High incomegroup

Lowincomegroup

Mediumincomegroup

- Male- Female

- Male- Female

- Male- Female

household users can record specifichousehold information and socialmobilisers can monitor certain activitiesdepending on the scale of theprogramme. While these constitute oneset of data, information derived fromthe participatory exercises form themost important source of M&E data.

After you conduct a participatory exercise, make sure to duplicate theresults on to another sheet of paper asthe ones worked upon during participatory exercises may use differentmaterials such as stones and beads toshow different resources or features.

Step 2: Differentiate social,economic & gender groupsUsing the social equity approach meansthat you have to differentiate the groupsaccording to social, economic or gendercharacteristics before you start ananalysis. You have to conduct thesedifferentiation exercises early on in theprocess. When you are assessing yourresult, the differentiation should look like the one in the following figure:

Analysing information from par ticipatory approachesParticipatory exercises can generate a

wide range of data. However, you needto analyse the information from thebeginning of the project to devise anunderstanding of the trends andpatterns that may evolve as youinterpret the data. In this section, somesimple steps are provided to follow theassessment of the data generated fromthe various exercises.

Step 1: Record all informationUsers and communities can generateand record some information for you.For example, the managementcommittee can record ongoing performance, the operators or

SECTION6ECTION6ASSESSMENTSESSMENT

OFF RESULTSSULTS



47

Example2: T riangulating r esults f romparticipatoryexercisesf orprovisionof

labourinputs by usersUsers: We are extremely busy planting crops in

March and April.Insta llers: March is the best season to install/construct projects because it does not rain.

Fi n an c i ers/NGOs: Financial dealings close by March. All major activities need to be carriedout before March. Additionally, thecommunities need to provide 25 percent of construction costs in terms of labour.

How do you solve such a problem when 'your'best time is the 'worst' time for the users?Often, project developers do not seriously consider users' perspectives. The above datamay come from stakeholder analysis and focusgroup discussions.

Take action: You can further use the timelinefeature or seasonal calendars to come up with asolution or plan for project or programmeintervention. All community members meet todiscuss the issue. All households could perhapsprovide a few days of labour despite their work or an 'additional financial contribution' that themanagement can decide with them. In addition,planners can play around with the costs andinput the labour or work earlier or later. Theusefulness of participatory approach will show only when you know how to use the dataeffectively and carefully implement, monitor

and evaluate responses.

Step 3: Differentiate and review the main themes The main themes of the PM&Eprocess may be determined before theparticipatory exercises. However, theremay be other information generatedfrom the discussions and exercises. Youneed to differentiate each theme andreview them. Expert facilitators willcarry out this differentiation process

while conducting the participatory exercise itself. If you have specificthemes, review them with the

stakeholders. If you do not, start by listing key themes such as overalldemographic information, socialcharacteristics, energy access, energy usage and management capabilities. Inthis way, you are providing a structurefor the analysis. Each participatory activity needs a separate analysis andcompletion at the end of each exercise.

Initial results need to be reviewed withusers/stakeholders. If you have a wide variety of results generated fromdifferent exercises, it is best that youreview it immediately when you are

within easy reach of the location andpeople. It will enable you to cross-check on information and avoid majordifferences in results.

Step 4: Triangulate the results After you have completed recording and reviewing the data, you will end up

with a wide range of results. Qualitativedata may also be subjective. Therefore,one of the most useful exercises toundertake is to triangulate the differentdata sets. In the following example, setat the initial stages of project

intervention, you will see that theanswers to the same questions fromdifferent stakeholders are triangulatedto form actions and plans. You canalternatively triangulate information toconfirm the results from differentsources such as observation andsurveys.



48

Step 5: Review initial results withstakeholdersIf the renewable energy interventionprogramme is a large one covering a

wide geographical area and a largenumber of individual projects, it isimportant that some of the initialresults be reviewed again withstakeholders, especially the users. Inthis way, you are not only informing them of their choices and progress, butalso providing another step towardsempowering them. Such a process will

lead to transparent negotiations andlesson learning and is found to be anextremely effective foundation on

which to base an exit strategy.

After each monitoring process, itshould be clear that the following havebeen achieved:

Problems are identified by all

stakeholders. Actions for the problems havebeen identified and discussed.

Responsibility has been taken forthe action.

Readjust any changes to theoriginal objectives.

Similarly, the evaluation process should:Identify whether the objectivesoriginally planned are completedor not.

Assess whether actions andresponsibilities have beensuccessfully coordinated andimplemented by various

stakeholders.

Evaluate the outputs and impactthat has been achieved.

Undertake a detailed examinationof the success and lesson learning factors for similar interventions.

Determine a possible exit strategy for programme and projectpersonnel.

If there are indicators that do not show any importance to the assessment of aparticular renewable energy technology (such as wood fuel savings for solarhome systems or pico hydro), omissionof the indicators needs to be reviewedat the monitoring stage.

Step 6: Reporting While you report the findings, theanalysis needs to come up with aconclusion on the progress of the mainthemes. All monitoring reports shouldclearly recommend the plan and actionsto improve on the themes that lack progress. Any omission or review of the indicators needs to be reportedconsistently. Reporting would alsomean providing structure to theanalytical results. The most commonstructure relates to the access and usageof energy (see table 12). Structure your

report layering the progress of eachmonitoring exercise so that it will beuseful for the mid-term or endevaluation. A report is effective whenall stakeholders assume responsibility and learn from the process, and mostimportantly make effective decisionsand action plans to move forward.



49

ENERGY SOURCE

Access Usage Usage Quantity Costs ExpenseIndistance

Lighting/cooking/radio

Time (no.hrs/ dayor month)

Litres/nos/day ormonth

Currency/day ormonth

Currency/day ormonth

Electricity-grid

Electricity-RE

LPG

Kerosene

CandlesParaffin

Batteries

Charcoal

Fuelwood

Twigs

Dung

Others

TABLE 12 Example - Energy Access and usage data needed forbaseline and M&E

Building ownership of par ticipatory monitoringand evaluation

Building ownership of the tools andprocess of a participatory approach toM&E is probably the toughestchallenge because of the demand foraccountability, transparency and theneed for co-ordination of allstakeholders. It is also an approach thatneeds to be built into programmes andthe system of implementation. Thereneeds to be ownership by the variousstakeholders of the programme. Onemust keep in mind that differentstakeholders have different perceptionsand expectations. However, theapproach is best suited if the sameusers were to own and maintain the

energy technology and its varioussystems. In this case, they need tomanage not only the maintenance of the technical system, but also theservicing and collecting of tariffs. It isin such cases that a participatory approach would prove to be a tool forreducing costs by allowing forimmediate mitigation activities if aproblem or conflict occurs.

The participatory approach also buildsownership with different stakeholdersas they develop their own skills and

discuss amongst themselves variousoptions to better implement andmaintain a project. If ownership ismaintained by the stakeholders, there isa higher chance for the energy projector programme to be sustained.



50

REFERENCESAnneckeW., EndelliM. and CarpioC. (2004). Report On The

Acceptability And Socio-Economic Impact Of Prepayment Meters in Merlo, Buenos Aires. For EdF, France.

BlackburnJ., ChambersR . and GaventaR . (2000). MainstreamingParticipation in Development . OED Working Paper Series. No. 10.World Bank. Washington D.C.

ChambersR . and BlackburnJ. (1996). The Power of Participation .IDS Policy Briefing Issue 7.

ESMAP(2003). Monitoring and Evaluation in RuralElectrification Projects: A Demand- Oriented Approach. UNDP/ TheWorld Bank.

EstrellaM. and GaventaJ. (1998). Who counts reality? Participatory Monitoring and Evaluation: A literature review . IDSworking paper 70. Brighton: IDS.

ITPower(2003). Socio-Economic Impact of Improved Water Mills in Chamoli district, Northern India.Report prepared for DFID. UKSrinivasan L . (1993). Tools for Community Participation.PROWESS/UNDP -World Bank Water and SanitationProgramme.



TheITPowerGroupworksinternationally, with a networkof offices in the UK, India,China, Mongolia, Kenya, South Africa, Australia and USA. Thegroup specialises in renewableenergy, climate change policiesand business consulting.IT Power's projects involve a widerange of sustainable energy technologies that will helpcountries meet their MillenniumDevelopment Goals, including

wind, photovoltaics, mini andmicro hydro, biomass, solar water heating and landfill gasutilisation. Since 1981, IT Power has undertaken over 900 projectsin more than 100 countries, for institutional and private clients.

www.it power.co.uk

TheRenewableEnergy andEnergy EfficiencyPartnership(REEEP) is a global private- public partnership launched by the United Kingdomgovernment along with other partners at the JohannesburgWorld Summit on SustainableDevelopment in August 2002. By providing opportunities for concerted collaboration amongits partners, REEEP aims toaccelerate and expand the global

marketplace for renewable energyand energy efficiency technologies. The partnership is funded by a number of governments including: Austria,Canada, Ireland, Italy, Spain,The Netherlands, The UnitedKingdom, The United States

and the European Commission.

www.reee p.org



Renewable energy technologies provide reliable andaffordable energy supplies to millions of people indeveloping countries. This publication is a toolkit designedto enable organisations and communities that install and

implement energy technology systems and programmes tocheck the efficiency and effectiveness of their work. Thetoolkit focuses on the participatory approach and describeshow to take account of users’ opinions and make them active

players in the installation and delivery of renewable energysystems.

Documents

PM&E toolkit_IT Power