The Newsletter of

viewpoint

No. 37 September 2007

Contents

Rethinking Monitoring and Evaluation ................................ 1

INTRAC Training .......................... 2

Rediscovering the Logframe? .......... 3

Stories and M&E ............................ 4

Learning at the Centre-Stage .......... 5

Getting your qualitative data taken seriously: using QUIP............ 6

INTRAC Conference News........... 7

INTRAC People ............................ 7

INTRAC Publications .................... 8

In this issue: Our theme in this issue is ‘RethinkingM&E’. Brian Pratt analyses the policyshift towards managerial values inmonitoring and evaluation; AnneGarbutt describes how nascent CSOs inOman welcomed the logframe’sstructure and clarity; numbers arecontrasted with stories in NomvulaDlamini’s M&E experience from SouthAfrica; Alix Tiernan calls for a paradigmshift to build learning into theprogramme cycle of NGOs, and KatieWright-Revolledo sets out the steps forlinking quantitative data and qualitativemethods in QUIP. Also, read aboutINTRAC’s latest book, ‘RethinkingM&E - Challenges and Prospects in theChanging Aid Environment’.

viewpoint

At the heart of the Monitoring and Evaluationprocess are the respective roles of state and civilsociety and the steady corporatisation ofdevelopment.

Every few years M&E is rediscovered asthe missing or weakest link in

international development cooperation.Words like ‘accountability’ and ‘learning’are banded about and solemn promises aremade that this time it will all be different.

Over the past fifteen years INTRAC hassponsored six international conferencesand regional workshops across the globe,each reviewing such new ideas on M&E.Yet, the last two international conferenceshave struggled to unpack and separateM&E ideas from the realities faced byNGOs and other social developmentactors. In 2002 we looked at the clashesbetween performance assessment and newmanagerial techniques, and participatoryapproaches in NGOs.We followed this upwith a series of workshops. Differencesbegan to emerge at workshops in Indiaand Peru, for example.

Participants were keen to make sure theycould engage with their clients/userswhile improving their basic professionalstandards in M&E. The dialogue inEurope, however, was very different andseemed to become entangled in a debateabout M&E and ‘managerialism’ – a moreexecutive style of M&E managementtaken from the corporate sector.

In analysing some of the current debate itwould appear that there is a wide schism in

development thinking which goes beyondM&E techniques and approaches. It goesto the heart of what people think they aredoing as development, but also how theyare doing it.We can see, on the one hand,an increasing concern with the survival ofdevelopment institutions and a focus onmanagerial techniques which are onlysuperficially concerned withaccountability. On the other hand, there isa continuous challenge by a range ofdevelopment practitioners to prioritiseusers’ interests and needs. Certaindichotomies are emerging in M&Epolicies and practices. In some cases theseare being formalised across a spectrum ofdevelopment actors and agencies.Addressing them is often a political ratherthan a technical challenge.

Emerging dichotomies Our new book Rethinking M&E:Challenges and Prospects in the InternationalAid Environment (INTRAC, Oxford 2007)tries to capture some of the debates in theSixth Evaluation Conference and its

Rethinking Monitoring andEvaluation

“The schism goes to the heart ofwhat people think they are

doing as development, but alsohow they are doing it.”

M&E of CBOs in East Africa.Sara Methven, INTRAC 2007

S4029 7/9/07 16:53 Page 1

2

related regional workshops. The bookdeals with many of the opposingdichotomies that are described in Table 1.It traces the confusion caused by thepersistent misuse of words and conceptssuch as when people talk about‘accountability’ whenthey actually meanaccountancy or speakof ‘participation’when they areactually referring totop-down programmes. Similarly, the term‘decentralised management’ often meansthe exact opposite because the parametersin which people work are so narrowlyconfined by corporate goals. It is thereforevitally important not to take terminologyused by agencies and individuals at facevalue, but to dig beneath the rhetoric todiscover the reality.

Some of the dichotomies are encapsulatedin the following opposing concepts (laidout in Table 1), although in many cases thereality lies in a spectrum between these.

Many of these are explored in detail inRethinking M&E. The question stillremains: how many of these are reallydichotomies we can choose between, andhow many are merely tensions which we

simply have to live with? Is it inevitablethat institutional interests consistentlyweaken the real function of M&E – whichwe understand as strengthening userfeedback (downward accountability) andenhancing learning? Instead we have a

predominance of short-term accountability interms of accounting forthe use of resourceswith little interest intheir real impact.We see

blatant abuse of concepts such as ‘aideffectiveness’, a term which has beenmisappropriated by those interested in theefficiency of public services rather thantheir impact. We agree that administrativeefficiency is a positive goal, but it shouldnot be prioritised above achieving realimpact in the lives of people. A simplechange in our priorities could have seriousimplications for the growth of self-servingorganisations if it puts people back at thecentre of development.Whilst it might beutopian to argue for a return to Real Aid,we can at least argue for real monitoringand evaluation, and an end to the misuseof these crucial developmental approaches.

Brian Pratt is INTRAC’s Executive Director.

Email: bpratt@intrac.org

“It is vitally important notto take agency terminology

at face value.”

Reporting Against Service contracts or Social development

Pre-agreed plans and contracts or Autonomous development

Deliverables or Impacts

Technical indicators or Poverty reduction

Overall Aims Serving the state or Serving people

Reform or Transformation

Management efficiency or Impact

Short-term gains or Long-term change

Formalising institutions or Developmental

(socio-economic-political)

Organisational Impetus Cooptation or Independence

Compliance or Empowerment

Rationalising reality or Enquiry

Short-term accountability or Learning to do better

Donor competition or Sharing learning

Creation of a comfort zone or Encouraging a culture of

challenge

INTRAC Trainingwww.intrac.org/training.php

New! Civil Society and AidArchitecture: Changing Dynamics ofAid Policy 17-19 October 2007Understand complex and changingdevelopment policies and how they playout through different mechanisms ofinternational cooperation.

New! Research Methods1-2 November 2007Gives you the tools to use research aseffectively as possible in your work.Includes planning research, managing theprocess, quality in data gathering andchoosing methods.

Supporting Southern Advocacy7-9 November 2007Learn how to identify core advocacyskills, find real issues, causes and solutions,as well as key strategies for policy changein difficult contexts.

Creative and Strategic Thinking21-23 November 2007Be creative and get involved in a dynamicprocess to be able to respond to changingcircumstances.

Rights-Based Approaches toDevelopment 5-7 December 2007Reflect on the implications of usingrights-based approaches in yourprogrammes and organisationaldevelopment.

Gender Analysis and Planning16-18 January 2008Learn to analyse, plan, and operationalisethe gender perspective into effectivedevelopment practice.

Advanced Participatory Monitoringand Evaluation28 January – 1 February 2008How to develop a cost-effectivemonitoring and evaluation system thatgenerates high enough quality data andenough information to provide adevelopment agency with a reliableunderstanding of the outputs, effects andimpacts of capacity building processes.

Table 1: Emerging M&E Dichotomies

S4029 7/9/07 16:53 Page 2

3

How far are logframes useful planning,monitoring and evaluation tools?

Despite their clear limitations, can they stillbe used in a participatory way that letsstakeholders agree on what they are goingto do and how they will measure this? Thedebates around logframes have typicallybeen highly polarised.There are those whohave made the decision that the LogicalFramework Analysis (LFA) is a ‘LockFrame’ that does not assist with planningor monitoring poverty reductionprogrammes. Others grin and bear it asthey struggle on with them,acknowledging their imperfections, butfeeling the logframe is still the best toolavailable.

In research for Sida conducted byINTRAC, it was concluded that the LFAmay be deeply flawed - but that it is acomponent of currently in-vogue results-based management and can involveintensive stakeholder participation, at leastat the planning stage.

Taking logframes to OmanINTRAC has been working in Omansince November 2005 with a small groupof CSOs registered with the Ministry ofSocial Development. Omani civil society isstill in its infancy - the first four CSOswere only registered in 2001. In March2007 I had the pleasure of working with agroup of Omani NGO leaders,introducing them to the practice of projectmanagement. We discussed the use ofdifferent tools to help with projectplanning, monitoring and evaluation.

As all the participants were newcomers todevelopment and project planning

Such positive feedback received duringand after introducing the logframe inOman made me begin to question themany negative connotations associatedwith it, and the frequency with whichlogframe discontent is voiced by bothNorthern and Southern developmentworkers.

This begs the question:Is the logframeinherently bad or can italso be used in a waythat is participatory andsensitive to local realitiesand context?

The Omani associations were introducedto the logframe as a management tool tohelp them think through what theywanted to achieve in their ownorganisations. It was not imposed on themas a crude matrix that they had tocomplete in order to obtain funding.Theparticipatory way in which they wereintroduced to the traditional logframe wasfocused on their own needs.

INTRAC’s long-established M&E workhas focused predominantly onparticipatory approaches. Our experienceof working with the Omani CSOs showsthat this approach does not excludeproviding support and understanding ofthe traditional logframe in order to enablea better understanding of differentcontextually-tailored approaches to M&E.

Anne Garbutt is INTRAC’sConsultancies Director

Email: agarbutt@intrac.org

processes, they had not been exposed tothe concept of the project cycle or theLFA. During a previous scoping visit, allthe associations’ leaders had clearly statedthat one of the major weakensses was thatthe projects they supported were oftenbased on one individual’s impression of acommunity’s needs rather than the wholecommunity’s perceptionof their collective needs.During our weektogether in March webegan a process ofidentifying the coreproblem, involvingstakeholders indeveloping a problem treefollowed by an objective tree and finallydeveloping a full logframe for a group ofseven associations.At these meetings the participantshighlighted three problems they face, anddesigned three projects using the logicalframework process. This was generallypopular with the participants as it allowedthem to look practically at communityproblems. They commented that it gavethem the opportunity not only to learnfrom the facilitator, but also to spend agreat deal of time learning from eachother. It also gave the facilitator the chanceto ensure that the learning process touchedon the reality of the Omani situation, andwas not simply based on external theoriesof no local relevance. They saw thelogframe as a simple tool that helped themthink through what they wanted to do,how they were going to do it and howthey would monitor the projectimplementation.

Rediscovering the Logframe?

“They saw thelogframe as a simpletool that helped themthink through whatthey wanted to do.”

S4029 7/9/07 16:53 Page 3

4

Many of us who work indevelopment understand that social

development can be distinguished fromeconomic development. However, wecontinue to see monitoring and evaluation(M&E) practices, as well as povertyreduction strategies, as primarily informedby an economic perspective. Economicperspectives imply a tendency to look forand value things that are measurable andtangible.

M&E thus continues to focus heavily onthe management of output and results.Wecontinue to hear a very strong plea forresults from donors and from their back-donors.Accordingly, many organisations inthe South, sponsored by their Northernpartners, are increasingly required todemonstrate results to ‘prove’ the impact ofthe financial support for their work.

While this economic perspective is ofvalue, it has created a world where thecomplexity, dynamism and multi-dimensional nature of specific localcontexts – which are central to socialdevelopment – are often overlooked. Byfailing to pay attention to the invisible,intangible forces that shape dynamicrelationships, we lose the opportunity toappreciate situations in their wholeness.These interconnections and relationshipsare constantly evolving. By approachingthem as if they are fully formed, we makethe mistake of treating them as finite anddiscrete. So, if we focus predominantly oneconomic indicators for measuring theimpact of our interventions in povertysituations, we overlook the dynamism,multi-dimensionality and complexity ofthe forces that have created them.

Measure the intangibleWe all use measurement in our work, andso we have to bear in mind thatmeasurement that only focuses on thevisible, tangible aspects does not do justiceto the more subtle aspects of life. Theproblem with this kind of measurement isthat is seeks to reduce, simplify and bringorder to situations and phenomena that areinherently messy, complex and chaotic.

Also, this kind of measurement, as part ofM&E, is routinely undertaken as anexternal act that denies dynamicinterconnections and relationships. It seeksto control and justify.

The challenge for those that focus onsocial development is to ensure thatmeasurement is combined with learning.We have to recognise the dynamism andcomplexity of situations and phenomenain order to make measurement morenuanced and insightful. Measurement thatseeks to apprehend meaning takes time. Ithas to be undertaken gradually andthrough approaches and methods thatreveal those invisible forces that patternwhat appears on the surface. Wheremeasurement seeks to apprehend meaning,it has to be seen as a longer-term,endogenous and creative process.

‘What changed for me’: Stories asM&EThose who have used stories can attest totheir effectiveness as vehicles forapprehending meaning. Recently, I wassupporting an organisation to collectstories in a rural community to find outpeoples’ perceptions of developmentinterventions and to establish thedifference these have made in the lives ofcommunity members. Before capturingstories, the organisation had undertaken abaseline survey which, it was envisaged, thestories would supplement.

In this way, people were heard and theirexperiences recognised and valued. Bylistening to the stories it became clear tome that any attempt to measure impactshould acknowledge and value theperspectives of community membersthemselves. Such perspectives provide acritical window and reveal a great dealabout the invisible forces that shape and

pattern the more tangible, visible aspects ofphenomena and situations. For example,the empirical baseline census undertakenby the organisation had indicated anincrease in the number of jobs created bythe scheme. However, it was through thestories that the organisation was able toestablish how the quality of peoples’ liveshad improved.

From this exercise, it became clear thatstories are an effective vehicle to helppeople construct meaning. Stories thusbecame the basic metaphor through whichpeople expressed their fears, frustrations,desires and aspirations for a life that wouldhave meaning and direction. It is throughstories that meanings were revealed. Suchan approach took into considerationpeoples’ ability to affect differentphenomena in their lives.We were able tolisten and see how these meanings hadchanged over time.This confirmed for methat meanings are steeped in the context.Through peoples’ stories, manyperspectives about the developmentinterventions emerged. The stories thushelped us to see that human engagementis, first and foremost, a process ofnegotiating meaning. Through storiespeople were helped to make sense of theirlives and, through this, to potentiallyremake them.

By using stories, the organisation was ableto move beyond measurement that onlyfocuses on the visible and was able topenetrate and apprehend those invisibleelements that constitute so much of whatdevelopment is about.

Nomvula Dlamini is a Practitioner at theCommunity Development Resource

Association (CDRA), South AfricaEmail: nomvula@cdra.org.za

“Peoples’ stories provide acritical window and reveal agreat deal about the invisible

forces that shape tangiblephenomena.”

Stories and M&E

M&

E o

f C

BO

s in

Eas

t A

fric

a.Sa

ra M

ethv

en,I

NT

RA

C 2

007

S4029 7/9/07 16:53 Page 4

5

The polarisation betweenaccountability and learning is a central

issue in today’s monitoring and evaluation(M&E) debates. In spite of a constantreference in all M&E literature to itslearning purpose, most M&E practitionersvery quickly admit that the majority ofmonitoring or evaluation efforts areprompted by accountability requirements.Even organisations that have developed in-house M&E systems and who arecommitted to organisational learning, findthat the information collected is usedprimarily to underpin annual reporting,reporting to donors or, in the best casescenarios, for downward accountability.M&E systems almost always pay lip-serviceto the dual goal of accountability andlearning, yet, in practice, few have managedto achieve this.We may learn by adheringto our organisation’s M&E system, butlearning is neither the main driver nor thekey output.

It may be said that learning from practice(your own and others’) is the best way toimprove performance.Therefore, if we onlyfocus on accountability, we may knowwhether we are improving performance, butthe question of how to improveperformance remains unanswered. Often,this is where accountability mechanisms failto improve effectiveness.

Conditions for LearningJust like accountability, the process oflearning requires many elements to be inplace to create favourable conditions for it.1. Organisationally, learning needs to be

promoted and rewarded, so that peoplebecome excited about innovation,experimentation and risk taking.

2. Staff need to have the space to slowdown and go from ‘action’ mode to‘reflection’ mode.The search to identifybest practice needs to be part of the jobdescription of those makingprogramme decisions.

3. Learning found in one programme orin one geographic region needs to be

generalised, internalised andapplauded across theorganisation.

Learning is often expected tohappen spontaneously, but this isonly rarely the case. Morecommonly, learning requiresfocused time and resources before itbecomes incorporated into the culture ofthe organisation. Clearly, without resourcesand organisational backing, learning willnever be at the core of programming.

Integrating LearningThere are many different ways of buildinglearning into programming. Most of theseare standard.They include:• partner meetings, programme meetings

and cross-organisational meetings;• partner exchanges and networking;• focus group meetings, workshops and

training courses for partners and staff;• participation of partners and project

staff at external thematic conferences;• presentations at different organisational

events;• evidence-based research that is fed back

into programming;• monitoring/evaluation of programme

progress, for example in annual reviews,and analysing the results.

However, while learning at partner,programme or even individual staff level isongoing, there seems to be a real difficultyin disseminating this learning in a way thatis useful to others in the organisation. Onceagain it seems to boil down to the same oldchallenge forever facing monitoring andevaluation – how to manage theinformation.

Many organisations have beenexperimenting with variations on fora forsharing knowledge. At M&E level, theseinclude the “Impact Reviews” in SCF,ActionAid’s Participatory ReviewReflection Programme PRRP andOxfam’s “Learning Reviews”.

Trócaire, too, is investigating the idea of alearning review. It seems to be key to thesuccess of a learning review to make it part

of the programme cycle, rather than anadditional activity. Because learningrequires the “slowing down” mentionedabove, it does not always take place if timepressure and workload squeeze it out of thework plans. Nonetheless, to learn andimprove, you have to slow down on day-to-day practical work. This feels almostcounter-intuitive to results-drivenprogramming.

Will we, then, need to create a newparadigm of programming if we want tomove learning centre-stage? One exampleof this would be to use AppreciativeInquiry in order to navigate through theprogramme cycle. Appreciative Inquiry isan approach that is designed to improvefuture programming by building onpositive experiences. It encourages you toconstantly ask constructive questions aboutthe performance and direction of yourwork. It cannot serve purely as an M&Etool, as it focuses on building, improving,re-generating and not on recording,reporting and justifying. Instead it requiresa paradigm shift, forcing you to re-thinkhow you analyse achievement and progress.

Moving towards such a paradigm, whileprobably the most effective way of movinglearning centre-stage, may not be an optionfor agencies bound to pre-set (donor-driven) approaches. This does not mean,however, that learning cannot become thedriver of an M&E system. It does mean thatnew ways of promoting learning withinorganisations must be researched, piloted,recorded and shared. This is necessarybecause the current learning models(developed mainly around organisationaldevelopment rather than programming) donot seem to be able to deliver.

Alix Tiernan is Trócaire's ProgrammeCycle Management AdvisorEmail:ATiernan@trocaire.ie

Learning at the Centre-Stage

“Will we need to create a newparadigm of programming if we want

to move learning centre-stage?”

Sara

Met

hven

,IN

TR

AC

200

7

Learning in an organisation is both spontaneous and structured.

Linda Lönnqvist, developmentcartoons.com

S4029 7/9/07 16:53 Page 5

6

IntroductionThe use of qualitative information is onthe back foot in M&E, even thoughqualitative information provides rich andrelevant information for learning.However, current management paradigmsare preoccupied with results-basedmanagement, quantifiable economicindicators and a preference forquantifiable, apparently ‘hard’, data.Development agencies have becomeincreasingly preoccupied with a drive todemonstrate (upward) accountability. Inorder to do so they tend to gear their‘performance information’ systems towardsuse of quantitative systems and indicators.

To link the two ways of thinking, there isQUIP – a more balanced approach that isaimed at collecting qualitative informationin a systematic and structured way. Itresponds to managers’ needs for data that isuseful, credible and timely.

What is QUIP? Practitioners are often keen to experimentwith mixed methods approaches but find itdifficult to get qualitative data takenseriously.To respond to this need, as part ofthe action-research Imp-Act programmein 2001-04, a protocol was designed at theUniversity of Bath called the QUIP(Qualitative in-depth individual impactassessment protocol). It helps NGOmanagers identify who benefits and wholoses out from the services they offer, andwhy this might be.The QUIP thus servesas a protocol, rather than as a blueprint.This example builds on M&E ofmicrofinance projects, but the QUIP canbe adapted to most development activities.

The QUIP in nine steps1. Deciding what data is neededValue-based M&E forces one to thinkcarefully about appropriate methods andanalysis based on accurately scoping whythe data is being collected and what needit is to serve.The data you collect needs tobe useable and intelligible without creatinginformation overload. Are there particulargroups of users or specific issues that you

need to know more about? Table 2provides a checklist that may help. TheQUIP should be open to unexpectedfeedback from users of the services thatNGOs provide.

2. Identifying baseline information: What isbaseline data and how can it be useful?Once you have decided who should do thework, it may help to identify baseline data.Baseline data is information about selectedservice users from an earlier date, whichcould be anything from six months to twoyears before an in-depth interview. Suchdata proves an invaluable reference point,making it easier to identify where positiveand negative changes have taken place overa given period of time. It can be used as areference point for specific questions andmeans in-depth interviews can be shorter.Repeated visits also deepen rapport andtrust, improving the quality of theinformation.

3. Identifying who should be in the sample forthe qualitative interviews.This might involve either identifying a

Getting your qualitative data taken seriously: using QUIP

sample of users that you would like toknow more about or selecting users ofservices at random.

4. Designing a semi-structured interviewschedule.This involves thinking about the questionsto ask, the way that they are asked and theorder of asking a combination of closedand more open questions. One way is toask an initial open question on eachtheme, followed by supplementaryquestions to probe for more detail.

5. Preparing for interviews.It is important to pilot the schedulethrough testing and revising drafts throughmock interviews, initially with staff, butalso with two real users who have a similarprofile but who are not part of your actualsample. Issues to bear in mind include:Wasthe interview too long/short? Were thequestions clear? Has a major topic beenomitted? Was the order of questionssatisfactory? How easy was it to take notesand record responses? Are the questionsgenerating the kinds of information thatwill answer your research questions?

6. Conducting interviewsGetting permission to interview andchoosing an appropriate time and placewhere respondents can relax with minimalinterruptions is important. It is alsonecessary to think through who will bepresent during the interview, since this willaffect what people are willing to disclose.Making a written record of interviews andtyping these up according to themes willsave time in subsequent analysis.

7.Analysing and interpreting narrative dataIt is best to type up notes following theorder of questions in the interview outline.Typed summaries or transcripts make iteasier to rearrange data into themes.Identifying key issues that came up undereach theme is a first stage. Copyinginformation from case studies into anotherdocument so that it is organised by themeand issue, rather than by interview, mayalso be useful. The next task is to write a

Questions

1. Who are my clients? (differentiated by age, gender,ethnicity, education, experience)

2. How and why is their demand for theservices we offer changing?

3. What are the current levels of satisfactionwith the services they use? (What do they like and dislike?How is their use of the servicechanging?)

4. What are the direct impacts of theseservices on the users?(Material impact, changes inknowledge, skills, attitudes andbehaviours, changes in the quality ofpersonal relationships)

5. What are the indirect impacts of theiruse of these services?(On gender relations, children,community activity etc)

Table 2: Checklist of possible research objectives

S4029 7/9/07 16:54 Page 6

7

INTRAC PeopleStaff brave flood watersDespite the approach roads to the office beingunder 18 inches of flood water, in late Julyseveral members of staff donned their designerwellies and came into the office.A sterling jobwas done in of lifting computers and books to‘higher ground’. Fortunately, although thewaters came up our drive, they did not get intothe building.Those who travel in by bike foundthat water resistance cycling is excellentexercise! Rob, from our IT support, ComputerAssistance, also braved the waters to give staffincreased remote access to documents so thatthey could work from home. Many thanks toall for a great team effort – we even had thehelp of our volunteer, Joan Turner, and Julius, astaff member’s son in getting books back intothe Library after the event!

PromotionIn April we celebrated the well-deservedpromotion of Zoë Wilkinson from EventsCoordinator to Events Manager. Zoë is tothank for running INTRAC’s sell-out events.

Welcome to new staff Tom Travers has joined us as FinanceManager.Tom has wide-ranging experienceboth in the not-for-profit and public sectors.We have kept him busy since joining us as hisfirst job was to do the end-of-year accounts!

We’ve been happy to work with volunteerMohamed Elamin from the RefugeeResource Centre. Mohamed helped withadministration and running the website, andprovided invaluable translation and researchon Arabic-speaking civil society.

June brought Luciane Correa Andrade towork with us as Office Administrator.Luciane is from Brazil and this has ensuredthat we have a member of staff from everycontinent – a truly international team!

Written by Shelagh Windsor-Richards,Projects Manager, INTRAC

Email: swindsor-richards@intrac.org

clear and thorough overview ofwhat the collected evidence revealsabout each issue and theme.

8.Analysing and converting qualitativedata into numerical scoresBy reviewing evidence ininterview transcripts it is possibleto give a score (on a scale of -5 to+5) for each theme. These scorescan then be added together toobtain a single measure of impactwhich can be compared fordifferent types of respondent,according to location, personalcharacteristics, services received,and so on. A more direct way ofarriving at a score is to askrespondents themselves to rankimpact in numerical terms at theend of each section of openquestions. Once each user has beengiven an overall score, it may thenbe useful to analyse what usercharacteristics are associated withhaving experienced positive ornegative impact. For example, userscan be divided into two categories– those with overall positive andthose with negative scores.

9. Using the data to help managersimprove the offered services Producing credible qualitative datais not enough. It is important toensure that the findings of this dataare fed back into the internallearning systems of the NGO interms of reporting results,incorporating them into decision-making and piloting majordecisions before they areimplemented.

For more information on theQUIP, and the piloting processes

that have taken place in Peru andZambia, please contact Katie

Wright-Revolledo (INTRACSenior Researcher)

Emails: kwright-revolledo@intrac.org

INTRAC Conference News

Southern Civil Society MeetingThe Appropriation of Development: Civil

Society Perspectives on the Security

Agenda Syria / September 2007

INTRAC has held a conference togenerate dialogue between Southern civilsociety representatives about theirexperiences of security impacts on theirwork.The main objectives were:• Disseminating the findings from

INTRAC’s regional workshops • Reporting back on the broader

consequences of current counter-terrorism measures (CTMs) for NGOsand for international development

Northern Policy MeetingCounter-Terrorism Measures, Security

and Development Worldwide

The Netherlands / 1st-2nd November 2007

INTRAC, ICCO and CORDAID will beholding a two-day internationalconference analysing the effects of securitypriorities on development cooperation.This is aimed at • Generating dialogue between civil

society and policy makers in Europe.• Providing support for a north-south

NGO network on issues of CTM anddevelopment priorities.

INTRAC Counter-Terrorism Measures and Civil Society EventsOver the past year INTRAC has organised a series of regional workshops around the worldinvestigating the impact of counter-terror strategies on the operations and relationships ofNGOs and CSOs.These workshops were held in Central Asia, Europe, the Middle East,South Asia and the USA. For further details see:www.intrac.org/pages/ctm_workshops.html

The central issues and findings from the regional workshops will be disseminated in thefollowing conferences (with attendance by invitation).

For information on INTRAC Events, please contact Zoë Wilkinson at zwilkinson@intrac.org

S4029 7/9/07 16:54 Page 7

Editors: Linda Lönnqvist, Researcher and Katie Wright-Revolledo, Senior Researcher, INTRAC.is the newsletter of (the International NGO Training and Research Centre). It is published three times a year. The contents of the newsletter

may be freely reproduced and translated, providing the source is acknowledged. wishes to thank the following organisations for their contributionstowards the production of : Concern Worldwide, Cordaid, DanChurchAid, Oxfam Novib, Save the Children Denmark, Save the Children Sweden, ICCO,

Trócaire and World Vision UK. Designed by Seacourt Ltd.

To subscribe to , please contact (info@intrac.org) indicating whether you wish to receive it by email (English, Arabic, Chinese, French, Portuguese, Russian or Spanish) or post (English and Russian only).

You can also subscribe on our website: www.intrac.org/pages/bulletin.html

PO Box 563 • Oxford OX2 6RZ • United KingdomTel: +44 (0) 1865 201 851 • Fax: +44 (0) 1865 201 852 • Email: info@intrac.org • Website: http://www.intrac.org

INTRAC is a limited company No. 2663796 and a registered charity No. 1016676

ISSN 1476-1327

INTRAC PublicationsOut Autumn 2007: ‘Rethinking M&E - Challenges and Prospects in theChanging Global Aid Environment’ Esther Mebrahtu, Brian Pratt and Linda Lönnqvist, £17.95

Do any of these issues sound familiar? • It feels like your monitoring and evaluation (M&E) has to demonstrate that your project is

perfect;• Your funder demands M&E data that’s too cumbersome to collect;• M&E feels more like an impediment than a resource to your work;• Your M&E doesn’t really touch on poverty alleviation, but only with project outputs;• You have spent more time designing your monitoring system than you do using it.

If you grapple with these questions, you’re not alone. Development practitioners worldwideare trying to deal with similar problems – and they are finding ways of doing so. INTRAC’snew book ‘Rethinking M&E - Challenges and Prospects in the Changing Global Aid Environment’ incorporates the goodexamples and innovative M&E solutions of 120 development professionals from an enormous range of countries, circumstances andspecialisms.

‘Rethinking M&E’ is based on INTRAC’s Sixth Evaluation Conference and regional M&E workshops in Ghana, India, Swedenand Peru, and includes perspectives from both NGOs and CSOs, donor ministries, activists, think-tanks and foundations.Emphasising Southern perspectives and covering a rich variety of experiences, it stresses the important role of M&E in challengingmany of our assumptions about poverty alleviation.

‘Rethinking M&E’ both analyses practitioner issues and situates them within wider aid trends.Within a policy context,‘RethinkingM&E’ provides innovative insights into such areas as M&E of NGOs as donors, the M&E of advocacy and non-written M&E.

For pre-ordering a copy at a 10% discount, please contact Kennedy Mbevi at kmbevi@intrac.org

Praxis Paper 19: Leadership in transition by Charles Buxton and Kazbek Abraliev. http://www.intrac.org/pages/praxisseries_publications.html

The Central Asian state of Kyrgyzstan is in a state of transition in which civil society could play a historical role. Kyrgyz civil societyorganisations could inspire society and offer a relatively ethical, open and democratic form of leadership. But what elements oftraditional or Soviet-style, top-down leadership can they build on? What can be learnt from the experience of those now managingcivil society organisations?

‘Leadership in Transition’ can be downloaded for free, or paper copies ordered for £5.95 as above.

Printed by Seacourt to the most stringent environmental systems using Waterless Offset (0% water

and 0% Isopropyl alcohol or harmful substitutes), 100% renewable energy and vegetable oil based inks.

Seacourt is registered to EMAS and ISO 14001, is a CarbonNeutral® company and FSC accredited.

S4029 7/9/07 16:54 Page 8