Guidelines for Conducting Joint Evaluations of Humanitarian Action

8/22/2019 Guidelines for Conducting Joint Evaluations of Humanitarian Action

1/21

WhatWeKnowAboutJointEvaluationsofHumanitarianActionLearningfromNGOExperiencesSection1of3:THEGUIDE

April2011


2/21

WhatWeKnowAboutJointEvaluations TheECBProject [email protected] April2011(v6) 2

ContentsACKNOWLEDGEMENTS......................................................................................... 3

ABOUTTHISBOOKLET .......................................................................................... 4

THEGUIDE.............................................................................................................. 5

CHAPTER1:WHYDOAJOINTEVALUATION? .......................................................... 5TheBenefitsofaJointEvaluation ............................................................................................... 5

TheDownsidesofaJointEvaluation ........................................................................................... 6

Conclusion ................................................................................................................................ 7

CHAPTER2:JOINTEVALUATIONWHEN,WHOANDHOW? .................................... 8Whenwillittakeplace? ............................................................................................................. 8

Whowilltakepartinit? ............................................................................................................. 8

Isthereenoughtimeforajointevaluation? ................................................................................ 8

Howwillitbepaidfor? .............................................................................................................. 9

Howcanthejointevaluationbemostusefultovariousstakeholders? .......................................... 9

CHAPTER3:WHATTODOBEFORETHEEVALUATION ............................................ 10Choose

alead

agency

and

agree

on

roles .................................................................................. 10

Setupamanagementstructure................................................................................................ 10

Estimatecostsandduration ..................................................................................................... 11

Communicatewhattheevaluationisabout............................................................................... 12

Findacompetentadministrator/manager................................................................................. 12

Carefullypickevaluationteammembers................................................................................... 12

Chooseafewobjectivestocover.............................................................................................. 14

Agreeonevaluationstandardsandmethods ............................................................................. 14

Writeaninceptionreport......................................................................................................... 15

Managecommunicationswithinthecollaboration..................................................................... 15

Prepare,prepare,prepare!....................................................................................................... 15

CHAPTER4:WHATTODODURINGTHEEVALUATION ............................................ 16

Briefthe

team

upon

arrival ...................................................................................................... 16

Sharefindingsasyougo........................................................................................................... 16

Ensurefindingsarereportedwithsensitivity ............................................................................. 16

FinalizingtheEvaluationReport................................................................................................ 17

CHAPTER5:WHATTODOAFTERTHEEVALUATION............................................... 18Developbothcollectiveandindividualrolloutplans................................................................... 18

Emphasizepeeraccountability ................................................................................................. 18

CHAPTER6:JOINTEVALUATIONSINREALTIME .................................................... 19Preparefortheevaluationbeforetheemergencystarts............................................................. 19

Takeagoodenoughapproachtotheevaluation..................................................................... 19

Callonadditionalresources ..................................................................................................... 19

Considersomeotherjointreflectionprocess............................................................................. 20

Referencesand

Further

Reading.......................................................................... 21


3/21


ACKNOWLEDGEMENTSManypeoplehavesharedtheirvaluableexperienceandtimeinthecreationofthisbooklet.Special

thanksgotoallofthem,particularly,inGuatemala,CarlaAguilarofSavetheChildrenUS,HughAprileof

CatholicReliefServices,BorysChinchillaofMercyCorps,andJuanManuelGironDurinioftheECBProject;

inNiger,JasmineBatesandMariannaHensleyofCatholicReliefServices,andJuliannaWhite,ofCARE;in

Indonesia,AdhongRamadhanandJosephineWijiastutiofCatholicReliefServices,AgusBudiartoandEvi

EsalyKabanofSavetheChildren,HariningMardjukiandAnwarHadipriyantoofCARE,andRichardusIndra

GunawanandYacobusRuntuweneofWorldVisionInternational.SpecialthanksalsogotoJohnWilding,

PaulineWilson,JohnTelford,MauriceHersonofALNAP,JockBakerofCAREandGuySharrockofCatholic

ReliefServiceswhohavegivencriticalinputintothiswork.MalaikaWrightwastheauthorofthefirst

paper.

TheApril2011versionofthebookletwasupdatedbyKatyLovefromtheECBProject,LorettaIshidaof

CatholicReliefServices,JockBakerofCARE,HanaCroweofSavetheChildren,andKevinSavageofWorld

Vision.ThebookletwasrevisedbasedonfeedbackandreportsfromthosewhoparticipatedinECB

supportedjointevaluationsin2010inIndonesia,Haiti,theHornofAfrica,andNiger.Thesepeopleserved

asevaluationmanagersandcoordinators,teamleaders,teammembers,SteeringCommitteemembers,

ECBfieldfacilitators,andECBaccountabilityAdvisersinjointevaluations,including:PaulOHagan,Greg

Brady,YvesLaurentRegis,AngelaRouse,KatyLove,andJockBaker(Haiti);YenniSuryani,PaulineWilson,

LorettaIshida,andLeAnnHager(Indonesia);KevinSavage,CheleDeGruccio,JimAshman,andWynn

Flaten(HornofAfrica);andKadidaMambo(Niger).TheECBProjectthanksallwhocontributedtothis

work.


4/21


ABOUTTHISBOOKLETThisbookletwaswrittentoshareknowledgegainedfromtheexperiencesofpeoplethathavebeen

involved

in

joint

evaluations

conducted

by

non

governmental

organizations

(NGOs).

It

mainly

profiles

the

workofNGOsinvolvedintheEmergencyCapacityBuildingProject(ECB),whichhasagoaltoimprovethe

speed,qualityandeffectivenesswithwhichthehumanitariancommunitysaveslives,improvesthe

welfare,andprotectstherightsofwomen,menandchildrenaffectedbyemergencies.

Thisbookletalsodrawsonthelessonsofmultiagencyevaluationsthatalreadyexistwithinthe

humanitariansector.Majorcontributionshavecome,inparticular,fromtheActiveLearningNetworkfor

AccountabilityandPerformanceinHumanitarianAction(ALNAP).

Wehopethatlearningfrompreviousexperiencescapturedherewillbeusefulforallthoseconsidering

leadingtheiragenciesthroughajointevaluation.Thelearningsharedhereistargetedatevaluation

practitioners,managers,andNGOscontemplatingajointevaluation.Additionally,wehopethatitwill

contributetoagrowingbodyofknowledgeontheseprocessesandshowthatwhiletherearemany

unansweredquestions

about

joint

evaluations,

there

is

alot

we

already

know.

Thisbookletiscomprisedofthreesections.Thefirstsection,TheGuide,canbereferredtoasahowtoforthosecloselyinvolvedinjointevaluations.Itprovidesaframeworkforthoseapproachingan

interagencyevaluation.Thesecondsection,TheStories,sharesseveralcasestudiesfromtheECBProjectsexperiences.Thethirdsection,TheTools,includesmanytemplatesandtoolsthatcanbeadaptedforevaluations,includingsampletermsofreferences,agreementdocuments,andchecklists.


5/21


THEGUIDECHAPTER1:WHYDOAJOINTEVALUATION?Whyshouldanagencyconsidertakingpartinajointevaluationofanemergencyresponseprogram?

1

Afterall,jointevaluationsrequirecollaboration,collaborationmeansmoreworkandtime,andtimeisa

scarcecommodityinemergencyprograms.

Inrecentyears,severalNGOshavesoughttoanswerthisquestionwhiletakingpartinthejoint

evaluationsprofiledinthisbook.Whiletheresultshavebeenmixedandthelearningcurveshavebeen

steep,jointevaluationsconfermanybenefits.Whiletheevaluationsthemselveshaveyieldedinstructive

andusefulfindings,agencieshavealsobenefitedsignificantlyfromthequalityoftheinteractionsthat

tookplaceamongpeeragencies.Jointevaluationsoftenserveasforumsforongoinglearning,dialogue

andevenbegincollaboration.

Agenciesalsoinevitablylearnthattherearesomepitfallsintheprocessofconductingjointevaluations.

Thoughajointevaluationisnotsodifferentfromasingleagencyevaluation,therearesomemajor

differences,someofwhicharehighlightedbelowandaddressedthroughoutthisguide.

Aboveall,likeasingleagencyevaluation,ajointevaluationprovidesanopportunitytolearnfrompast

actionsoastoimprovefuturedecisionmaking.

Itshouldbenotedthatthisguidesetsouttheidealprocessesandstructuresforajointevaluation.Inan

emergencysetting,ofcourse,constraintsemergethatmaketheidealprocessachallengetoachieve.

Evaluators,therefore,mustbeflexibleandwillingtoadapttotherealitiesonthegroundinorderto

achievesomeifnotalloftheobjectivestheysetouttoachieve.2

TheBenefitsofaJointEvaluation1.

Seeing

the

Big

Picture

Oneevaluatorsaid,You[may]thinkyouvecoveredtheworldbutyouveonlycoveredone

villageinten.Emergencyresponsestypicallyinvolveseveralhumanitarianactors.Whenthe

responsesofmorethanoneactorareputsidebysideandexamined,theoverallpicture

becomesclearer,revealinghowfactorssuchasgeographiccoverage,sectorspecific

interventions,andcommunityinvolvementallfittogether.Jointevaluationsgofurthertowards

measuringimpactbylookingatthecollectiveeffortsofseveralactorstomeetbeneficiaryneeds

andtoidentifywhatgapsexist.

2. BuildingCoordinationandCollaborationtoImproveResponseGiventhescaleofdisastersandthedisproportionateamountofsufferingtheycause,agencies

workingalonearegenerallynotabletohavealargeimpact.Infact,agenciesthatcoordinate

responsesandworktogetherduringemergenciesarebetterabletomeettheneedsofdisaster

affectedpopulations.

By

comparing

agencies

responses

side

by

side,

joint

evaluations

are

better

abletopointoutareaswhereNGOscouldhaveactedinacomplementaryfashionandmake

1Themodelanddefinitionofjointevaluationsusedinthisbookletisanyevaluationthatlooksattheworkofmorethanoneagency.

Thisusuallymeansthatinadditiontomoreactorsbeinginvolved,thereisagreaterbreadthofprogrammingbeingexamined.2ReaderslookingforfurtherguidanceshouldreviewShoestringEvaluation:DesigningImpactEvaluationsUnderBudget,Time,and

DataConstraintsbyBamberger,Rugh,Church,andFort.


6/21


recommendationsforhowtheycoulddosoinanticipationofthenextemergency.Evaluation

reportsrepeatedlyshowthatbettercoordinationwouldhaveledtoamoreeffectiveresponse.

Insomecaseswhereagenciesarealreadyworkingtogether,ajointevaluationcanbealogical

conclusiontoajointactionorresponse.InIndonesiaandinNigerin2010,agenciesagreedto

conduct

a

joint

evaluation

to

assess

the

impact

of

their

joint

activities.

3

Theprocessofcollaboratingontheevaluationitselfcanalsobeapowerfulwayofbuilding

relationshipsamongpartneragencystaffthatendureforthelongterm.IntheECBexperience,

someoftheserelationshipshaveledtoongoingactivitiesandeventheformationofanNGO

coordinationforum(seeNiger).InHaiti,thejointevaluationhelpedtobuildrelationshipsamong

nationalstaffandmanagers,servingasastartingpointforlongerterminteragency

collaboration.

3. WieldingWeightierConclusions,ImprovingPeerAccountabilityandTransparency

Jointevaluationscanbemoreauthoritativebecauseofthecombinedweightofthosebacking

them.As

such

evaluations

are

available

to

awider

audience,

there

is

likely

to

be

greater

pressure

toactupontherecommendations.Additionally,theyprovidealargerbodyofevidencefor

purposesofjointadvocacy.

Whenagenciesopenuptooneanotherbysharingweaknessesandstrengths,theyincrease

transparencyandmakeiteasierforthemtoholdonanotheraccountableforactinguponthe

recommendations.Transparencyiscriticalforagenciesinhumanitarianresponses,andsharing

thefindingsofevaluationsacrossagencieshelpstobecomemoretransparent.Infact,agency

peersmaypressuretheagencyactonrecommendationsfromanevaluation.

4. LearningfromandRelationshipBuildingwithPeersPartnersinajointevaluationhavearareopportunitytolearnabouteachothersprogramming

and

operations,

and

may

share

technical

knowledge

through

the

evaluation

process,

but

also

throughtheongoingrelationshipsthatareoftenestablished.Onepractitionernotedthat

workingwithstafffromotheragenciessometimesbringsnewperspectivesorevenchangesher

thinkingaboutaparticularissue.

Therelationshipbuildingthatoccursthroughajointevaluationallowsagencystafftoidentify

otheragenciesstrengthsandcapacities.Therelationships,foundedontrust,thatarebuilt

throughajointevaluation,mayresultinagencycooperationinthefuture.

TheDownsidesofaJointEvaluation1. MoreComplexity

Ittakes

time,

skill

and

patience

to

get

agencies

to

agree

to

do

ajoint

evaluation,

agree

on

amanageablelistofobjectives,diffuseanytensionsthatmayarise,ensurethatgroupdecision

makingprocessesareclearandrespected,allwhiledealingwithhiringandsupervisingan

evaluationteam,settingupinterviews,ensuringlogisticsareinplace,etc.Thisbecomeseven

harderduringanemergency.

3Seethe2010Indonesiajointevaluationreportformoreinformation,availableatwww.ecbproject.org/resources


7/21


Withoutaleadagencytotakeontheprimaryresponsibilityforthesetasks,andacommitted

steeringcommitteethatcanjointlyhandlestrategicdecisionmaking,ajointevaluationcanbe

frustratingandunsuccessful.

2. LessDepthOften

it

is

not

feasible

or

relevant

to

go

into

much

detail

on

any

particular

agencys

programs

as

wouldhappeninasingleagencyevaluation.Thereforemanyoftheevaluationquestionsof

interesttoeachagencymaynotgetanswered.3. MoreExpensive

Giventhenumberofactionsinvolved,jointevaluationscansometimesbemorecostlythan

singleagencyevaluations.Ifagenciesagreetosharethecostsoftheevaluation,however,

additionalcostsperagencywillbeminimal.

ConclusionJoint

evaluations

allow

NGOs

to

learn

from

multiple

perspectives

and

given

them

amore

complete

understandingofanemergencyresponse.Theyhelpusworktogethernowandinthefutureandleadto

relationshipsthatcanbeveryproductive. Forthesereasonstheycanbeenrichingexperiencesandhave

aprofoundimpactonthewaywedothingsasindividualagenciesandascollectives.Itisimportantto

havearealisticunderstandingofwhatcanandcannotbeaccomplishedbyajointevaluationbefore

conductingone.


8/21


CHAPTER2:JOINTEVALUATIONWHEN,WHOANDHOW?Youmaywanttodoajointevaluationandhavegoodreasonstodoso.Butfirst,makesuretherewill

beenoughtimefortheevaluation,willingpartners,andhuman,financialandotherresourcestogetit

done.Thefollowingquestionsaremeanttohelpyoudeterminewhetherajointevaluationisfeasible.

Whenwillittakeplace?Evaluationscantakeplaceatdifferentpointsofaresponse(during,immediatelyafter,orseveralmonths

after).Thetimingdependsonwhattheagencieswanttogetoutoftheevaluation.Realtimeevaluations

duringaresponseprovideresultsthatcanimprovetheresponsegoingforward(seeChapter6).

Evaluationsconductedneartheendofanemergency,ontheotherhand,captureexperiencesand

learningwhileitisstillfresh.Evaluationsconductedwellafteranemergencyendscanstillbeusefuland

cancapturelongertermimpactofaresponse.

Oneimportantfactorwhencreatingatimelineforjointevaluationsistorememberthatworkingwith

multipleactorscanslowyoudown.Thereisrarelyaperfecttimetoconductajointevaluation,asall

agenciesarebusy.Therefore,especiallyforrealtimeevaluations,itisimportanttostartplanningasearly

aspossibleduringtheemergencyresponse.

Whowilltakepartinit?Approachotheragenciesthatmayalreadybeconsideringanevaluationforthesamehumanitarian

response.Consideragenciesthathavethesameoverallgoal(e.g.ensuringaffectedpopulationsareable

torecoverquicklyfromthedisaster),andthathavesimilartypesofprogramsingeographicareasthatare

closeenoughtogether.Identifytheappropriatepersontocontact,ideallysomeonewhoprovides

strategicdirectionforthecountryoffice.Explainwhatwillbegainedfromdoingthisevaluationjointly

(seeChapter1).Listentotheirviewsandnotethemdown.Dontbediscouragediftheyarenot

interested.Keeptalkingtootheragencies.

Whentalkingtootheragencies,findouthowtheyapproachevaluations.Dotheyconductthembecause

donorsrequirethem?Howdotheyusethefindings?Whatresourcesdotheydesignateforevaluations?

Takenoteofthistogetasenseofhoweachagencywillapproachtheevaluationandusethefindings.

Theiranswerswillalsohelpprepareyouforpotentialareasofconflict,suchaswillingnesstocontribute

stafftime. SeeTheTools,SuggestedTopicsforDiscussionwithProspectivePartners.Besuretheagencies

arewillingtocommitstafftimeandresourcestosupporttheevaluation.

Forajointeffort,andbecauseevaluationsmayrevealsensitiveissues,itsalsoimportanttobuildtrust

amongtheagencies.Todoso,agreeonthefocusoftheevaluationtogether,ratherthanapproaching

otherswithyourvisionandaskingiftheyareinterestedinjoiningin.Continuecollaboratingby

communicatingclearly,

being

transparent

with

information

and

intentions,

and

following

through

with

commitments.

Isthereenoughtimeforajointevaluation?Besuretoallocateenoughtimefortheevaluationteamtogetthejobdone.Unlessthelogisticsofgetting

toandfromfieldsitesisunusuallytimeconsuming,athirtyorfortydaycontractfortheleadevaluatoris

reasonable. Ensuretheevaluatorhasatleasttwodaysbeforeofficiallystartingtheevaluationtodo

preparatoryworksuchastoreviewdocuments,proposemethodology,andplanlogisticswiththe


9/21


agencies.Notensuringenoughtimefortheadditionalworkwillcompromisethequalityoftheevaluation.

Ensurethattimeisbuiltintoaccountformisseddeadlines,asmayoccurwhenmanyactorsareinvolved.

Howwillitbepaidfor?Jointevaluationsusuallytakemoretimetoconduct,andmayrequirearelativelylargeteam.Costs,

therefore,may

be

higher

than

for

single

agency

evaluations.

The

costs

can

be

spread

out

among

agencies,

andthisshouldbediscussedaspartoftheearliernegotiation.

Havearoughideaofwhattheevaluationmaycost.Themaincostsareforhiringconsultantsandsupport

staff.Comparethiswithwhatfundsmaybeavailableandwhatotheragenciesmaybewillingto

contribute,includingstafftime,lodging,andvehicles.Ifinsufficientfundsareavailablefortheevaluation,

considerajointpeerreviewtoreviewoneanothersprogramsandcometogethertodiscussfindings.

Properandrealisticbudgetingiscritical.

Donorsarelikelytobereceptivetojointevaluationsiftheybringaboutabetterunderstandingofthe

contextandtheoverallhumanitarianresponseandsomedonorscommissionjointevaluations

themselves.Therefore,ifabudgetfundedbyagivendonoralreadyaccountsforanevaluation,thedonor

maybeopentoredirectingthatactivityfromthesingleagencysevaluationtocontributetoajoint

evaluation.

Howcanthejointevaluationbemostusefultovariousstakeholders?Evaluationstakealotofresourcesandeffortandeveryonewantsthemtobeuseful.Jointevaluations

canbeusefultodifferentstakeholdersindifferentways.Inalargeemergency,agencystaffatregional

andgloballevelswilllikelybeinterestedinthefindings.

Talktopeopleattheheadofficelevelinthecountrywhere

theemergencyhappened,attheregionallevel,andat

headquarterslevel.Evenifthefindingsrefertoprograms

thathaveended,cantheybeusedtoinformother

programs,systemsandpolicieswithintheorganization?

Iftheproposalfortheevaluationcamefromheadquarters,dothoseinthefield,particularlycountryofficeleadership,

believethatthiswillbeausefulexerciseforthem?Ifnot,

theymaynotwanttoengage,andtheevaluationwillprove

hardtocarryout.Howwilltheyusethefindings?How

committedwilltheybetotheevaluation?Theirinterest

andengagementneedtobehightomakethisasuccessful

experience.

Thereshouldbeareasonablelevelofconfidencethatthe

findingswillbeusedbeforeproceedingwiththe

evaluation.Ifnot,theevaluationteamwillstruggletoachievetheobjectives.

TheideafortheECBsupportedGuatemala

evaluationcamefromheadquarters.Theteamin

GuatemalafeltthatthiswasanotherHQdriven

initiative,sotheirparticipationinsteering

committeemeetingswaslimited.Theagencieson

thegroundtriedtocustomizetheobjectives,butin

retrospectfelt

they

should

have

started

from

scratch.Thisnegativelyimpactedtheevaluation

processandthustheusageofthefindings.

Incontrast,theideafortheECBsupportedjoint

evaluationinJogyakartaalsocamefrom

headquarters.However,theparticipatingagencies

onthegroundtooktheleadondefiningtheir

objectives,withadvicefromheadquarters.This

helpedensurethepartnersweremoreincontrolof

theevaluationprocess.


10/21


CHAPTER3:WHATTODOBEFORETHEEVALUATIONIfyouhavedecidedtopursueajointevaluation,herearesomethingstoconsider.

ChoosealeadagencyandagreeonrolesECBhashadthebestresultswhenoneagencyleadsthejointevaluationprocess.Thoughsomesharingof

responsibilityisdesirable,agenciesshoulddesignatethemajorityofthedaytodaymanagement

responsibilitiestotheleadagency.Anyoftheagenciesbeing

evaluatedcouldserveasthelead;whatmattersisthattheagencyis

capableofcarryingouttheresponsibilities.

Theleadagencyhiresandsupervisestheevaluationteam,coordinates

travellogistics,providesteammemberswithworkspaces,organizes

meetings,andgivesleadershipregardingthedefinitionofthe

objectives.Ultimately,itisthisagencythatisaccountableforensuring

thatthe

evaluation

takes

place.

Asteeringgroupmadeupofrepresentativesfromeachagencycan

cometogethertoagreeontherolesoftheleadagency,theroles

assignedtootherparticipatingagencies,andsharethemwithall

involvedpersons,staffandevaluators.(Formore,seesectionbelow

onManagementStructure.)

SetupamanagementstructureWhensettingupamanagementstructurefortheevaluationitsimportanttorecognizethatyouare

managingnotjustanevaluationbutacollaboration.Dontsuccumbtopressuretomakechoices

favorabletotheleadagency,steeringcommitteemembersorhighlevelsponsorsoftheevaluation(e.g.

wemust

have

x,

y,

and

zrepresented,

and

any

individuals

will

do).

Seek

out

individuals

for

the

steering

committeeandevaluationteamwhoarecommittedtoasuccessfuloutcome,eveniftheyarenot

conventionalchoices.Wherethereisaneedforagencyrepresentation,createspacefortheseindividuals

insomehighprofile,butlesscriticalfunction.

Ajointevaluationmanagementstructurecouldlooksomethinglikethis:

Asteeringcommittee. Thisgroupwillberesponsibleforstrategicdecisionmakingveryearlyonregardingobjectives,timing,andresourceallocation,includingstaffandfunding.Thesteering

committeewillalsobeactiveinreviewinganddebatingthefindingsandactingupontheir

implicationswithintheiragenciesandbeyond.Itisnormallychairedbytheleadagencyandhas

representationfromeachoftheparticipatingagencies.Thecommitteewouldideallybekepttoa

maximumoffive,makingoversightanddecisionmakingmorefocusedandachievablein

reasonableamounts

of

time.

This,

however,

supposes

that

agencies

involved

are

willing

to

delegatestafftoacommittee.

Theidealsteeringcommitteememberisseniorenoughtospeakonbehalfofhis/heragencyand

hastheauthoritytomakedecisions.Thisindividualmusthaveagoodknowledgeofhisorher

organizationsemergencyprogramsandongoingdevelopmentwork.Inaddition,heorshe

shouldbeabletothinkstrategically,andknowenoughaboutevaluationstoadviseonthe

evaluationmethodstobeusedandonthefieldlocationstobecovered. Theseindividualswill

alsobethosemostlikelytofollowuponrelevantrecommendations.

Aleadagencythatplaysits

rolewellcanmakeamajor

differenceintheprocess.The

headevaluatorinECBsNiger

evaluationfoundthelead

agencysorganizationofthe

evaluationprocessand

logisticstobethemosthelpful

thingto

him

in

carrying

out

his

work.Itwasoneissuewe

didnthavetothinkabout;it

wassowellorganized,he

noted.


11/21


Steeringcommitteemembersshouldagreeonanddocumentclearprocessesandstandardsof

efficiency,transparencyandaccountabilityregardingrolesandresponsibilities.Theyshould

agreeonhowdecisionmakingwouldwork,howtoresolvedisagreementswithinthesteering

committee,andhowtoshareinformation. Steeringcommitteemembersshouldbethesame

throughout

the

entirety

of

the

evaluation.

When

members

rotate

on

and

off

the

committee,

decisionsandguidancemaychange,whichwillcomplicatemattersfortheevaluationteam.

Otheragreementsconcernthereportformat,theuseofagencylogos,theownershipofthe

productsoftheevaluation(i.e.intellectualcopyright),howagencieswillusethefindingsandif

theywillholdoneanotheraccountable,etc.

Achairperson.Thispersonisbasedattheleadagencyfortheevaluationandisamemberandchairofthesteeringcommittee.Heorshehasmostofthestrategicdecisionmaking,operational

andcollaborationresponsibilitiesoftheevaluation.Thisindividualusuallyassumestheroleof

evaluationmanagerandisthedirectreportinglinefortheteamleader.Thechairpersonshould

managethebudgetandtrackexpenses.

A

manager

or

administrator.

This

person

must

ideally

be

based

in

the

lead

agency

with

a

certain

percentageofhisorhertimededicatedtotheevaluation.Seethefollowingsectionformore

detailsonthemanagersresponsibilities.Heorshecouldalsositonthesteeringcommitteebut

withoutvotingrights.

Theevaluationteam. Theteamistypicallycomposedofoneortwoindependentconsultants,andarepresentativefromeachofthepartneragencies.Thisteamisaccountabletothesteering

committee,particularlythecommitteechair.

Therearevariationsonthisstructure,ofcourse.Mostevaluationsalsohavehigherlevelsponsors

thatmayalsoformasuperstructure.Sectorexpertsmayalsobeneededontheevaluationteam.

SeeTheToolsforaSampleAgreementsDocument.

EstimatecostsanddurationBasedonthedraftitinerary,thesteeringcommitteeshouldagreeonadraftbudgetandcostsharing

arrangements.Typicallyagenciesshareconsultantcostsequallyandprovidefundingforthestaffmember

theyappointtojointheevaluationteam.Thinkthroughfundingimplicationsforallaspectsoftheprocess

andhowlongeachactivitywilltake.Forexample,ensurefundsforgoodqualityediting,formatting,and

presentation,asthesecanmakeasignificantdifferenceinhowwidelythereportisread.Berealistic

aboutthetimeitwilltaketheevaluationteamtogetthejobdone.Atleast30daysorevenfortydaysare

recommendedfortheteamleader.Theconsultantwilllikelybethelargestcost,butitisessentialto

budgetfor,ashis/hertaskswillinclude:

Reviewdocuments,preparemethodology,andcorrespondwiththesteeringcommitteepriorto

theevaluation.

Conductfieldvisitstoatleastthreesitesforeachoftheagencies.

Interviewagencystaff.

Interviewotherstakeholders.

Presentthefindingstostakeholdersincountry.

Prepareadraftofthereport. Incorporateeditsandcommentsonthereportfrommultipleactors.


12/21


CommunicatewhattheevaluationisaboutAjointevaluationisanewsworthyevent,butnoteveryonewillunderstanditspurpose.Makesurepeople

insideandoutsidetheparticipatingagencies,includingbeneficiaries,areawareoftheevaluationsothat

theywillbemorelikelytoreviewandmakeuseofthefindings.Draftaonepageinformationalsheet

abouttheevaluationforwidespreadsharing,andespeciallywithcountryofficestaffwhoneedtobe

awareof

the

evaluation

(though

they

would

ideally

be

engaged

throughout

the

process).

Itisparticularlyimportanttohavepreparatorydiscussionswithbeneficiarycommunitiestoensurethey

understandthepurposeoftheevaluationandtheyagreetoparticipateintheevaluation.Theyshould

understandthatevaluatorsdonothaveanyassistancetogive.4InHaiti,theevaluationteamtrained30

nationalstafffromtheparticipatingagencieswhospokeHaitianCreoletoberesponsibletoengage

beneficiaries.Theydidsobyconductingfocusgroupdiscussions,askingopenendedquestionsto

understandpeoplesexperiencewiththeemergencyresponse.5

Findacompetentadministrator/managerConsiderhiringsomeonewhocanspendasignificantamountoftime(50100%)focusedonthe

evaluation,especiallyinthemonthortwoleadinguptoit.Thispersonmaybeanadministrator,but

shouldalso

be

supervised

by

asenior

person

who

can

advise

on

strategic

issues.

Asuperbadministratorcanmakeamajordifferenceinthesuccessofanyevaluationbutparticularlya

jointevaluation.Ideally,anationalstaffpersonshouldbehiredorsecondedfromoneoftheagencies.He

orshewillberesponsibleformeetingthelogisticalandadministrativeneedsofboththesteering

committeeandtheevaluationteam.Asampletasklistforthispersoncouldlooklikethis:

Organizetherecruitmentfortheindependentconsultant(s).

Draftandprocesscontractwithconsultant(s).

Arrangeschedulesandmanagethecalendar.

Arrangelogisticalarrangementsfortravelinginthefield.

Coordinateinformationexchangebetweentheagenciesandtheevaluationteam,such

ascollectingtherelevantbackgrounddocumentsfortheevaluationteam.

Arrangemeetingsbothwiththeparticipatingagenciesandwithoutsideactors.

Helpdocumentwhoisresponsibleforwhatandsharethiswithallparties.

Agreeonnormsforperdiemandotherpolicies.Typically,eachagencyfollowstheir

ownandthecoordinatingonehirestheexternalsandusestheirperdiems.

Meetwiththeevaluationteam.

Thesteeringcommitteeorchairpersoncouldappointtheevaluationadministratorormanager.Ideally,

thesteeringcommitteewilldefinetheauthorityleveloftheadministrator,whoheorshewillreportto,

andwhatlevelofauthorityheorshewillhavetomakedecisions.Itshouldbemadecleartheamountof

timeanadministratorwillprovidetosupporttheevaluationteam.

CarefullypickevaluationteammembersSelecttherightteam.Inadditiontothetechnicalskillstheyneedtoconducttheevaluation,team

memberswillalsohavetobegoodatbalancingtheneedsofmultipleclientswithsensitivity.Whiletheir

rolesshouldbemadeclearbeforetheevaluation,experiencehasshownthattheywillneedtobeflexible

4SeeTool9intheGoodEnoughGuidetoImpactMeasurementandAccountabilityinEmergencies

5SeetheCARESavetheChildrenjointevaluationreportatwww.ecbproject.org


13/21


oncetheevaluationbegins.Theteamshouldnotbetoolargethatitisdifficulttomanage.Threetofour

teammembersisusuallysufficient. Atypicalteammaybecomposedof:

Anindependentconsultant/teamleader.Thispersonknowsalotaboutevaluationandalsohasstrongmanagementandleadershipskills,theabilitytostaycalmunderpressureandtobe

adaptive

in

the

face

of

the

unexpected.

A

joint

evaluation

team

leader

also

needs

the

ability

to

dealwithmultiplelayersofmanagementandbalancevariousexpectations,andthusmusthave

strongdiplomacyandcommunicationskillsbothwrittenandverbal.Theteamleadershould

alsohavehadpreviousexperienceasateamleader,sincethisisitselfaspecialskill.Team

leaderswiththeseskillsetsaresometimeshardtofind,anditiscriticalthatthereisabudgetline

topayforthem.

Thoughitsnotalwayspossibletorecruitateamleaderwhohaspreviouslyledjointevaluations,

confirmthatheorshehasexperienceinimpactanalysisinemergencies,asheorshewillneedto

understandhowthevarioussetsofdatacometogethertoformabiggerpicture.Notethat

consultantsoftencomewiththeirownideasandmethodologies,andtheywillneedguidance

andparametersfromthesteeringcommittee.

Anationalconsultant.Thenationalconsultantprovidescriticalguidanceonthepolitical,social,andculturalcontextoftheemergencytotheteam,especiallytotheteamleader,whoisoftenan

expatriate.Havingsuchapersononhandforajointevaluationcanhelpinnetworkingwith

nationalstakeholdersandensurethatknowledgeisquicklytransferredtotheevaluationteam

aboutkeyactorsandeventsandcanminimizesomeofthecomplexityofthedataandfactorsto

beanalyzed.

Asectorspecialist. Ajointevaluationwillchallengetheteamofevaluatorstoaddressthewiderangeofprogramareasbeingcoveredwhilealsofocusingonselectedkeyandpriorityaspects,

especiallyaseachagencymayhaveuniqueinterests.Ifagenciesneedmoreindepth

examinationofaparticulartypeofprogram,theyshouldconsiderbringingasectorspecialistto

theteam,freeingupothermemberstofocusontheoverallpicture.

Agencyteammembers.Eachagencytypicallyappointsonerepresentativetotheevaluationteam.Theseindividualsarenotactingonbehalfoftheiragencybutrathermustbeimpartial

evaluators.Theskillsetsofthesepeople,forexampletheirexpertiseincertainsectoralareas,

languageandfacilitationskills,andevaluationexperienceareveryimportanttotheoverall

successoftheteam.Ensuringthatagencyandevencountryofficestaffarerepresentedon

theevaluationteamwillincreaseownershipoftheevaluationfindings.

Itmaybehardforagencyteammemberstobeavailableforthefulllengthoftheevaluation,but

experiencehasshownthatcontinuityisimportanttoevaluationqualityandthelearning

experienceisalsogreatlyenhanced.Agencymanagersshouldthereforemakeeveryeffortto

ensurefullparticipationofagencystaffontheevaluationteam.

Giventhe

importance

of

getting

competent

team

members,

its

important

to

start

the

recruitment

processearly.Goodindependentconsultantsnationalandinternationalalikeareoftenbookedfor

weeksorevenmonthsinadvance.

OncethesteeringcommitteehasfinalizedtheTermsofReferenceforteammembersandtheskillsthey

want,agenciesshouldconsiderrequestinghelpfromtheirheadorregionalofficesinrecruitingtheteam,

suchasdoingtheinitialadvertisingandscreeningandthensendingashortlistofcandidatestothelead

agency.


14/21


Bythetimehiringbegins,theobjectivesfortheevaluationshouldhavebeenwellenoughdefinedthatthe

steeringcommitteeisclearonwhatprofilesareneededontheteamandthus,whotohire.

SeeTheToolsforSampleTermsofReferenceforEvaluationTeamMembers.

ChooseafewobjectivestocoverTheparticipatingagenciesmayhavedifferentinterestareastheydliketocoverinajointevaluation.But

itisnotpracticaltoaddresstoomanyobjectivesinajointevaluationasthereisalreadymorecontentto

cover.Ideally,thereshouldbenomorethanthreeorfourobjectiveswithinthetermsofreference,and

thescopeshouldbeasnarrowaspossible.Forexample:

Howwelldidthevariousagenciescoordinatetheirresponses?

Howappropriatewastheintervention?

Howtimelywastheintervention?

Howwelldidtheresponseassistpeopleinrecoveringfromthedisaster?

Objectives

that

concern

the

overall

impact

of

the

response

are

usually

best

for

a

joint

evaluation.

Objectivesofuniqueconcerntooneortwoparticipatingagencies,suchasissuesofoperationalefficiency,

arenotgenerallyappropriate.Inareaswheremoredepthisneeded,hireanadditionalteammemberto

focusspecificallyonaparticulartypeofprogrammingorissue.

Doaskforinputonthescopefromstaffatdifferentlevelsofeachagencywhomyouexpecttousethe

evaluationfindings.Atthesametime,itiswisenottoconsulttoowidely,asyouwillruntheriskofadding

toomanyobjectivesandanunrealisticscopefortheevaluation.

Objectivesshouldbeagreeduponbeforetheevaluationteamishired.Infact,considerbringinginan

externalfacilitatortonegotiatethescopeoftheevaluationaheadoftime.Oncetheleadevaluatorjoins,

heorsheshouldhavethechancetotelltheparticipatingagencieswhatisfeasibleandrealistic. Itis

criticaltofindabalancebetweenwhatagencieswantandwhattheleadevaluatorbelievesispossible.

SeeTheToolsforaTermsofReferenceTemplate.

AgreeonevaluationstandardsandmethodsJointevaluationsshouldincludeadocumentreview,keyinformantinterviewsandfocusgroupdiscussions

withstaffandbeneficiarygroups.

Theteamleaderwillbuildanapproachtoexamineeachagencysworkwithenoughrigortoinspire

confidenceinthefindings,butnotdetracttoomuchfromafocusontheoverallimpactoftheagencies

response.However,thesteeringcommitteeisexpectedtoadvisethisprocessandalsocommunicatethe

criteriatheywilluseforvillageandbeneficiaryselectionfortheinterviews.

Inadditiontomorelocationsforfieldvisits,forjointevaluations,theremayalsoneedtobemore

interviewswithotheractors,suchasUNagencies,representativesfromcivilsociety,nationalandlocal

partners,andgovernmentofficials.

Certainindicatorswillbenonnegotiabletobeinlinewithacceptedinternationalstandards,suchasthe

OECD/DACstandardsforevaluation.6 Spherestandardsareanotherkeypointofreferencewhichshould

6http://www.alnap.org/resources/guides/evaluation/ehadac.aspx


15/21


beassessedduringanevaluation,notonlyfortechnicalsectors,butalsocorestandardsofparticipation.

Onesetofstandardsshouldbeusedforconsistencyinmeasuringperformance. Beclearonwhat

organizationalminimumstandardsare.ReferencetheKeyElementsofAccountabilityontheECBProject

website.

WriteaninceptionreportTheevaluationteamshoulddevelopaninceptionreportonthetermsofreferenceandwithaworkplan.Thisreport,writtenbytheteamleader,willensureexpectationsareagreedonbythesteeringcommittee

andtheteamitself.Italsoallowsfortheevaluationteamleadertodialoguewiththesteeringcommittee

aboutwhatisrealisticandfeasible,givenavailabilityofstaff,budget,anddeadlines.

ManagecommunicationswithinthecollaborationAgenciesconductingajointevaluationneedclearagreementsaroundcommunication.Facetoface

meetingsarecriticaltomakesureunderstandingsareclearandtobuildcohesion.Decidinghowtostore

keydocumentsisalsoveryimportant.Onesolutionistosetupasimplewebpagetouploaddocuments,

contactlists,schedulesandotheressentialinformation.It

is

also

important

to

have

regular

opportunities

along

the

way

for

the

evaluation

team

to

discuss

any

concernswithsteeringcommitteemembers.Forexample,earlyintheprocess,theteamcangive

feedbackastohowwelltheevaluationmethodsareworkingandcheckwiththesteeringcommittee

whethertheseshouldbemodified.Ifthesteeringcommitteeisengaged,theevaluationwillbemuch

morelikelytosucceed.

Itisalsoimportanttoagreeinadvanceonprinciplesoftransparency withevaluationresults,including

communicatingresultsinatransparentwaytobeneficiarycommunities(whichcouldbeintheformofa

discussionorroundtable). Tryingtocoveruporhideevaluationresultsisnotonlyagainstprinciplesof

accountability,butunderminesorganizationallearningandcanoftenbackfire.

Prepare,prepare,prepare!Our

experience

has

shown

that

agood

amount

of

work

can

be

undertaken

even

before

the

evaluation

teamarrives.OncetheTermsofReferencefortheevaluationhasbeenestablished,alistofkeyinformant

interviewscanbedetermined,meetingsestablished,focalpointsready,andpreparatorydocumentscan

beemailedtotheevaluator.


16/21


CHAPTER4:WHATTODODURINGTHEEVALUATIONIfrealisticobjectives,managementstructure,andacompetentteamhavebeenchosenandestablished,theevaluationshouldbeeasiertomanage.Theteamwillstillneedgoodlogisticalsupport

andguidance.Hereissomeadditionalguidanceonconductingtheevaluation.

BrieftheteamuponarrivalEnsurethattheteamhasachancetodiscusstheTermsofReferencewitheachofthesteeringcommittee

members.TheleadevaluatorshouldalsogoovertheTermsofReferencewiththesteeringcommitteeas

agroup.

Theadministrator/managerorsteeringcommitteechaircanbriefteammembersonrolesand

responsibilitieswithintheevaluationstructure.Theteamwillneedtobebriefednotonlyonthelogistics

andtheprocessoftheevaluation,butalsoontheresponseprogramswhicharetobeevaluated.The

team

will

need

to

be

clear

on

how

the

evaluation

is

run,

the

role

of

the

lead

agency

and

the

other

agencies,towhomtheteamreports,wheretheywillgetlogisticalsupport,andhowtheywillmaintain

independence.Whenthesearenotclear,confusionabounds,andtheteamwillstruggletoachieve

objectives.Anticipatetheextraconsultationtimeneededwhenestimatinghowmuchtimetheteamwill

needfortheevaluation.

SharefindingsasyougoTheteamleadersshouldalsoensurethatthesteeringcommitteeandthestakeholders(asmentioned

below)receiveregularupdatesthroughouttheprocess.Ifthesteeringcommitteeandstakeholdersare

wellbriefedabouttheprogressandinitialfindingsoftheevaluation,therewillbenosurprisesattheend.

Dailydebriefsamongtheevaluationteamdrawsoutpreliminaryfindingswhichtheteamleadercanuse

toprovideregularupdates.

EnsurefindingsarereportedwithsensitivityReceivingandreviewingthefindingsofajointevaluationcanbeanexcitingtimefortheagenciesbutalso

atimeofapprehension.Theleadevaluatorshouldpresentfindingsinawaythatwillnotmakeanyagency

feelinferiororunfairlycomparedwithothers.Agencieswillalsoinevitablylookformentionsof

themselvesandjudgewhethertheythinkthefindingsarefair.Findingsthatarecriticalinnatureshould

bephrasedinaconstructiveway,supportedbyreasonableevidenceandbalancedwithpositivefeedback.

Inadditiontothemainreport,theevaluatorcouldalsocreateshortindividualreportsforeachofthe

agencies.Inpractice,however,thismaynotbeworththeadditionaleffortsincejointevaluationstendto

bebetteratlookingattheoverallresponseandcoordinationbetweenagencies(i.e.fromabeneficiary

perspective)thanlookingatindividualagencyoperationsindetail.

Anotherapproachisfortheevaluationteamtodoapreliminaryanalysisthatcompilesandgroups

findings.Throughaworkshopormeeting,theteamcanfacilitatestafffromtheparticipatingagencies(especiallythosewhowillusethefindings)tocollectivelydrawconclusionsandrecommendations.With

thistypeofparticipation,agenciesaremorelikelytoaccepttheconclusionsandfeelresponsiblefor

actingontherecommendations.

Ultimately,whenjointevaluationsarewellplannedandagenciesandtheteamcommunicatethroughout

theprocess,agenciesarelesslikelytotakeissuewiththeresults.Afocusonlearningmakeseventhe

leastflatteringfindingsmorepalatablebecausetheycanbeinstructive.


17/21


Oncetheagencieshavehadachancetodiscussanddebatethefindings,theyshoulddiscussthemwitha

broadergroupofstakeholders,especiallythosethatwereconsultedduringtheevaluationprocess,such

asUNagencies,beneficiaries,localNGOs,andlocalgovernment.Onewaytodothisistoholdaninter

agencyvalidationworkshopwherestakeholdersaregivenanopportunitytoconfirmordisputethemajor

findings

and

recommendations.

FinalizingtheEvaluationReportTheevaluationreportshouldbeeasytoreadandrelativelyshortnomorethan30pages.Itisimportant

tofocusonthatwhichhasgonewell,andgoodpracticeshouldbehighlightedinthereport.

Assumingstakeholdershavebeenbriefedthroughouttheevaluation,thefindingsandrecommendations

intheevaluationshouldnotcomeasasurprise.Doanticipate,however,thatstakeholderswillnotagree

withallfindingsandthesteeringcommitteeshouldbepreparedtoaddressthis.

Itiscriticaltosetoutaperiodtoreceivefeedbackonthedraftofthereport.Beclearandrealisticabout

thetimelineforthisperiod.Itneedstobelongenoughtoallowtherightpeopletoprovidefeedback,but

notsolongthatthefindingsarenolongerrelevantbythetimethereportiscompleted.Afterensuring

thatallofthepeoplewhoneedtogivefeedbackareinformedofthescheduleinadvance,twotofour

weeksisareasonabletimeinwhichtoallowpeopletosubmitfeedback.

Theevaluationteamleaderisultimatelyresponsibletomakethedecisionsaboutwhichfeedbackis

incorporatedandwhichisnot.Ifthereisenoughdisagreementaboutcertainfindingsorconclusions,

thesecanbeaddressedinamanagementresponsethatisannexedtothefinalreport.


18/21


CHAPTER5:WHATTODOAFTERTHEEVALUATIONWiththeevaluationworkdoneandthefindingsdetailed,onepartoftheprocesscomestoaclose.But

inotherways,therealworkisjustbeginning.Hereissomeguidanceformakingthemostofthe

completedevaluation.

DevelopbothcollectiveandindividualrolloutplansBecausejointevaluationshaverelevancetoawiderangeofactors,agenciesshouldsharethereportwith

humanitarianbodiesandsuchnetworksastheActiveLearningNetworkforAccountabilityand

PerformanceinHumanitarianAction(ALNAP),inadditiontoheadquarters.Sharingthereportfromajoint

evaluationreportwidelydemonstratestransparencyandacommitmenttocontributetolearningwithin

thebroaderhumanitariansector.

Theagenciesmaywanttodevelopsimplecollectiveandindividualcommunications plansincluding

distributionlistsforthereportandsmallactionplanningmeetingstodiscussandpresenttheimplications

ofthefindings.

EmphasizepeeraccountabilityWithjointevaluations,agencieshavetheopportunitytoholdoneanotheraccountableforprogresson

recommendations.Theymaychoosetoworkonsomerecommendationstogether.Theymayagree

beforehandtoholdafollowupworkshopinsixmonthsorayearstime.Atsuchatime,theycouldthen

discusshowthefindingswereshared,whatprogresswasmadeandwhatwastheoutcomeofanyactions

taken.


19/21

CHAPTER6:JOINTEVALUATIONSINREALTIMEConductedwhiletheemergencyresponseisstillongoing,realtimeevaluations(RTEs)arevaluable

toolsforrectifyingproblemsandmakingimprovementsinprogrammes.However,ajointRTEcanbe

especiallychallenging.

Here

are

some

things

to

consider

iffacing

the

decision

of

conducting

ajoint

evaluationinrealtime.

PreparefortheevaluationbeforetheemergencystartsIncasesofslowonsetemergencies,theremaybetimetoplanmonthsinadvance.Evenwithrapidonset

emergencies,preparednessispossible.Youcanjointlyoutlinegenericplansfortheevaluationwhichcan

beturnedintoactualplansinthefaceofadisaster.Theseplansshouldincludeasmanyoftheaspectsas

possiblethatareoutlinedinthisbookletonhowtoorganiseajointevaluation,including,crucially,the

designationoffocalpoints.Thesefocalpointsareonstandbyandwillhavetheresponsibilityofgettingan

RTEprocessstarted,conveningthevariousactors,etc.andwhowillideallyremainaspointpersons

duringtheprocess.

TakeagoodenoughapproachtotheevaluationYoumayhavetotakesomeshortcutsanduseagoodenoughapproach.Goodenoughdoesnotmean

secondbest:itmeansinanemergencyresponse,adoptingquickandsimplesolutionsmaybetheonly

practicalpossibility.Whenthesituationchanges,youshouldaimtoreviewyourchosensolutionand

amendyourapproachaccordingly.

Forexample,youcansimplifymanagerialstructures.Withindaysofthepartiesagreeingtodoajoint

evaluation,youmayagreetoestablishasmall,rapidlyorganizedmanagerialstructure(whichcan

subsequentlytransitiontoamorerobustoneatalaterstage).Duringthefirstweek,forinstance,that

groupcouldlookatwhatisminimallynecessary,andcreateapracticalquickanddirtytermsof

reference.Themanagementcommitteecoulddelegatemuchofthedaytodaymanagementtooneor

twokeyactors,andthusspendlesstimeongroupdecisionmakingandconsensusbuilding.

Eachparticipatingagencywouldbetrustedtocarryoutthetasksassignedtotheminaccordancewith

predeterminedplansandstandards.Oncetheprocesshasbeenstartedandtheevaluationisinmotion,

agenciescanthengraduallybuildintighterqualitycontrolmechanisms,morefocusedtermsofreference,

andamoreinclusiveprocess(e.g.alargermanagementgroup).

Suchagoodenoughapproachisnotanidealevaluativeprocess,butlikelyrelevantforRTEsbecause

agenciesareparticularlybusywiththeimplementationofaresponse.However,certainaspectsofthe

jointevaluationshouldnotbesubjectedtoshortcuts.Theseincludeethicalstandards,suchasthe

confidentialityandindependenceoftheevaluativeprocess.

CallonadditionalresourcesParticipating

agencies

could

consider

calling

on

additional

internal

support.

A

staff

member

could

be

secondedtoacountryofficeforsomeweekstohelpwiththejointRTE.Unlikecountryofficestaff,who

wouldpresumablybepreoccupiedwiththeemergencyresponse,thispersonwouldhavetimetofocuson

theevaluation.Heorshecoulddoaninitialscoping,astakeholderanalysisandholdameetingcollectively

orindividuallywithpartnerstogettheirviews.Heorshecouldalsoassistwithpracticalpreparationsfor

theteam,includingsettingupfieldvisits.


20/21


ConsidersomeotherjointreflectionprocessIfajointRTEisnotrealistic,considerotherlearningprocesseslikeajointafteractionrevieworapeer

review.Agenciescandoquickassessmentsoftheirwork(seeImpactMeasurementandAccountabilityin

Emergencies:TheGoodEnoughGuide)andgettogetherforashortmeeting/workshop,orinvitean

experiencedexperttoprovideadviceonhowtheoperationmaybeadapted.


21/21

ReferencesandFurtherReadingJointEvaluationsGuidanceforManagingJointEvaluations.DACEvaluationSeries,OECD2006.

http://www.oecd.org/dataoecd/29/28/37512030.pdf

JointEvaluations:RecentExperiences,LessonsLearned,andOptionsfortheFuture.DACEvaluation

NetworkWorkingPaper,OECD,2005.

LessonsAboutMultiAgencyEvaluations:AsianTsunamiEvaluationCoalition.http://www.tsunami

evaluation.org/NR/rdonlyres/9DBB5423E2EF43ABB6D2

2F5237342949/0/tec_lessonslearned_ver2_march06_final.pdf

GeneralEvaluationsUSAIDCenterforDevelopmentInformationandEvaluation,PerformanceMonitoringandEvaluationTIPS

series.

http://evalweb.usaid.gov/resources/tipsseries.cfm

WesternMichiganUniversity,EvaluationCenter.EvaluationChecklists

http://www.wmich.edu/evalctr/checklists/checklistmenu.htm#models

ShoestringEvaluation:DesigningImpactEvaluationsUnderBudget,Time,andDataConstraints. M.

Bamberger,

J.

Rugh,

M.

Church,

and

L.

Fort,

The

American

Journal

of

Evaluation,

2004.

UtilizationFocusedEvaluationChecklist.MichaelQuinnPatton

http://www.wmich.edu/evalctr/checklists/ufe.pdf

Documents

Guidelines for Conducting Joint Evaluations of Humanitarian Action