52
. . . . . . . . . . World Food Programme A Report from the Office of Evaluation Full Report of the Evaluation of the AZERBAIJAN PRRO 6121.00 - Relief and Recovery Assistance for Vulnerable Groups (25 October – 09 November 2001) Rome, April 2002 Ref. OEDE/2002/06

A Report from the Office of Evaluation - documents… · Annex 5: Documents reviewed and documents consulted Maps ... Though the trigger for this episode of conflict was the disintegration

  • Upload
    dinhdan

  • View
    222

  • Download
    0

Embed Size (px)

Citation preview

. . . . . . . . . .

World Food Programme

A Report from theOffice of Evaluation

Full Report of the Evaluation of theAZERBAIJAN PRRO 6121.00 -Relief and Recovery Assistance forVulnerable Groups

(25 October – 09 November 2001)

Rome, April 2002

Ref. OEDE/2002/06

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

i

Acknowledgement

The evaluation team visited Azerbaijan from 25 October to 09 November 2001.This document was prepared by the mission team leader on the basis of themission’s work in the field.

On behalf of the team, the author wishes to extend thanks to all those whofacilitated the team’s work in the field and in Headquarters.

Responsibility for the opinions expressed in this report rests solely with theauthors. Publication of this document does not imply endorsement by WFP of theopinions expressed.

Mission Composition

• Mr. John Kirkby, Mission Leader, WFP/OEDE Consultant• Ms. Rena Ibragimbekova, Gender Expert, Local Consultant, WFP/Azerbaijan• Mr. Paolo Mattei, Evaluation Officer, WFP/OEDE

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

ii

AcronymsADRA Adventist Development and Relief Agency InternationalANAMA Azerbaijan National Agency for Mine ActionARRA Azerbaijan Reconstruction and Rehabilitation AgencyCCS Common Country StrategyCLAU Caucasus Logistics Advisory UnitCO Country Office of WFP AzerbaijanCW WFP Commitments to WomenDAC Development Assistance CommitteeDP Distribution PointEMOP Emergency OperationEU European UnionFAO Food and Agriculture OrganizationFFT Food-for-trainingFFW Food-for-workFSU Former Soviet UnionGoA Government of AzerbaijanGoUSA Government of the United States of AmericaHQ Headquarters WFPICRC International Committee of the Red CrossIDD Iodine Deficiency DisordersIDP Internally Displaced PersonIMF International Monetary FundIP Implementing partnerLoA Letter of AgreementM and E Monitoring and EvaluationMDM Médecins du MondeMICS Multiple Indicators Cluster SurveyNFI Non-food itemsNGO Non governmental organizationNK Nagorno KarabakhOECD Organization for Economic Cooperation and DevelopmentOSCE Organization for Security and Cooperation in EuropePRRLA Programme for the Resettlement and Liberation of the Liberated AreasPRRO Protracted Relief and Recovery OperationRSG Representative of the Secretary General on IDPsSCF Save the Children FundSCRIDP State Committee for Refugees and Internally Displaced PeopleSFDI Social Fund for Development for Internally Displaced PeopleSPFS Special Programme for Food SecurityUN United Nations OrganizationUNCT United Nations Country TeamUNDMT United Nations Disaster Management TeamUNICEF United Nations Children’s FundUSAID United States Agency for International DevelopmentVAM Vulnerability Analysis and MappingWFP United Nations World Food ProgrammeWVI World Vision International

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

iii

Table of Contents

Executive summary .......................................................................................................v

1. Introduction to the evaluation.............................................................................11.1 Evaluation Objectives ..................................................................................11.2 Method........................................................................................................1

2. The Context of the Protracted Crisis and WFP’s PRRO...................................12.1 Background on the evolution of the protracted situation ...............................12.2 Food insecurity and poverty among those affected .......................................22.3 National policies and priorities for relief, recovery and food security ...........32.4 Role of food aid and rationale for WFP assistance........................................3

3. Quality of the Recovery Strategy and PRRO Design .........................................43.1 Assessment of the recovery strategy.............................................................43.2 Assessment of the PRRO design ..................................................................5

4. PRRO implementation ........................................................................................74.1 Efficiency Issues..........................................................................................74.2 Assessment and targeting.............................................................................84.3 Logistics and leakages .................................................................................94.4 Security .....................................................................................................104.5 Monitoring and evaluation system..............................................................104.6 Management ..............................................................................................11

5. Results of the PRRO..........................................................................................125.1 Protracted relief .........................................................................................125.2 Returnees and Resettlement ......................................................................135.3 Recovery (FFW and FFT)..........................................................................14

6. Partnership and Co-ordination.........................................................................16

7. Resourcing Issues of the PRRO.........................................................................16

8. Meeting WFP’s Commitments to Women ........................................................17

9. Environmental Concerns...................................................................................19

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

iv

Annexes

Annex 1: Terms of Reference of the EvaluationAnnex 2: Evaluation methodsAnnex 3: Checklist for Meeting Commitments to Women and Mainstreaming

a Gender PerspectiveAnnex 4: Mission itinerary and persons interviewedAnnex 5: Documents reviewed and documents consulted

Maps

Map 1: WFP Areas of Operation in AzerbaijanMap 2: WFP Activities in AzerbaijanMap 3: WFP/UNICEF Pre-School Centres

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

v

Executive Summary

WFP, which had been implementing EMOPs in Azerbaijan since 1994, began a three-yearPRRO 6121 in July 1999, following advice from a Joint WFP/Donor mission. The plannedPRRO objectives were: to provide protracted relief for 405,000 beneficiaries, 390,000 IDPsand 15,000 vulnerable people; to aid 50,000 returnees to resettle; and to help the recovery oflivelihoods in agriculture, agro-forestry and the improvement of social and economicinfrastructures through food-for-work (FFW) and food-for-training (FFT) for 30,000 IDPs.WFP provides a ration equivalent to 50 per cent of daily needs as an addition to that whichbeneficiaries are able to secure from other sources. The total cost to WFP is $US 23,152,786.

Between 1988 and 1994 up to 600 000 Azerbaijani people were internally displaced byviolent conflict with Armenia over the enclave of Nagorno Karabakh (NK). Most are stillunable to resettle in their home areas in NK and surrounding districts. While they await thehoped for, but long delayed peace settlement, IDPs live in tents, flimsy prefabricatedshelters, railway wagons, dugouts, unfinished buildings and other unsuitable shelters. Thepoor condition of the crowded shelters contributes to ill health among IDPs. Because thecountry’s economy is still suffering the disruption caused by the economic stresses andpolitical changes associated with the collapse of the former Soviet Union, and because theadditional income from oil has been slow to develop, the Government of Azerbaijan (GoA)is unable to fully support the food needs of the IDPs. These IDPs find it difficult to providefor their own needs because they have little access to land or the means of earning sufficientincome. Consequently, IDPs suffer from malnutrition and the other health problems such asgoitre and malaria, as revealed by recent nutritional and health surveys.

With World Vision International (WVI) as the principal implementing partner, the protractedrelief programme has succeeded in maintaining the nutritional status of IDPs with a bi-monthly distribution of rations through a network of 140 distribution points. The Missionrecommend a programme of regular systematic nutritional surveys, that WFP consider thepossibility of school feeding and the extension of FFW and FFT support for improvements inliving conditions and trauma counseling. WFP has carried out the relief activities efficientlyand effectively, with efficient logistics, management, supervision and monitoring andevaluation (M and E). WFP Country Office (CO) faced considerable difficulties inimplementing the planned reduction to 70 000 beneficiaries by the third year of the PRRO.The Mission recommends that VAM methodology be used to refine the targeting ofbeneficiaries so that those who are most food-insecure and the most vulnerable areprioritized.

WFP supported a joint GoA/EU/UN pilot resettlement project at Horadiz village for 2000IDPs, but, for a number of reasons discussed in the report, the planned large-scaleresettlement has not been possible, nor has the planned FFW farming activity, becausesuitable land has not been made available. Very few IDPs were interested in the extension ofincome generating activities through micro-finance. A successful joint WFP and UNICEFprogramme of pre-school trauma reduction programme for more than 2 000 children hasbeen supported by FFW and FFT. WFP could use these mechanisms to support the extensionof psycho-social interventions to other needy sections of the IDP community such asadolescents, adults and returnees. The Mission recommend that WFP support FFW and FFT

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

vi

activities to improve the deplorable living conditions of the IDPs for example in watersupply, sanitation, drainage, shelter maintenance and repair, nutritional advice and psycho-social counseling.

The PRRO has been relatively well supported and successful as a mechanism for funding.There have been few problems with the pipeline. In future there will be a need for additionalCO staff, whether the outcome is towards peace and resettlement or towards conflict andfurther relief activities. WFP performance in relation to Commitments to Women has beengood and the presence of women in positions of authority in the field was notable andcommendable. Environmental degradation was not a serious issue but the Missionrecommends that WFP seek to use the FFW/FFT mechanisms to improve the livingenvironment of the IDPs.

Beneficiary needs-based targeting can be further improved by using vulnerability analysisand mapping (VAM) methodology to help identify the most food insecure and vulnerablegroups. A VAM mission from Headquarters could assist the country office in targeting thenew PRRO.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

1. INTRODUCTION TO THE EVALUATION

A three-person Mission (consultant on humanitarian assistance, member of WFP OEDE and nationalexpert on gender) visited Azerbaijan from 25th October to 9th November 2001, to evaluate PRRO 6121.00AZE. During the final two days, the Mission met the Formulation Mission for the next phases of thePRRO and discussed the Mission findings and recommendations.

1.1 Evaluation Objectives

The evaluation has three objectives. The first objective is to assess WFP assistance in PRRO 6121.00against the standard evaluation criteria. These standard criteria are effectiveness, efficiency, impact,sustainability and relevance. In addition, in 1999, OECD/DAC recommended several other criteria inGuidance for Evaluating Humanitarian Assistance in Complex Emergencies. These criteria: timeliness,appropriateness, connectedness, coverage, coherence and coordination are considered where appropriate.The second objective is to assess the value of the PRRO category in resourcing and programming. Thethird objective is to provide accountability to the Executive Board.

1.2 Method

The Mission had discussions with the State Committee for Refugees and Internally Displaced People(SCRIDP); UN partners; USAID; Mercy Corps; and World Vision (WVI). Five days were spent visitingWFP and WVI implementation sites in the field. During these visits the Mission witnessed distributions;visited warehouses and saw the living conditions of beneficiaries; observed implementation activities;and interviewed beneficiaries, representatives of local government, WFP and WVI field staff. Details ofthe methodology are presented in Annex 2 and people interviewed are listed in Annex 4.

2. THE CONTEXT OF THE PROTRACTED CRISIS AND WFP’S PRRO

2.1 Background on the evolution of the protracted situation

WFP started EMOP 5302 in Azerbaijan during 1993 in response to violent conflict and the internaldisplacement of Azerbaijanis. In 1989 Azerbaijan became involved in a military conflict with Armeniaover the enclave of Nagorno Karabakh (NK). (Map 1 shows the location of NK and the occupiedterritories.) Though the trigger for this episode of conflict was the disintegration of the FSU, the roots ofthe conflict lie some 200 years back. Azeris started to be displaced from Khan Kandy in NK as early asFebruary 1988. Some 200,000 Azeris formerly living in Armenia became refugees in Azerbaijan in theearly stage of that conflict. Many Azeri people were displaced between early 1992 and mid 1993. Violentconflict continued until a cease-fire in 1994, by which time some 600,000 Azeris were internallydisplaced throughout the country. With the exception of sporadic skirmishes and the occasional firing ofshells the cease-fire has held since 1994. In addition to Azeri refugees and IDPs, Azerbaijan hosts otherrefugees including 50,000 Meshketi Turks, Chechnyans and other groups from surrounding countries.These refugees have been integrated into the population to different extents. At present, the Governmentof Azerbaijan (GoA) estimates that more than one tenth of the total population of about eight millions isa refugee or an IDP. Despite continuing negotiations through the Organization for Security and Co-operation in Europe (OSCE), no resolution has been achieved for the status of NK so that it has not beenpossible for IDPs to return to their home areas in NK or for the larger number to return to the sevensurrounding occupied districts. About 20 per cent of the 1988 land area of Azerbaijan which includessome of the best quality agricultural land and which previously produced some 40 per cent ofAzerbaijan’s agricultural production, is currently occupied and therefore inaccessible for IDPs.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

2

The economy of Azerbaijan has not yet recovered from the collapse of the former Soviet Union (FSU)and FAO now identifies Azerbaijan as a Low Income Food Deficit Country. Agricultural production hasbeen limited by:

• three years of drought 1998-2000;• the disruption of production caused by privatization of land;• the under-capitalization of the privatized farms;• the small size of privatized farms, many of which are between 0.1 and 0.5 hectares in extent;• the collapse of local and export markets for agricultural products;• deterioration of the badly-designed irrigation system, only two thirds of which is currently operating;• progressive salinization of irrigated land; and• the importation of cheap grain from Russia following the devaluation of the ruble in 1998. These

cheaper imports have reduced the competitiveness of Azerbaijani agriculture so that farmerscontinue to be unable to invest in agricultural inputs and little farm employment is available forIDPs.

Other areas of the economy continue to stagnate. Due to the regional specialization of production in theFSU, there have been insufficient local markets for the former industrial products which had previouslybeen produced on a very large scale and with secure markets in the command economy of the FSU. Inconsequence, many factories have closed down. As a legacy of centralized decision making in the FSU,entrepreneurial skills are lacking. All of these factors reduce the possibility of IDPs being able to findemployment. Oil and gas revenues, worth only $52 millions in 2000, have increased more slowly thananticipated. In response to the economic decline caused by these problems, the International MonetaryFund (IMF) and World Bank have introduced a programme of structural adjustment, but this has led toreductions in welfare spending. Thus, due to a range of continuing political and economic problems theGoA has not been unable to maintain the former generous welfare payments. This has been a problemparticularly for IDPs who have few other opportunities to secure an income because of the decline andstagnation of the economy. Many of the IDPs served by WFP are located in remote rural areas atconsiderable distances from any possible employment.

2.2 Food insecurity and poverty among those affected

Many IDPs arrived without any material assets and others have already sold or are now forced to sellassets as part of their survival strategies. As mentioned in section 3.1, the poor state of the economy hasmade it difficult for them to find paid work. Though many of the IDPs had previously been farmers, theyhave been unable to access land; others who had previously worked in industry or services have beenunable to find comparable employment in which they could use their existing skills. Thus IDPs have hadlittle opportunity to create income to buy food and other household needs, contributing to malnutritionand ill health. By 1998, global malnutrition1 of IDP children from six months to five years had risen to30 per cent and among adults aged over 60 to 12 per cent. Investigations in 1998-9 by Médecins duMonde (MDM) Greece discovered the high incidence of goitre, a diet-related disease, caused by iodinedeficiency. Iodine deficiency disorders (IDD) are also known to contribute to miscarriages and tostunting among children.

The UNDP Azerbaijan Human Development Report of 1998 reported that IDPs spent 80 per cent ofgeneral household budgets on food and that bread completely dominated the diet, while they consumedprogressively smaller amounts of dairy products and meat. World Vision (WVI) nutritional surveys ofIDPs in 1997 and 1998 reported that 80 per cent of families skip meals as a response to shortage of food

1 Global malnutrition comprises severe and moderate malnutrition and the prevalence is expressed as thepercentage of the specified population (usually under fives) with weight for height under -2 Z scores and / or withoedema. A rate between 5 and 10 percent gives cause for concern: rates were much higher among Azerbaijani IDPsat the start of the PRRO.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

3

and that 80 per cent had no access to home gardens, reducing the possibility of their producing food forthemselves. Of non-pregnant women 80 per cent suffer some level of anaemia. The mission spoke to anumber of IDPs who said that they had reduced their meals from three to two per day, a classic foodshortage coping mechanism.

At the start of the PRRO, and after more than five years of displacement many IDPs still live inunfinished buildings, former factories, disused military barracks, barns, stables and other unsuitablesituations. Though many IDPs live in what might be described as camps, considerable numbers also livewithin local settlements. The mission visited a rail wagon in which a total of eight people lived in twoseparate households. Some 55 per cent of families lived in one-room shelters of one sort or another. Twoof the dugouts visited had seven and eight people living in one room. The WFP October 2001 HouseholdFood Economy Survey shows an increase in family size so that overcrowding and the threat to health hasincreased since 1998.

2.3 National policies and priorities for relief, recovery and food security

Within the GoA, a number of agencies and departments support the needs of IDPs: AzerbaijanReconstruction and Rehabilitation Agency (ARRA); SCRIDP; and the Social Fund for Development forInternally Displaced People (SFDI). Though the existence of these units indicates the intention of GoA torespond actively to the needs of IDPs, the economic status of the country severely limits these units’performance. The WFP Country Case Study on Internal Displacement reports that many of the arms ofgovernment that were intended to respond to the problems of IDPs, lack both capacity and resources.Since 1992, when large numbers of people started to be displaced, the GoA has recognized additionalentitlements for IDPs, and thus provides subsidies for bread, for children under 16 in poor families, warpensions for loss of a family member (martyrs), and invalidity pensions.

The GoA wishes to control the numbers of IDPs in the main urban centres, particularly Baku andSumgait, where some IDPs were said to be involved in crime (including drug dealing) and prostitution.Some IDP children are said to have become street children in the two cities. With the exception of asmall number of specified vulnerable groups in hospitals and orphanages, all WFP food aid to the easternareas was stopped in 1998 (Map 1).

During 1999 several ministries of the GoA met UN agencies and NGOs to discuss complaints fromNGOs that their staff had suffered harassment by police and customs, with demands for illicit payments.This conference, in May cleared the situation for the start of the PRRO, but WFP led the request for afurther conference in May 2000 to ensure more acceptable behavior by police and customs.

In principle, the GoA intends to allocate 47,000 hectares of agricultural land to IDPs to enable them toachieve greater food security through self-provision while awaiting the opportunity for resettlement. Fora number of reasons, discussed below, little land has been allocated. The GoA’s highest priority is therecovery of the lost territories and resettlement of the IDPs. In conjunction with UN agencies and the EU,the GoA has aided the resettlement of IDPs in liberated land in Fizuli, Agdam and Terter. WFP suppliesa three-month full ration or six-month half ration to aid resettlement. The GoA considers that WFP foodaid is critically important for maintaining the nutritional status of IDPs and for the maintenance of publicorder.

2.4 Role of food aid and rationale for WFP assistance

WFP has responded to the food insecurity of IDPs in Azerbaijan since 1994 through EMOP 5302, whichthey replaced with the current three-year PRRO 6121 in July 1999. The GoA is not yet fully capable ofcoping with the economic and social problems, poverty and food insecurity mentioned in section 3.2.Therefore, in order to maintain the nutritional status of the IDPs, WFP provides rations of approximately50 per cent of daily needs to supplement food that IDPs are able to access through other methods. A 1998WFP/WVI food economy survey, just before the start of the PRRO, reported that the equivalent of 24 per

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

4

cent of IDP household income was provided by the WFP ration. This finding supports the GoA view ofthe great significance of WFP food. WFP also provides rations for six months to facilitate resettlementand as (FFW) and (FFT) incentives to encourage self-reliance. The rationale for WFP assistance is:

a) the inability of GoA to provide sufficient food and welfare support for IDPs and some othervulnerable people;

b) the poor integration of IDPs into the local economy, with consequent inability to self-supply andthus satisfy the range of human needs and in particular to provide an adequate quantity and qualityof food; and

c) the continuing poor state of health and nutrition of the IDP population and specific groups ofvulnerable people.

WFP intended that an increased level of self-sufficiency in food would allow self-provision of both foodand non-food needs, thus using food to secure nutritional objectives but also, more broadly, to allowIDPs to have greater control of their own lives. This is a significant element of recovery, a return to morenormal conditions of self-reliance, and a step forwards from relief.

3. QUALITY OF THE RECOVERY STRATEGY AND PRRO DESIGN

3.1 Assessment of the Recovery Strategy

Starting the work in November 1998, the WFP Country Office (CO) prepared the PRRO document andrecovery strategy, with documentary advice (such as Guidelines for the Preparation of a PRRO,Azerbaijan Country Case Study on Internal Displacement and From Crisis to Recover) but with almostno other expert advice from the Regional Bureau or HQ. The only advice given was in the modificationsproposed by HQ after their review of the document and by the Joint WFP/Donor Mission of January1999. WFP Azerbaijan CO was fortunate in having an experienced programme analyst already in postand able to prepare the PRRO, though without specialist training for this activity. According to thecriteria of Guidelines for the Preparation of a PRRO, the IDP situation in Azerbaijan is suitable for aPRRO, with its overall objective of “the re-establishment of livelihoods and household food security”(Introduction p.3) through protracted relief activities, in protracted refugee (IDP/returnee) circumstancesand with transitional recovery activities. The main reasons for WFP’s continuing to provide relief foodfor IDPs are the lack of opportunities for their full self-provisioning and the present inability of the GoAto supply sufficient resources. But the following factors suggest that it is possible and advisable to seekto move onwards from relief towards recovery and more developmental activities:

• for some years, conflict has been restricted to small areas and is at a low level. It is not a threat toIDPs in the areas where they are currently living;

• for some years, there has been little change in resolving the status of NK;• for eight years there have been virtually no new IDPs;• IDPs left their homes in the social and economic circumstances of the FSU. For a successful return

they need to be prepared for life in a non-centralized political economy;• children from the age of about 12 and adults had experienced the trauma of war: violence,

bereavement, deprivation, loss, disruption of their lives and other stresses. Continuing to live in thepresent state of displacement will not help their recovery and may well lead to deterioration;

• younger children are living their formative years in a restricted and limiting environment and in thecontext of some level of trauma among the rest of the population. These restrictions must beovercome through developmental activities if the culture of deprivation is not to be reproduced andreinforced. In particular the traditional high educational standards, which underpinned an relativelyadvanced economy, must be recovered: a continued emphasis on relief alone, does not favor suchrecovery; and

• relief activities may at best maintain, but do not create human assets. Recovery activities are neededto create human assets.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

5

A Joint WFP /Donor Mission (10-15 January 1999), including WFP, FAO and USAID personnel, maderecommendations for activities and this was a key input to the CO’s preparation of the document. TheJoint Mission was able to reflect on the success of previous WFP activities, identify unchangingproblems and limitations on WFP action, identify scope for action, and set priorities. But in itself theJoint Mission Report is not a strategic plan for recovery. The main recommendations of the Joint Missionwere:

• to continue the WFP programme, focusing on vulnerable IDPs;• to continue distribution of food relief but extending FFW to create community benefits;• to develop an exit strategy with hand-over to GoA, particularly of social welfare activities;• to seek suitable partners for technical components of income-generating activities and clarify at the

design stage the intended beneficiaries;• with partners to support resettlement of IDPs in home areas;• that WFP should participate in an integrated international strategy; and• annual review of the exit strategy.

The present report shows that WFP followed these recommendations in the development of its PRROstrategy. In several aspects, however, the strategy is also a continuation of EMOP 5302.04, for examplein maintaining the emphasis on relief activities for IDPs, and for other particularly needy groups such asthe Sumgait orphans, tuberculosis children in Baku and Chernobyl victims; and continuing to support theexisting successful programme for children traumatised as a result of their difficult living circumstances.All of these groups are manifestly still in need of support. In seeking to increase participation in FFWand FFT, the strategy also maintains some of the aims of the preceding EMOP.

The document follows very closely the format prescribed in Guidelines for the Preparation of a PRRO.Operational modifications to the programme reflect the largely static political and economic situations.As on previous occasions, the hoped for political settlement of the NK problem has not occurred and theeconomic benefits for GoA from the oil industry have been very slow to arrive. The project documentmakes movement away from relief to more developmental activities explicitly dependent upon thesepolitical and economic improvements. WFP CO continuously reviews the Contingency Plan, whichreflects on political events, broad security issues and economic circumstances. During the Mission’svisit, following statements by the President of Azerbaijan, the CO considered that the risk of outbreak ofhostilities had increased and modified the WFP Contingency Plan accordingly.

The recovery strategy includes a rationale, substantially as outlined above in section 3.4, for thecontinuing provision of a half ration to vulnerable people and a justification for the use of food aid. Targetareas are not explicitly identified in the project document, though the area coverage is clear in practice,being a continuation of those districts previously covered in EMOP 5302.04. In this the strategyrecognizes the relatively unchanged humanitarian situation in those districts. The PRRO documentexplains assistance modalities. The risk assessment identifies increased military activity as the main threatand, as does the WFP Contingency Plan, and stresses that a transition to recovery activities is dependenton both military security and economic stability. In the exit strategy WFP follows the Joint Mission instressing that phase-out will be possible only if the GoA is better able than in 1999 to cater for socialcases, and discusses the consequent need for the possible extension of relief activities. In this the PRROproject document follows Recommendation 3 of the Joint WFP/Donor Mission. Continuous review of theWFP Contingency Plan provides, in effect, the annual review of the exit strategy recommended by theJoint Mission.

3.2 Assessment of the PRRO Design

In designing the PRRO, the CO was influenced by the recommendations of the Joint WFP/DonorMission reviewed above, that: rations continue to be provided for vulnerable IDPs; that Commodities beused to support FFW and FFT for recovery and resettlement and that an exit strategy be prepared with

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

6

handover to GoA, but other objectives were included. Four main PRRO objectives are listed in thePRRO document:

a) the improvement of food security among IDPs and vulnerable groups;b) the encouragement of voluntary resettlement of returnees in their places of origin;c) the encouragement of the integration of IDPs into communities;d) the increase of opportunities for beneficiaries to become self-reliant.

The main emphasis of the PRRO allocation, with 83 per cent of food resources, is on the first objective,through the distribution of food relief to maintain the nutritional status of the IDPs. In the circumstancesprevailing at the time of the formulation of the strategy this was an appropriate aim and an appropriateweighting. In the project document (paragraph 33) the intention had been to distribute a full ration tobeneficiaries: because it has not been possible to reduce the number of beneficiaries, beneficiaries areprovided with a half ration, a sensible response in the circumstances. If it were possible to achieve them,the other three aims would accomplish a significant movement towards recovery. Little food(approximately 2.5 per cent) was allocated in the PRRO document as support for resettlement, thoughthis is the ultimate aim of GoA and of the UN system generally. It is also the stated aim of the IDPs towhom the Mission spoke. Though this small allocation might seem to indicate lack of resolution on thepart of WFP, in fact it shows an accurate perception of the real difficulties of securing resettlement.Allocation of no more than 14 per cent of food resources to support for FFW and FFT may also appear tohave been over-cautious, but the unavailability of sufficient land for agricultural activities was alreadyclear in 1999, so the relatively modest target was appropriate. Had it been possible to increase theenvisaged level of participation in FFW and FFT, the CO could have invoked the provision for budgetrevisions of up to 10 per cent (paragraph 57 of the PRRO document).

The PRRO document discusses monitoring and evaluation, though many of the items to be monitored arenumerical, tangible and delivery-related rather than in themselves outcome-related and thus indicative ofsuccess in achieving the objectives. Design of the PRRO is broadly in line with From Crisis to Recoveryand incorporates a range of activities some of which the latter document classifies as appropriate to early,and some to late, recovery phases.

In the PRRO document the intended reduction in the number of beneficiaries of additional feedingdepends on the success of FFW and FFT activities.2 In the event, as explained elsewhere in thisdocument, and for reasons beyond the control of WFP, it was not possible to carry out many of theseactivities. Consequently the reduction in the number of relief beneficiaries has been less than intended inthe PRRO design. This slowness in reducing beneficiary numbers reflects the difficulty in the presenteconomic and political context of moving away from relief. Nevertheless the Mission is of the view thatthe planned growth of FFW and FFT activities was over-ambitious in the uncertain circumstances at thetime of planning. This goal would, however, have been more achievable if the FAO Special Programmefor Food Security (SPFS), planned to start in 1999, had been implemented. It must be added, that theSPFS programme aim of “eradicating food insecurity at the national and household levels” would bedifficult to achieve even in economically and politically favorable developmental contexts.

Elsewhere, the present report considers, and makes recommendations on, several other aspects ofprogramme design, such as the targeting of beneficiaries (see section 5.2), which is not considered indetail in the design of the PRRO.

In conclusion:

2 The term “supplementary feeding” as used in the project document is misleading because WFP normally uses theterm to describe the provision of food to particular vulnerable groups in addition to provision of a general ration.In the PRRO there is no general ration. This report uses the term “additional feeding” to describe the part rationprovided for a section of the population, assuming that they are able to self-provide to some extent.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

7

Ø The PRRO was designed in a limited time, with some helpful guidelines but no visits fromspecialist advisers.

Ø The recovery strategy was based on the expectation of available agricultural land for FFW.Ø Little of this land became available however; the planned three-year reduction in the number of

beneficiaries was thus unachievable.Ø Although the Government of Azerbaijan had intended to release 47,000 ha of land for IDPs, it

was difficult to obtain plots of land in areas lacking good-quality productive land for localpeople.

Ø In some cases, the land made available was inaccessible or of poor quality; some was 50 kmfrom homes.

Ø The introduction of charges for irrigation water, which had previously been free and IDP’sinability to pay for non-labour inputs ensured that few IDPs were able to participate, even whereland was available.

Ø The exclusion of FFW participants from further rations was another disincentive.

Recommendation:• For the preparation of the new PRRO, WFP Azerbaijan should use the Country Contingency Plan and

the different scenarios to identify the circumstances in which relief or recovery activities shall beimplemented, and should modify the PRRO relief and recovery strategies continuously according tounfolding events.

4. PRRO IMPLEMENTATION

4.1 Efficiency Issues

WFP registers beneficiaries by family, defined as husband, wife, unmarried children and any adoptedchildren, using data provided by GoA and screened by WFP and WVI to remove multiple, inappropriateand bogus registrations. Beneficiaries need to show that they are bona fide IDPs through the productionof a valid passport3 and birth certificate and that their status as IDP accords with the recognized date ofoccupation of their home area. Registration data is further cross-checked with other organizationsdistributing elsewhere in Azerbaijan, for example IFRC, to eliminate multiple registrations but to allowIDPs to move between districts. WFP and WVI distribute rations on a two-monthly cycle using anetwork of 140 distribution points (DPs) served from three field logistics offices/sub offices, which haverail-side locations and sufficient warehousing (Map 2). WFP distributes directly to 9 districts and WVI to13 districts, and both WFP and WVI share one district making the total districts 23. The pattern of DPshas been devised so as to minimize travel distance for beneficiaries while keeping distribution costs low.In practice it is not economic to distribute to less than 100 households in one day, meaning that someIDPs must travel long distances for rations. In some cases, IDPs pay a small charge to lorry owners totransport commodities. In all areas the distribution can be achieved in one day. Each team distributes toapproximately 130 households per day.

The minimizing of travel distance for women is important, if the aim of ensuring that food is mainlydistributed to the women, is to be achieved. Thanks to extensive house-to-house investigations andefficient data management, WFP now has an excellent up-to-date, disaggregated (and credible)computerized database on all present and many former beneficiary families. Individual family recordcards, issued and updated for each distribution, are securely stored. Record keepers also showremarkably detailed knowledge of individual household characteristics, helpful in identifyinginappropriate registrations. Beneficiaries produce the former Soviet Union passport, or replacement Form9, for stamping at distributions, thus independent double records are created. Mission members have

3 The so-called passport is in fact an identity card, dating from the FSU. Those who can prove their entitlement, butlack the passport are provided with the replacement Form 9.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

8

never previously seen such thorough record keeping for IDPs or refugees. Such detailed records are anideal to which other IDP operations might aspire; though probably unachievable without a highly literatepopulation and a legacy of extensive documentation.

In achieving a high level of efficiency in the PRRO, WFP took good advantage of a generally favourableinfrastructural and cultural contexts and the possibility of recruiting high quality staff locally. WFP hadconsiderable previous experience of working with the main IP, WVI in the preceding EMOP. Thisprevious partnership had developed an efficient series of modalities which could be maintained in thePRRO. In the same operating environment WVI had themselves already developed an efficient team.Existing high levels of basic literacy and numeracy in Azerbaijan were an advantage in the recruitment ofall staff, but also in dealing with beneficiaries, virtually all of who are able to understand and act oninformation posted on notices, and for example, information on leaflets relating to food preparation andthe rules for entitlement. With the educational and cultural legacy of the FSU and in the context of astagnant economy WFP were able to recruit high quality office, administrative and managerial staff, whowere already highly educated, used to the disciplines and cultures of work and responsibility, and showhigh levels of commitment. Further training allowed the achievement of the sophisticated computer-based information system and allows the collection of credible assessment and M and E data. Thoughthere were some weaknesses in infrastructure, for example the road system had suffered lack ofmaintenance, WFP have used the network of improved roads and the existence of large, unused buildingssuitable for secure storage, in developing a distribution system which is both efficient and effective.

4.2 Assessment and targeting

In addition to the monitoring data from EMOP 5302 a number of documents, mainly produced by WFP,were used for the assessment of need and as a basis for targeting. Some of the most significant are:FAO/WFP (1999) Special Report Crop and Food Supply Assessment Mission to Azerbaijan; WFP (1998)Emergency Food Assistance to Internally Displaced Persons and Other Vulnerable Groups inAzerbaijan; World Vision (1998) Azerbaijan Household Food Security Survey of IDPs, UNDP/WFP(1999) Refugees and Internally Displaced Persons in Azerbaijan; WFP (1999) Donor AssessmentMission Report; WFP ( 2001) Gender Survey in Lieu of Food Security. The most recent is: WFP (2001)Household Food Economy Survey among the IDPs in Azerbaijan.

From 1996, before the PRRO, and in agreement with the GoA, WFP started to target distributionsgeographically by omission of the eastern strip of Azerbaijan, which was perceived to offer a greaterrange of livelihood opportunities, and to which GoA did not wish to see more IDPs attracted because thiswas the area with the largest cities. It is widely accepted that refugees or IDP migrants to cities are lesslikely to return to home areas and governments in general are concerned to avoid large influxes ofdisadvantaged people to large cities. IDPs, even from agricultural backgrounds were attracted to Bakuand Sumgait because they perceived these cities to offer better opportunities for the creation oflivelihoods than location in rural areas. Thus, with the exception of some 5 000 people in specificvulnerable groups, such as invalids and orphans,4 WFP now supplies only the dominantly rural areas.

WFP currently use entitlement criteria based on former Soviet Union and Government of Azerbaijanpractices: a system, which existed before the PRRO, and was used in the EMOP. Beneficiaries areaccustomed to and understand these criteria. WFP uses further exhaustive lists of criteria for exclusionfrom entitlement. These criteria were codified in June 1998 and further discussed with GoA in April2000, during the early stages of implementation of the PRRO. The checking of the validity ofentitlements involved house-to-house surveys by field monitors. Since 1999 and in order to reduce itscaseload according to the targets in the PRRO, WFP has accepted no new registrations of beneficiaries.The CO explains that some of these excluded people in fact self-exclude through making decisions in the

4 The institutionalization of orphans is unusual in the Azerbaijani IDP community. The Mission met one womanwho had three of her own children and was also caring for the four orphans of her brother. WFP provides rations forall of the household

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

9

knowledge of the consequences. For example prisoners are permanently excluded, as are people whohave missed two distributions; in both cases, it is argued, people are aware of the implication of theiractions. Those people who miss distributions through age or infirmity are able to have their commoditiescollected by a registered person. Undoubtedly some non-attenders are not in fact entitled bona fide. Otherpeople are excluded on the basis that they are probably able to self-provide because they are more able tofind paid employment. University students and returnees form military service are examples of thepresumed-employable group.. Since 1999, WFP has stopped any new registration of beneficiaries inorder to reduce its caseload. Newborn children have therefore not been admitted to the list ofbeneficiaries. It can be argued that becoming pregnant is a conscious and voluntary decision. Theexclusion of all new-born children is problematic.

Even with rigorous application of deletion criteria, however, WFP has failed to achieve the plannedreduction to 70 000 beneficiaries by year three of the PRRO. At present, there are 143,000 beneficiaries.Many IDPs complained to the Mission and to WFP staff accompanying the Mission that their exclusionor that of family members was unjustified. WFP considers each case carefully and consistently, keepingthe GoA informed of principles, actions and outcomes. The Mission witnessed the careful way in whichfield and CO staff dealt with these cases in the field and examined documents reporting to GoA theoutcome of appeals by beneficiaries. The commonest reason for complaint is the question of collection ofrations by someone other than a member of the beneficiary family. WFP rules are clear: invalids incategories 1 and 2, mentally ill and pensioners may have their rations collected by a close relativeprovided that this is registered through the office of the local authority’s Notary. In the opinions of theMission believe that targeting would be more efficient and equitable, and ultimately less contentious, if itwere based on the principle of inclusion on the basis of need, identified through the use of VAMmethodology. This would help to ensure that the most vulnerable receive the highest priority, and wouldbe more equitable than exclusion on the basis of presumed ineligibility In view of the strong feelingsexpressed by non-beneficiaries it is important that beneficiary communities are consulted in therefinement of targeting.

4.3 Logistics and leakages

Commodities are normally imported by rail via Batumi and Poti in Georgia and delivered directly to rail-side EDP warehouses (Map 2). The intervention of the Caucasus Logistics Advisory unit (CLAU) from1993 had significantly improved the efficiency of transport in Azerbaijan. Sealed rail wagons areinherently more secure than road trucks and transport agents are responsible for the security ofcommodities up to the EDPs. Comprehensive checks on warehouse stocks are made two or three timesper month, and in some cases more frequently. In one warehouse the manager physically checked allcommodities daily. The WFP senior monitor visits warehouses regularly to check stocks and usesstandard WFP criteria to make detailed investigations of aspects of security and good management. WVIhas prepared comprehensive manuals for commodity management, has trained staff in good practice andplaces illustrated posters in warehouses as reminders. Store keepers make a monthly check of allcommodities to guard against infestations. There has rarely been need for fumigation.

WFP/WVI advise beneficiaries 10 days in advance of distributions. Trucks used for final distribution areloaded during the afternoon before the distribution day and stored in secure conditions at the logisticssite. Loading of the commodities is according to a fixed layout within the enclosed truck, allowing arunning check on stocks remaining and increasing the efficiency and speed of distributions. Atdistribution sites, beneficiaries, almost all of whom are functionally literate, a legacy of the FSU, are ableto read posters giving details of entitlements for different family sizes, the rules of entitlement, and theidentification of the donor of each commodity. WFP and WVI staffs maintain continuous checks on theaccuracy of scooping equipment. WVI staff also maintain a close and transparent check on the weight ofrations through the use of good quality scales. Small amounts of temporarily undistributed rations aresecurely stored in locked containers at distribution points. A pipeline break at the start of implementationof the PRRO led to a two-month reduction in rations with some adverse reaction from beneficiaries. Atother times the pipeline has functioned effectively.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

10

WFP and WVI are congratulated on the minimal losses throughout each stage of the operation and onhigh standards of warehouse and distribution management.

4.4 Security

Security is not at present a significant issue in Azerbaijan, except occasionally near the occupied areasand not in areas where WFP is currently working. Some NGOs have had problems from illicit demandsby officials for payments and commodities but this has not been an issue for WFP. Azerbaijan is not aviolent society and, apart from the conflict with Armenia, ethnicity is not a problem, nor are there theprivate armies, warlordism and irregular troops that characterize many contemporary emergencies inwhich WFP operates.

In the future, however, insecurity will certainly become more problematic, whether military conflictrecurs or large-scale resettlement can occur. Resumption of conflict would probably create more IDPsand make WFP activity more dangerous. But a successful resolution of NK, allowing IDPs to resettlewould entail activities in dangerous environments. Mines and unexploded ordnance are widespread, andin many cases unmapped, in the areas where resettlement is now being attempted and will ultimatelyoccur. The present occupied districts have extensive mine fields which will probably take decades toclear, despite the efforts of the Azerbaijan National Agency for Mine Action (ANAMA) and specialistNGOs such as Halo Trust.

There have been no recent serious security incidents at distribution sites or warehouses or forbeneficiaries, though WFP/WVI sometimes ask police to attend distributions if they think that there is arisk of unrest. There were some threats to WFP/WVI staff when supplies were interrupted by the shortpipeline break in late 1999. Responses to the 2000 earthquake have revealed some small problems intele-communications in the Baku area. Codans are now being transferred to new vehicles, but WFPmakes extensive and effective use of mobile phones in the field. The UN family had one week of securitytraining early in 2001.

4.5 Monitoring and evaluation system

Three household food economy surveys were important inputs to the monitoring system: the 1998 WorldVision Azerbaijan Household Food Security Survey of IDPs, and two Household Food Economy Surveyamong the IDPs in Azerbaijan, carried out by WFP (one in 1998-99 then a follow-up in October 2001).The 1998 surveys found that seriously food-insecure families existed throughout the areas served, evenin the most favored districts and some of the elderly food-insecure were not receiving rations at that time.The main findings of the October 2001 survey are that compared with three years ago:

• Average household size has increased from 4.9 to 5.6;• The non-acceptance of new registrations has necessitated more ration sharing;• Only 22 per cent of households report a steady income;• More households report that they are unable to meet their food and non-food needs;• Almost half report that are unable to access fish, poultry and meat;• Only one percent of families report that they are able to grow food on their home gardens;• Sale of assets continues.

In every category, food insecurity appears to be increasing.

WFP and its main implementing partner WVI have effective monitoring systems, which build on thosedeveloped for the EMOP. WVI has 12 gender balanced field monitors and WFP has three, including theChief Field Monitor in the CO. WVI and Masalli sub office provide daily reports of distributions anddetailed weekly reports. At distributions, staff are available to discuss immediately the issues raised bybeneficiaries. A monitor is present at each distribution, and monitors make post-distribution visits to

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

11

houses. Two person WVI teams of female field monitors visit house-to-house to collect information onbeneficiaries. In 1999 WVI investigated the effect of the two month cessation of distributions when thepipeline was broken. CO staff also make frequent field visits, with systematic reports. Bi-monthlyrandom post-distribution monitoring gives detailed information on conditions in individual homes andprovides an opportunity for beneficiaries to discuss issues in confidence and allows the rigorousexclusion of multiple registrations. The gender balance of monitoring staff allows both men and womenbeneficiaries to access appropriate staff to raise issues relating to food. Monitors have also used theopportunity to discuss aspects of the ration; an example is the discussion with women on the methods ofcooking pulses, which led to the production of a cooking advice leaflet. The database of beneficiaries issupplemented by detailed knowledge of individual households gained in field monitors’ visits. In thefield visits it was clear that field staff were familiar with, and were recognized by, large numbers ofbeneficiaries.

The Mission was concerned that the special needs of children should be considered, because of theirvulnerability and the implications for the future of society. Some 35 per cent of IDPs are under 15 years.In 1995 one in seven of the IDP children was without a parent. Some 20 per cent of households weresingle-parent-headed and of these 60 per cent were women household heads. An increasing number, andalmost all under tens, have been born in IDP conditions. WFP support for work with war-traumatisedchildren is discussed in section 6.3. Many children seen by the Mission were noticeably small for theirages (stunted) in comparison with their normal sized parents. This observation accords with the findingsof ad hoc nutritional surveys showing stunting and wasting in IDP children. The Multiple IndicatorsCluster Survey (MICS) of 2000 showed 20 per cent with severe and moderate stunting; 8 per centshowed wasting. Some 9.5 per cent were of low birth weight. Almost half of under-five IDPs wereanaemic. A 1996 survey of IDP children found that one quarter had experienced diarrhea in the previousfortnight. The 1996 WFP nutritional survey of under fives found three per cent with severe malnutrition,seven per cent with moderate and twenty per cent with mild malnutrition. WFP has no in-house capacityfor nutritional monitoring but works with competent partners, including UNICEF, WVI and other INGOsto gather this information, though ad hoc rather than systematically.

In view of the former, high standards of education, school attendance and the quality of educationreceived by IDP children has been problematic. This is partly because of the cost of suitable clothing andequipment and partly because of the tendency to separate IDP students from local students in schools.5

WFP cannot help directly in the provision of equipment or clothing6 but school feeding programmescould help to address some of the problems of over-five child malnutrition, could possibly help reversethe claimed and much to be regretted fall in education standards generally and could also help reduce thedrop in school attendance, especially among girls. Findings of nutritional surveys indicate continuingcause for concern: it would be advisable to undertake regular, standardized, nutritional surveys todiscover whether nutritional status is being maintained. The household food security surveys in 1998carried out by WVI, with support from SCF and USAID and in 2001 by WFP indicate that there areserious cases of need due to poverty and serious food insecurity in all districts.

4.6 Management

WFP Management systems are essentially unchanged from those of EMOP 5302.04. The CO, based inUN House Baku, has eight staff with two expatriates: the Representative and the Emergency Officer, andtwo drivers. WFP has been fortunate in being able to recruit and retain high quality local staff. ThePRRO uses three logistical points (Map 2): Masalli, a WFP sub-office with 12 support staff; and theWVI offices at Imishli and Mingechevir with a total of 99 support staff. This system of logistical pointsis cost-effective in distributions and allows close supervision of most implementation activities. The

5 IDP children are normally educated in the same schools as local children but in some cases they are taught atdifferent times, characteristically local children morning and IDPs afternoon. In other cases IDP children are taughtin separate schools. In such circumstances it is possible that the outsiders will receive an inferior education.6 The Mission was informed that many IDP children were dressed in second-hand clothes.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

12

number of women receiving commodities falls to about 30 per cent where the distribution points aremore distant from homes, whereas it is more than 60 per cent at those sites nearer their homes. Inmeeting commitments to women any wider spacing of DPs would be very unfavorable. Compared withother situations known to Mission members, however, the pattern of distribution sites is favorable tobeneficiaries. Staffing of both WFP and WVI is gender balanced throughout the operation and down tofield level. The CO staffing is sufficient for present needs but a change in the conflict/political situationwill necessitate an increase in staff whether the change is to conflict or resettlement. Either change wouldneed additional staff for VAM, logistics and monitoring. The CO is congratulated on the efficiency ofmanagement systems.

Recommendations:• As an aid to future targeting, a VAM mission should be requested to advise on needs-based

targeting techniques in order to identify the most vulnerable groups. In the identification of newtargeting criteria, involvement of IDP communities should continue with the cooperation of theGovernment of Azerbaijan.

• WFP should continue to cooperate with other United Nations agencies, civil society, partners andthe Government of Azerbaijan to secure regular systematic nutritional surveys.

• The introduction of a school-feeding programme should be considered.

5. RESULTS OF THE PRRO

Discussion of the effectiveness and sustainability of PRRO activities is structured according to the keyprogramme objectives of the PRRO document.

5.1 Protracted Relief

At present the GoA considers that it is unable to guarantee provision of food relief for IDPs withoutcontinuing WFP support, and that WFP rations are essential both for the maintenance of nutritional statusand to maintain public order. In relation to the latter issue the Mission were informed that there had beena near riot when the pipeline had been interrupted in late 1999 and a distribution was not possible. TheMission is also aware of incidents in which would-be beneficiaries have reacted strongly to theirexclusion from distributions. Security events in Georgia during the Mission’s visit and comments duringinterviews with GoA point up the significant risk of breakdown in public order.

In November 2001 WFP provided a two-monthly ration to each person of:• 12 kg. wheat flour• 1.8 kg. pulses (yellow split peas)• 1.2 kg. vegetable oil• 0.9 kg. Sugar• 0.3 kg. Salt (iodized) supplied pre-packed in polythene bags• 0.185 kg. tinned fish (tuna)• 0.3 kg. tea was provided for each family, supplied pre-packed in polythene bags.

WFP also provided tinned cheese when the commodities were available. Iodine is added to the salt tocounter IDD, Vitamin A is added to the vegetable oil and the wheat flour is fortified with iron to givesome protection against anaemia. Beneficiaries told the Mission that they were satisfied by the quality ofthe rations, with the possible exception of the peas, for which WFP, in consultation with beneficiaries hasprovided recipes and cooking instructions.

The Mission considers that the WFP ration of 50 per cent of daily needs has contributed verysignificantly to the nutritional maintenance of a large number of food-insecure IDPs. This opinion isbased on evidence from the field visits and includes:

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

13

• interviews with beneficiaries;• the fact that there was no evidence of sale of WFP commodities in street markets near distribution

centres and in the Mingechevir central market after distributions;• the evidence of careful storage of WFP rations;• the widespread use of the sugar ration in preserving fruit (a traditional method of food preservation

and in effect an amplification of the ration); and• the limited amount of non-WFP food in houses.

The apparent prevalence of stunting in children, which is supported by nutritional survey evidence,suggested to the Mission the need for continued support for nutrition, as does the high level of ill-health.Evidence from household food security surveys indicates a decline in food security from 1998-2001.Surveys show continuing poor nutritional status (section 5). The large number of unemployed young andmiddle aged men and women seen during the day around their homes for example in Mingechevirconfirms the food security finding that few will be able to buy or self-provide much food.

The Mission noted at distributions that, in line with the monitors’ reports, which show that over 60 percent of recipients were female if the DP was near home, falling to 30 per cent if the DP was more distant,more than half of recipients they saw were women. Almost all recipients were adults, though childrenover the age of 12 would be allowed to collect rations.

From the evidence above it seems unlikely that WFP rations are creating dependency. Nor does it seemthat half rations are an incentive not to seek work or create an alternative income. If IDPs are notworking it is because there is at present little opportunity for working.

5.2 Returnees and Resettlement

In the early stage of the PRRO, as part of a joint GoA/UN/EU project, WFP assisted some 2000 IDPsreturning to Horadiz Village in Fizuli District , in the liberated territories, by providing six months of 50per cent of food needs. Some IDPs were returning after seven years absence but a few others hadreturned as early as 1994. In the project document (paragraph 34) it is foreseen that many IDPs might beable to resettle during the second phase of the project. The hope of the GoA is to resettle some 35 000IDPs in an integrated programme involving de-mining, shelter, agriculture, irrigation, income generation,health, education, transport, energy and water supply. A WVI nutrition team found that there were fewerthan expected returnees. Half of the returnees had nothing left to sell to raise money, and one quarterdepended on food aid for more than half of their food. Reasons suggested for the limited number ofreturnees were:

• the prevalence of mines and unexploded ordnance;• the fact that not all returnees were farmers;• the almost complete destruction of infrastructure;• reluctance to return older girls to an area with many soldiers;• lack of hospital facilities;• lack of a food market; and• continuing danger from the occasional shelling and the perceived risk of a recurrence of violent

conflict.

The PRRO document, paragraph 34 intends that 51 per cent of the resettled would be women. Theinsecure conditions mean that, in fact, men have tended to be the pioneer resettlers. In StrategicDirections towards Development (2000), the President of Azerbaijan identified psycho-social traumas aspossibly the most significant problem in resettlement activities. These problems may be associated withthe stress of return to a destroyed and still-dangerous home environment but also to the difficulty ofsurvival and the need to make a fresh start in a radically different political/economic context and in aseverely stressed physical, infrastructural and social environment. The limited resettlement of Horadiz

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

14

raises the question of the feasibility of achieving GoA target of 35,000 returnees, but the PRROdocument had envisaged the return of even more, some 50,000 people. This figure was based on the hopeof a political settlement of the NK problem: no such political settlement materialized. Even thoughalmost all IDPs express a strong desire to resettle, any large scale resettlement depends on the solution ofpolitical and other problems beyond WFP control.

Recommendation:• If WFP should in future be involved in the support of resettlement, it should provide three months of

full rations and longer-term support through FFW for reconstruction activities and FFT forpsycho-social and other needs.

5.3 Recovery (FFW and FFT)

In the PRRO document WFP had planned to assist 5,000 IDPs in year 1, 10,000 in year 2 and 15,000 inyear 3 through recovery activities with 65 per cent of participants being women. Agriculture, agro-forestry, social and economic infrastructure were to be the areas of activity. Though GoA had intended torelease 47,000 hectares of land for IDPs, it was understandably difficult to obtain the cooperation of localauthorities, as they would be losing land to IDPs with no gain to themselves. The mayor of Imishliexplained that with a local population of 110,000 it was not possible to provide land for 20,000 IDPs. Infact local people would lose the benefit of the low-paid labour at present offered by some IDPs. Suchland as was made available was inaccessible or of poor quality; some allocated land was 50 kilometersfrom homes. The introduction of charges for irrigation water, which had previously been free and theirlack of capital to pay for non-labor inputs ensured that few IDPs were able to participate even where landwas available. The exclusion of FFW participants from further rations was an additional disincentive.Allocation of land rights to the community rather than individuals might be more acceptable to localcommunities and GoA.

WFP circulated NGOs on 27 September 1999 requesting proposals for implementation of FFW and FFT.These requests elicited only one potential participant. Their reply on 10 August 2000 proposed quailfarming: the NGO’s expectations of the level of support that could be provided by WFP were unrealistic.

Two other types of FFW were implemented with partners. In cooperation with UNICEF, 36 pre-schoolcentres for development, had been set up during the preceding EMOP (Map 3), intended to secure andstimulate the development of 2 000 children born and brought up as IDPs.7 Support for these pre-schoolscontinued in the PRRO; the number of children is now 2540 and the number of trained care providers254. Care providers have been trained by the GoA Ministry of Education with help from UNICEF, withWFP providing FFT rations during the trainings and FFW while working. WFP provided a sandwichsnack for children. These kindergartens are run throughout the year, without holidays, on a two-shiftbasis. Care providers show a high level of commitment for a modest return: having been trained almostall care providers continue their child care activities. The Mission were impressed by the quality of workundertaken by the care providers and the supervising head teacher, and believe that such work isnecessary if the trauma associated with IDP status and the legacy of war in the older generation is not tobe reproduced in another generation. Care providers have developed skills that could be useful andsustainable assets for the community as a whole, and might be used as a basis for the development oftrauma counseling with adult members in the IDP. Children within the conventional school system wouldalso benefit from such support. Traumas associated with conflict and severe deprivation may persist formany years and even intensify with time so that skills in counseling could be of very long term value.

7 The term “war traumatised children” has been used to describe the children presently being treated. This ismisleading as the form of words is widely used to describe children who have directly experienced the impact ofwar: witnessing violence, violent death, extreme fear, destruction and other such events. War traumatised maydescribe the children who are now adolescents (but are receiving no help). It is more accurate to characterisepresent under-fives as children deprived by the limiting environment of displacement and negatively influenced bytheir elders’ war trauma.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

15

The second FFW supported activity was the reconstruction of four schools, two with ADRAInternational and two with Relief International. WFP also supported Relief International with FFT over15 days for 680 volunteers training in community health, basic medical assistance and in peace/tolerance.

In another support for recovery WFP investigated with WVI the possibility of supporting IDPs in incomegeneration through a micro finance programme. Only 1.44 per cent of the WFP beneficiaries wished totake up any loans at all and very few indeed (144 out of a possible 110,000 WFP beneficiaries) seek asecond loan. The taking of a second loan was taken by WVI to be an indicator that the project wasgaining momentum. To some extent, this low uptake may be explained by the lack of entrepreneurship.This may be attributed to the long period of state-centered decision making, to the fact that the bettereducated had been employed in state industries and activities, such as health and education whereentrepreneurship was not encouraged, and to the limited opportunity for self-managed economic activityin the FSU. It was also suggested to the Mission that micro finance could serve only the needs of thearistocracy of the poor and that for most IDPs the underlying problem was the IDPs’ poverty.

In the field, the Mission examined the deplorable living conditions of IDPs. All the settlements visitedhave poor sanitation, leaking buildings, and lack of waste-disposal facilities. Railway wagons have noinsulation, causing them to be hot in summer and cold in winter. Most of the buildings, of whatever type,have limited and unreliable water supply, if any. IDPs living in the dugouts visited by the Mission saidthat they had to transport water some two kilometers by donkey. At Kurdamir, water is, in effect,rationed so that people need to be at home to compete for limited amounts of water when the bowserarrives, which might be as rarely as once a week. Some settlements are badly drained; there areinfestations by insects, including in some cases malarial mosquitoes, vermin, rats and even, in thedugouts, snakes. Few buildings allow privacy either within or between families. Sanitation is, at best,rudimentary, with several families sharing toilets, which are consequently badly maintained. Theseconditions probably contribute to the high incidence of physical and mental sickness of people of all agesthroughout the community and to the reproductive ill-health of women. FFW and FFT could help toameliorate these environmental factors. WFP has on several occasions unsuccessfully requested GoAparticipation in environmental rehabilitation. The Mission understands that GoA would now be willing toconsider providing material support for improvements to the environments of settlements.

Recommendations:• The preschool partnership with UNICEF should be continued and, if circumstances allow, its

activities extended to other areas.• WFP should support FFT and FFW activities to improve the living conditions of IDPs, for example

through FFW to maintain and repair water supplies, sanitation and shelters and to construct camproads, and through FFT to provide nutritional and psycho-social advice.

• WFP should seek Government of Azerbaijan non-food items (NFIs) for FFW activities. Thefeasibility and desirability of modifying the mechanisms for FFW should be assessed, for example,as employment-generating schemes.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

16

6. PARTNERSHIP AND CO-ORDINATION

WFP has three types of partnership in the PRRO: first with the GoA; second with UN agencies; and thirdwith NGO implementing partners. All these partnerships are governed by letters of agreement (LoA) orletters of understanding (LoU). The WFP Representative has frequent contact with the GoA, particularlySCRIDP. Much of the WFP negotiation with local communities, local governments and IDPs is throughthe GoA, specifically SCRIDP. This is partly because at the local level negotiations can become tied upin bureaucracy, a legacy of FSU. Negotiations with the IDPs are facilitated by the Deputy PrimeMinister’s deep interest in, detailed knowledge of, and close contact with the IDP community. WFP wasa major participant in the IDP National Conference on Development Strategic Directions towardsDevelopment, (2000). This conference led to better and more open relations between GoA, the UN,international non-governmental organizations (INGOs) and local NGOs. The conference also clarifiedand sought solutions to some issues relating to perceptions of corrupt and obstructive behavior towardshumanitarian agencies by some officials in Azerbaijan.

From 1999, most UN agencies have been in UN House, making UN coordination easier in practice. Aspart of the United Nations Country Team (UNCT), WFP has played a key role in the preparation of the2001 Common Country Assessment (CCA), leading on the Food Security Theme and chairing the FoodSecurity Committee. WFP also led the preparation of the UN Contingency Plan for Azerbaijan.Following earthquakes and a landslide in 1999 and 2000, and with WFP as a member, a United NationsDisaster Management Team (UNDMT) will coordinate responses to future emergencies.

WVI, operating in Azerbaijan since 1994, is the main WFP implementing partner (IP). WFP signed a(LoA) with WVI in 1996 and WVI was the key IP for EMOP 5302. WVI reports daily to WFP and inboth Baku and the field there are good working relationships. As a large, experienced, well-funded andwell-staffed organization, able to undertake specialized monitoring assessment activities, WVI isresponsible for 117,902 beneficiaries. Local NGOs would not currently have the capacity or experienceto undertake this level of activity. ADRA International and Relief International are also implementingpartners. The LoA with WVI specifies in detail gender-disaggregated reporting and fulfillment of WFPCommitments to Women. There is no specific reference to environment in The LoA: such a reference isnot needed in this case. Coordination with the main IP, Government and UN agencies is unchanged andcontinues to be good, building on working relations developed in the implementation of EMOP 5302.

7. RESOURCING ISSUES OF THE PRRO

In October 2001, the PRRO had a 30 percent resource shortfall, of approximately 14,000 tons of food.In-country stocks were 6,712 tons. During the later 1990s donors had shown decreasing interest incontributing to the protracted operation, hence the reduction in the number of INGOs in Azerbaijan.USAID, the largest donor is the only donor that it was possible for the Mission to meet. In thesecircumstances, the PRRO has been successful as a funding channel, though WFP has continued to lobbyfor funds. France, EU, Germany, Japan, Switzerland, USA and Canada have contributed through bilateralaid. The USA, Sweden and others have contributed multilateral aid. In total The USA has donated 51 percent of the contributions delivered up to 15th October. At the current rate of consumption, and if thepresent resource shortfall is secured, the balance of food would enable the continuation of the programmefor at least six months.

Although much of WFP’s previous experience in Azerbaijan has been in emergencies and relief ratherthan developmental activities, the Mission agrees with the CO judgement that currently the CO hasadequate staff resources to implement the PRRO. Additional staff will, however, be needed when thepresent “no war, no peace” situation ends, whether the outcome is peaceful and allows resettlement orviolent and leading to the need for further relief activities.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

17

The $400,000 budgeted for NFIs to be used in agriculture has not been utilized yet. These resources arein principle viable and could be used in support of other FFW and FFT activities. Because theopportunities for agriculture are limited at present, it would be desirable to seek other uses for these NFIfunds. Improvements in living conditions and social support are areas where the funds could be used toadvantage. It has not yet been necessary to use the contingency mechanism to deal with securitysetbacks. Budget revisions have been restricted to the additional donations of cheese and canned fish.

At the start of the PRRO late commitments from donors were partly covered by carry-over stocks andpartly by the delayed delivery of EMOP stocks. A temporary shortfall led to the suspension ofdistributions for two months. Resource constraints have been similar to those in the EMOPs.

During their visit, the Mission were able to talk with only one of the donors, USAID, which is theprincipal donor. For several reasons, USAID regards the PRRO category favorably in comparison withthe EMOP. USAID approves the emphasis on recovery activities, the encouragement of a move fromcontinuing relief and the reduction in the likelihood of aid dependency. USAID also perceives WFP tohave advantages compared with other channels: this was partly related to a previous negative experiencein Azerbaijan, during the EMOP, in using one large INGO as a channel, and in which they had to askWFP to take over the INGOs case-load. Specifically they commended WFP on the effectiveness andefficiency of their programme of detailed screening to avoid multiple registrations, their partial successin reduction of the number of beneficiaries and their willingness to look towards the rebuilding oflivelihoods, even though there was limited success in the focus on livelihood. Though USAID considersthat there is a possibility of the development of a culture of dependence, they also acknowledge and areconcerned by the risk of social unrest if relief rations are reduced, noting the problems that occurredwhen WFP had to cease distributions for two months. Transition to development is hampered by the 907criterion which has up to November denied the provision of USA aid to Azerbaijan in support fordevelopment, as opposed to humanitarian activities. It is believed that the GoUSA may change theirposition relating to the status of Azerbaijan in relation to development aid in the near future. This couldhave beneficial effects both in relation to the PRRO and to the possibility of GoA’s being able to supportmore quickly the handover of PRRO activities according to the exit strategy. A change in GoUSA 907criteria would also have implications for the WFP Azerbaijan Contingency Plan.

Recommendations:• WFP should explore the possibility of using the resources allocated to NFIs ($400,000) for FFW

activities aimed at improving IDPs’ living conditions.• WFP should continue to encourage and facilitate donor visits to implementation sites.

8. MEETING WFP’S COMMITMENTS TO WOMEN

Annex 4 provides detailed and systematic analysis of WFP response to CW. Other sections of the reportconsider relevant aspects of CW. Immediately before the preparation of the PRRO, in December 1998,WFP undertook the Mid-term Evaluation of WFP’s Commitments to Women, in which a WFP genderspecialist reviewed favorably WFP Azerbaijan’s achievements during the preceding EMOP. During theimplementation of the PRRO WFP also produced, in 1999, the Midterm Review of Implementation, in2001 the Gender Survey in Lieu of Food Security and in 2000 the Gender Action Plan for Azerbaijan2000-2001. The last was considered by a visiting WFP gender expert to be the best of the dozencountries reviewed and of excellent quality. Following the recommendations made by the ExecutiveBoard with regard to the 1999 mid-term evaluation, two senior staff were assigned as gender focal points.WVI has gender focal points at its field logistical points. One of the Mission members is the gender focalpoint in the Cabinet Office of GoA and confirms the strong link with WFP Azerbaijan in gendersensitivity. In relation to Commitments to Women (CW) WFP has carried out the following actions:

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

18

• firstly, ensuring gender balance among WFP and WVI staff overall, the latter stipulated in the LoA;• secondly, ensuring gender balance at field level in both WFP and WVI staff. In the field,

beneficiaries thus see women in positions of authority and female beneficiaries have access to femalestaff;

• thirdly, WFP seeks to resolve gender issues relating to IDP women and girls, for example in ensuringthat the pattern of distribution points, as far as possible, favors the attendance of women; and inensuring that female field monitors visit houses to elicit information from women.

In the field the Mission witnessed the significant part played by women both in the part of theprogramme involving contact with beneficiaries and in other activities. Women head several of the WVIdistribution teams, a woman is in charge of the WVI Mingechevir warehouse and several act insupervisory positions in WVI offices.

As a legacy of the FSU, women in Azerbaijan, including IDPs have a more equal share of control overtheir circumstances than in most of the areas in which WFP works. In accordance with Azeri traditionand the rules of the FSU, marriages are monogamous. Women have been well educated and many IDPwomen are highly skilled for example as doctors, teachers, and lawyers. Unfortunately there are atpresent few opportunities for employment in these, or any other, fields and maintaining skills isproblematic. Whereas this is undoubtedly frustrating for women, the high levels of skill are potentially anopportunity when recovery activities become more feasible and will be when improvements in livingconditions or psycho-social rehabilitation are carried out.

There are, however, a number of causes for concern among IDP women. In 1997 the World BankAzerbaijan Poverty Assessment noted a high level of domestic violence against women. The 1998 theMid-term Evaluation of WFP’s Commitments to Women commented on increasing levels of prostitution(though not specifying IDP women). The Mission were informed that pressure was put on some womento become involved in trafficking of drugs from Afghanistan. In the field, the Mission met many womenwho were suffering a variety of illnesses, confirming the findings of, for example Médecins du Monde(Greece), and also noted the small stature of many children. The Mission also noted that many girls hadvery short hair, unlike traditional cultural norms, and possibly a response to difficult living conditionsand the shortage of water for washing hair. It was suggested by some IDPs and by others that schoolattendance was falling, particularly for girls: this will be counter to the developmental aims of the PRROif allowed to continue. WFP has a limited capacity to respond to these negative trends but the FFW andFFT activities recommended elsewhere could help to alleviate a number of these problems.

The Mission was impressed by the gender-disaggregated data on adults and children: these data allowmonitoring of the gender implication of activities and may be used in the identification of gender-relatedproblems. WFP registration cards show that 49 per cent of households are female headed and thatoverall, women collect more than half the rations. Women prepare almost all food and men alone makedecisions about food management in only 15 per cent of households. In pre-school development centres,supported by FFW and FFT, all the employees are women: this may not be entirely an advantage inrelation to role models for young children. The Mission envisage that many of their recommendedFFW/FFT activities would be suitable for women, particularly extensions of war trauma interventions, inwhich women’s capacity has been demonstrated in the existing pre-schools. They also envisage thatwomen will be fully involved in the planning of FFW/FFT activities: in the field they made manycomments on the need for such activities to improve living environments.

Currently no additional food is provided for pregnant and nursing women, though normally these groupsare considered, both in themselves and on behalf of the child, to be in need of additional food. A VAMMission could identify the particular needs of other food-insecure groups of women such as the loneelderly. One particular problem for some elderly women that the Mission interviewed was the cost ofpurchasing fuelwood for cooking, because they were too infirm to collect it themselves. Though theseobservations relate to women, it should be noted that in issues of health, elderly men are alsoexperiencing difficulties.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

19

Recommendations:• In continuing food support to IDPs, the extra nutritional requirements of expectant and nursing

mothers should be taken into consideration.• WFP should continue to give women an equal opportunity to participate in FFW and FFT.• The detailed gender-disaggregated monitoring of all activities should be maintained.

9. ENVIRONMENTAL CONCERNS

In common with other wars, the conflict in and around Nagorno Karabakh has caused considerableenvironmental damage in those areas, particularly through the widespread and deliberate destruction ofbuilt environments and infrastructures. The presence of minefields and unexploded ordnance continues tolimit the possibility for resettlement until expensive clearance has been carried out. Minefields constituteabsolutely insecure environmental insecurity, at least until their extent is precisely mapped, andenvironmental no-go areas until they are cleared to humanitarian mine clearance levels (until all minesare removed). ANAMA is involved in this activity which will take decades to complete to humanitarianmine and ordnance clearance standards. The occupied districts are thus both degraded and hazardousenvironments: in some areas they are, in effect, completely useless.

Elsewhere, large numbers of IDPs have been concentrated in small areas of Azerbaijan for almost tenyears: in the case of Imishli, 20,000 IDPs have been added in a very short time to the local population of110,000. There has been no plan, resource or opportunity to accommodate the environment to theirpresence, through the creation of new environmental resources such as trees or water supply, or by thedevelopment of infrastructures to protect the environment, for example drainage systems or durable roadsurfaces. Nor has it been possible to create infrastructure beyond the absolute minimum for survival. Insome cases, the infrastructure available (railway wagons, barns, derelict factories as homes for eightyears) is scarcely the minimum. It might be anticipated that the presence or actions of IDPs would causeconsiderable environmental degradation. The Assessment of Drought Damage and Priority Needs forAssistance report in fact suggests considerable degradation caused by IDPs cutting wood, but theMission saw little evidence of such degradation of trees except along a road north of Masalli. Electricity,kerosene and gas, where and when available, though in the case of electricity with many cuts of supply,particularly in winter, are supplied free of charge to IDPs. Some IDPs use these fuels for cooking andsome for space heating, and thus the demand for fuel wood is reduced. The Mission saw some womencooking with wood fuel, though dung was more widely used. Potentially dung fuel can have extremelyadverse effects on women’s health, affecting eyesight and causing heart disease. Cooking of the rationsprovided by WFP does not create an exceptional demand for fuel.

Relief activities have few negative effects on the physical environment. Supplies are imported direct toEDPs by rail, an environmentally benign transport mode, and larger roads are generally reasonably wellsurfaced, so that the trucking of supplies to DPs does not cause undue road destruction. None of the DPshas so many beneficiaries as to create additional pressure on environment. If it had been possible toundertake the planned recovery activities in agriculture and agro-forestry, and because agriculture hasexperienced under-investment, environmental improvements might have been achieved throughinvestments in labor, for example in improvements to irrigation systems or through well-planned treeplanting. Unfortunately suitable land was not available. Similarly, the resettlement activities at Horadiz,because of the small scale of the pilot and practical difficulties, achieved little in the way ofenvironmental reconstruction. In any future resettlement activities, though, WFP food could be used inthe sustainable re-creation and restoration of environment, through FFW.

By far the most significant current environmental and public health issue for IDPs is their degraded,degrading and unhealthy living circumstances. While these are in no way created by WFP activities, foodmight be used to create quick improvements in these deplorable and debilitating conditions. The Missiondiscussed with SCRIDP the possibility of GoA contributing materials for such environmental

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

20

improvements in partnership with WFP, and using the FFW mode. SCRIDP were optimistic that theywould be able to supply suitable materials using funds from a recent World Bank loan. WFP could alsouse NFIs intended for agriculture for these purposes. It is important that IDPs should be fully involved inthe prioritization, planning and eventually in the management and maintenance of these improvements aswell as being the source of labor in their implementation. The method for achieving this full participationmust be decided by WFP CO on the basis of their experience in the field of negotiation with IDPcommunities. The Mission appreciates that such negotiations must be within the political frameworks ofAzerbaijan and that environmental improvements have implications and possible benefits for localcommunities. Skills developed for the planning, management, implementation and maintenance ofenvironmental improvements will also be directly usable in the resettlement phase, when these activitieswill need to be carried out by the returnee community, rather than falling back on the post FSU state toplan, provide and manage. It is anticipated that IDPs will be involved in maintenance activities right upto the time of their resettlement when the same skills will be immediately applicable. Thus improvementsto the present living conditions will help prepare for the achievement of the longer-term aims of thePRRO, while at the same time satisfying immediate needs. The limited success of the Horadiz projectmay partly be explained by the lack of such long term preparation of the IDPs for the challenge ofresettlement.

Recommendations:• WFP should use FFT and FFW to support health and environmental improvements in the living

conditions of IDPs, with NFIs from the Government of Azerbaijan and with the participation of IDPsin prioritizing and planning.

• That WFP-AZE use FFT and FFW to support environmental improvements in the living conditionsof IDPs, with non-food inputs from GoA.

• That WFP-AZE discuss with GoA the implementation of environmental improvements with theparticipation of IDPs in prioritizing and planning.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

Annexes

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

Annex 1

Terms of Referencefor the Evaluation of Azerbaijan PRRO 6121.00

1. Background to the PRRO Category

The PRO category - covering Protracted Emergency Operations for Refugees and Displaced Persons -was first established by the CFA on the recommendation of WFP in May 1989 (WFP/CFA:27/P/7). Thecategory responded to (i) the growth of migrant groups – both refugees and displaced people - as a resultof civil strife, and (ii) the persistence of the problems, which led to their flight and the consequentperpetuation of their status over long periods. In short, a decade of civil conflicts had contributed to asense that the EMOP category was not suitable for protracted conflict situations. At the same time, therewas a need for an adequate and predictable funding base for refugees. By creating a subset of its“development” resources, WFP hoped to preserve the development and emergency resource bases fortheir original purposes while attracting additional net resources to deal with burgeoning needs in this newcategory.

In April 1998, WFP introduced a significant enhancement to the PRO category when it endorsed WFP’spolicy proposals in the paper “From Crisis to Recovery” (WFP/EB.A/98/4). The new PRRO – ProtractedRelief and Recovery Operation - brought two major modifications to the category. Firstly, all protractedemergency operations – and no longer just refugee and displaced persons operations - would betransformed into PRROs and brought before the Board for approval, generally after two years. Secondly,the transformation of EMOPs into PRROs would be done based on the preparation of a recoverystrategy. This strategy would provide the rationale for continued assistance and, to the extent possible,emphasize recovery activities in addition to ongoing relief needs and contribute to conditions for findingsustainable solutions to protracted crises. This emphasis on recovery was very much in line withinternational support for linking relief and rehabilitation work to longer-term development interventions.Refugee and displaced persons operations – the old PRO category – would remain a subset of the newcategory and would benefit from the introduction of a recovery strategy.

The new PRRO category also stressed two important resource dimensions:

• First, within a PRRO, WFP would have the flexibility to move funds freely between relief andrecovery activities as the situation on the ground required - to seize opportunities for recovery, but tobe prepared to meet unforeseen critical relief needs;

• Second, the new category called for longer-term financial commitments from donors – forpredictability and planning as well as for building partnerships for recovery.

During its sessions in October 2000 and February 2001, WFP’s Executive Board considered issuesrelated to the funding of PRROs and raised questions about the effectiveness of the new category. TheBoard endorsed a review of the PRRO category as a whole to be undertaken by OEDE based on thefindings of 10 PRRO evaluations scheduled for 2001-2002.

Of particular interest to the Board are the following broad issues:

• What substantive improvements in the delivery of WFP assistance have been achieved as a result ofthe introduction of the policy From Crisis to Recovery and the resulting conversion of EMOPS andPROs to the new PRRO category?

• Is WFP identifying sustainable recovery activities through the PRRO category and is it attracting andinvesting adequate resources - resources that complement its traditionally strong base of relieffunding - to implement those activities effectively?

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

2

2. PRRO 6121.00/ Relief and Recovery Assistance for vulnerable groups - An Overview

Azerbaijan PRRO 6121.00 was approved by EB2.99 in May 1999 for a three year period (July 1999 toJune 2002). It is valued at some US$23 million, with 47,880 tons of food and 485,000 beneficiaries. Themajor beneficiary group is IDPs and socially vulnerable people. Azerbaijan is classified as an LIFDC.

ODR is expecting to extend the PRRO for a further period of time and a formulation mission will visitthe country in early November. It is hoped that the last days of the evaluation mission will overlap withthe new operation formulation mission. The new PRRO should be presented to EB2 in May 2002.

The goals of this PRRO are meant to help the most needy among the IDPs and vulnerable groups toensure that minimal food security is reached and that self-reliance is encouraged. The specific objectivesto achieve the above goals are to:a) contribute to improving food security among IDPs and vulnerable groups;b) encourage voluntary resettlement of returnees in their places of origin;c) encourage integration of IDPs in their communities; andd) increase opportunities for beneficiaries to become self-reliant.

3. Objectives of the Evaluation of PRRO 6121.00

The objectives of the evaluation are:1. To assess the efficiency, relevance, effectiveness, coherence and sustainability8 of WFP assistance

under the PRRO in order to improve the implementation of the current operation and assist withplanning the next phase;

2. To assess the usefulness of the PRRO category both as a resource window and as a programminginstrument; and

3. To provide accountability to the Executive Board.

4. Scope of Work

The evaluation of PRRO 6121.00 will focus initially on the PRRO’s recovery strategy, assessing how itwas formulated, how it has evolved over the life of the project and its relevance to the situation analysis.

Based on its recovery strategy, PRRO 6121.00 identified activities within the following componentelement(s): protracted relief, protracted returnees and recovery. The component element(s) will beassessed individually to determine a) if the activities are taking place, outputs are being delivered andtargets are being reached and b) if this is sufficient to achieve the stated objectives.

On a practical level, this will include assessing the systems and support (financial, staff, partnerships,etc.) underlying the PRRO. On a more general level, the strategic linkages between the componentelements will be assessed to determine whether the PRRO has successfully seized opportunities forrecovery.

The evaluation will also examine the participation of implementing partners, sister agencies, and otherstakeholders: first, to determine their contribution to meeting WFP’s objectives; and second, to shed lighton whether the PRRO has contributed “to the process of transforming insecure, fragile conditions intodurable, stable situations…”9 .

8 Standard evaluation criterion such as efficiency, effectiveness, relevance, coherence and sustainability used inthese TOR are defined in Guidance for Evaluating Humanitarian Assistance in Complex Emergencies (OECD,Development Assistance Committee, 1999).9 “From Crisis to Recovery”, WFP 1998.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

3

The evaluation will also consider how effectively the Commitments to Women have been integrated inthe design of the operation. In broad terms, it will assess how well the consideration of gender relations,which can be a major impediment to improved food security, have been mainstreamed into theoperation’s activities.

5. Key Issues and Sub-Issues

The evaluation will address the following issues and sub-issues:

5.1 PRRO Recovery Strategy:

• Are the PRRO activities based on a well-prepared recovery strategy, which clearly identifies theconditions under which the PRRO will move from relief to recovery?

• Is the recovery strategy aimed at creating conditions for a sustainable solution to a protractedsituation?

• When and how was the recovery strategy developed for the PRRO? In what ways does it differ fromthe strategy under its predecessor EMOP? Has it been periodically reviewed or modified over thelife of the operation?

• What resources were devoted to the development of the PRRO recovery strategy and whatmechanisms for consultation, partnership and review were utilized?

• Does the recovery strategy adequately: provide the rationale for operating in the protracted situation;identify and justify the role of food aid; identify target areas and groups; and define assistancemodalities?

• Does the recovery strategy identify the conditions under which the PRRO will move from relief torecovery?

• What risks to the PRRO were foreseen in the recovery strategy (e.g., the resurgence of violence, theinflux of additional returnees/IDPs, loss of donor support, lack of complementary inputs) and wereappropriate contingency plans made?

• Does the recovery strategy include an exit strategy?

5.2 Design of the PRRO:

• To what extent can the PRRO’s overall design (i.e. a recovery strategy linked to specific outputs,outcomes and goals) be considered relevant to the situation on the ground, to the needs ofbeneficiaries, and to WFP’s broader strategic objectives?

• Are the PRRO objectives clear and coherent in terms of collectively contributing to the achievementof the stated goal?

• Is the PRRO in line with the policy “From Crisis to Recovery”? If not, how and why not?• Is the PRRO targeting the appropriate beneficiaries? Does the design include adequate needs

assessment methods for moving to a more targeted approach as required?• Have both the relief and recovery needs of the beneficiaries been accurately identified? Are the

planned outputs and activities relevant and appropriate to the needs of target groups?• Are the food rations and other WFP inputs appropriate in light of the desired outcomes?• Were all relevant external factors affecting the implementation of the PRRO considered at the design

stage?• Has the operational context changed since the design? If so, are the PRRO’s stated objectives still

relevant? Has the PRRO been adapted appropriately?

• Does the design of the PRRO adequately reflect the different roles played by men and women in therelief and recovery process? Are the Commitments to Women appropriately addressed in the design?

• Are the objectives and activities of the PRRO compatible with and complementary to those of otherinterventions currently being implemented by the CO? Have appropriate linkages been made with therecovery or development activities of other WFP interventions?

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

4

• Does the design of the PRRO adequately taken into account the policies and aims of the Governmentand other actors who are addressing similar issues? Are the activities and outputs of the PRROconsistent with the policies of the Government, other donors and organizations?

• How is the PRRO linked with the other multilateral or bilateral development strategy tools (e.g., theUN Country Strategy Framework, the World Bank Poverty Reduction Strategy Programme, etc.), theConsolidated Appeals Process (CAP), and any contingency planning exercises?

5.3 Implementation of the PRRO:• How efficiently is the PRRO producing the intended outputs and reaching the intended target groups

at a reasonable cost?• Are the existing management systems working well?• Have opportunities been identified and pursued for making the transition from relief to recovery

activities (in particular, restoring livelihoods) where appropriate? Has there been an effort tointegrate developmental approaches as early as possible in relief and recovery activities and buildeffective partnerships for this?

5.3.1 Assessment & Targeting• How have food insecurity, vulnerability and beneficiary figures been assessed and subsequently

adjusted as the operation has evolved?° At the country level? (e.g., FAO/WFP Crop and Needs Assessments, WFP/VAM, GIEWS,

FEWS, JFAMs, composite household surveys, camp registration)° At the community level? (e.g., RRA, PRA)° At the household level (e.g., women’s organizations, relief committees)

• Are there mechanisms to signal the need (or provide guidance in the case of resource shortfalls) forfurther targeting, for ration adjustments, or for phasing down?

• On what basis have beneficiaries been selected? Do women play a role in selection?• Is there evidence that the targeted beneficiaries are being reached? Has the PRRO been effective at

making use of UN Frameworks for ensuring access to beneficiaries?• What is the share of female beneficiaries by distribution type?• Is the cost of the assessment and targeting process reasonable? Has the use of an assessment method

for targeting resulted in budgetary savings?• What information on expected funding has been available during the formulation and

implementation of the operation and how has this influenced targeting?• Are the activities and outputs in line with local needs? Are the assets being created relevant for the

beneficiaries in terms of the PRRO’s intended outcomes and goal?• What is the nature of the interaction between IDPs and the local population and how has the

operation addressed and weighed the needs of these groups?

5.3.2 Logistics• Has the pipeline been adequately maintained? In the case of resource shortfalls, have breaks in the

pipeline been signaled beforehand and properly managed?• How have outputs been affected by breaks in the pipeline and resource shortfalls? Has the move

towards targeted approaches under the PRRO assisted with coping with such shortfalls?• Was planning for logistics requirements adequate and what have been the major challenges to the

smooth functioning of the PRRO?

5.3.3 Leakages• What is the extent of losses within the PRRO? What steps are being taken by WFP to avoid or

minimize the level of diversion or taxation – through, for example, selection of delivery channels,supervised distributions and post-distribution monitoring?

5.3.4 Security

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

5

• In the context of the local security situation, have adequate and appropriate measures beenintroduced and adhered to in order to minimize the risk to WFP staff and implementing partnersinvolved in the implementation of the PRRO?

• Are there significant security challenges to the smooth functioning of the PRRO?

5.3.5 Monitoring & Evaluation Systems:• Are appropriate and functioning monitoring and evaluation systems supporting the implementation

of the PRRO?• Were baseline data collected and appropriate indicators identified at the outset for measuring results?• What is the type and frequency of reporting for the operation, including periodic rapid participatory

appraisals? Is the information analyzed and used to make decisions regarding the management of thePRRO?

• What are the constraints to monitoring – such as staffing, funding, access, security, manipulation ofdata – that affect the operation and how does the CO attempt to overcome these?

• Have monitoring criteria been established to signal when to shift activities from “relief” to“recovery”, and from “recovery” to “development” ? If so, are these being applied effectively?

5.3.6 Management• Did the preparation and implementation of the PRRO (compared to the previous EMOP) result in

management changes?• Is there an adequate structure of sub-offices and field offices to manage the operation and maintain

effective operational relationships with implementing partners?• Do sub-offices and field staff have adequate authority to carry out their functions? Are reporting

arrangements within the sub-offices and to the CO clear?• Is appropriate training available to PRRO staff to enhance and maintain their skills, particularly in

line with the programmatic objectives of the PRRO category?

5.4 Results of the PRRO:• To what extent are the expected outcomes of the PRRO being achieved, and how sustainable are

they?• To what extent are the outputs of the different components contributing to achieving the desired

PRRO outcomes?• What have been the actual outcomes for the beneficiaries (men, women, children, IDPs), and what

assets have been gained?• What have been the nutritional outcomes of WFP assistance for returnees, IDPs and other targeted

vulnerable groups? How is this being assessed?• What are the prospects for the sustainability of each main PRRO activity?• Have there been ration reductions or phasing out of “general” food assistance and transitions to

targeted feeding approaches? If so, on what basis and to what effect?• What unexpected outcomes (positive or negative) have occurred?• Has there been any added value to transforming long-standing returnees and IDP operations into a

PRRO in terms of building linkages and improving the likelihood of sustainability?• To what degree has WFP food aid contributed to promoting resettlement and food self-sufficiency, as

applicable?

5.5 Co-ordination and Partnerships:• How successful have co-ordination and partnership arrangements been?• Are the capacities and comparative advantages of potential implementing partners (IPs) assessed by

the Country Office prior to entering into partnerships and agreements?• Are the number and nature of IPs under the PRRO adequate and appropriate for implementing the

range of activities? Is there an appropriate balance between local, national and international IPs?• Has there been a trend towards or away from using local implementing partners?• Are WFP’s policies and priorities (eg, those on gender and the environment) reflected in MOUs with

various IPs?

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

6

• Do agreements with IPs specify the need for gender disaggregated data? Are partners aware ofgender issues, including WFP’s Commitments to Women?

• To what extent have partnership arrangements successfully or negatively contributed to theimplementation of the PRRO?

• Has preparing and implementing the PRRO broadened and improved coordination compared to thepredecessor operations? If so, how?

• What are the mechanisms for coordination with government, donors, UN agencies (OCHA, UNHCR,FAO, IFAD, UNICEF, UNDP, etc.), NGOs, etc.?

• Are appropriate and suitable mechanisms in place to ensure strategic co-ordination at the nationalpolicy level regarding relief and recovery issues?

5.6 Resourcing of the PRRO :• How effective has the PRRO been as a resourcing window in building a more predictable basis for

funding a recovery operation?• Has there been sufficient flexibility to move funds freely between relief and recovery activities as the

situation on the ground requires?

5.6.1 Budgets and financial resources for preparation and implementation• Has the preparation of the PRRO resulted in a change in the ratio of dollars spent per ton of

commodities delivered compared to the previous EMOP?• Does the Country Office have the resources required – staff and cash – to prepare and implement the

PRRO as foreseen in the “Guidelines for Preparation of a PRRO”, including staff or consultants witha “combination of development and emergency experience, and strong background in planning,design and strategy formulation”?10

• Is the PRRO budget appropriate in relation to its objectives/activities, and what factors (such as COsize, additional staff requirements for PRRO implementation) have affected individual budget items,and particularly DSC?

• What has been the overall resourcing result of presenting and tracking the PRRO budget bycomponent elements (relief or recovery), including earmarking resources to particular components?

5.6.2 Flexibility of PRRO budget and shifts in activities and resources• Have the contingency mechanisms intended to deal with setbacks, reversals and new

emergency/disaster outbreaks – such as PRRO budget revisions – been employed?• To what extent has the Country Director utilized his/her authority to transfer funds between

components and geographic areas?

5.6.3 Predictability and regularity of resources and impact on PRRO• To what extent have the resourcing requirements for the operation been met and how has the CO

managed shortfalls? How predictably and regularly have resources been supplied to the PRRO?• What have been the major resource constraints for the PRRO and have they changed as a result of the

transformation of the operation from an EMOP to a PRRO?• Has transformation to a PRRO resulted in longer-term (more than 1 year) financial commitments to

the operation?• How successfully has the PRRO resourced its non-food inputs and what, if any, have been the

constraints?• What role, if any, has resource availability and predictability played in building relief-recovery

linkages?

5.6.4 Donor perception of the PRRO and advocacy with donors and partners• What has been the extent and nature of Country Office advocacy for the PRRO with donors and other

partners inthe field? How do local donor representatives view the PRRO?

10 Guidelines for the Preparation of a PRRO, section 3.3; the Guidelines suggest the establishment of a “planner post” forpreparing a PRRO.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

7

• What effect does having multiple and varied components within the PRRO (e.g., relief feeding andrecovery FFW) have on WFP’s ability to successfully obtain donor support?

5.7 Cross-Cutting Issues:

5.7.1 Meeting Commitments to Women:• Are appropriate opportunities being sought during implementation to improve the status of women

and to address their strategic needs? How is WFP trying to improve the status of women and create amore gender equitable society?

• Has adequate effort been made to mainstream gender considerations?Are there any changes required in a future phase to ensure better compatibility with the Commitmentsto Women?

5.7.2 Environment:• Have environmental concerns been adequately addressed within the PRRO, particularly with regard

to energy-related issues (e.g., cooking time) and the placement of IDP/returnees camps?• Effect of Relief Activities: What effect have the camp sites had on the environment? What effect have

rations and cooking requirements had?• Effect of recovery activities: What has been the environmental impact of recovery activities within

the PRRO? Are the PRRO activities helping to restore the natural habitat through rural infrastructureand reforestation schemes?

6. Notes on Methodology

6.1 Stages of the evaluation

The evaluation will be divided into three phases:

Phase I – Preparation and Desk Review (3 days):

Prior to the in-country mission, the team will review all relevant background documentation. The teamleader will join the OEDE evaluation officer at WFP in Rome for a one-day briefing with ODR priordeparture.

In order to help structure the evaluation and ensure a systematic examination of the project’s effects, theevaluation team – with the assistance of the responsible OEDE officer - may wish to prepare a “mock”logical framework of the PRRO prior to arrival in-country. The logframe should first draw a linkbetween the PRRO objectives and the situation analysis as presented in the project document. Objectivesshould then be linked to each PRRO activity and its expected outputs and outcomes. Finally, theevaluation team should develop an explicit strategy for gathering information on the specific outputs andoutcomes (e.g., statistical review at WFP Country Office, beneficiary interviews, etc.). A summary chartof the logframe may be used to structure the initial briefing with the Country Office and amended basedon Country Office input.

Prior to the departure of the mission, the WFP Country Office and OEDE should assemble keyinformation. This includes:

Basic Documents to be reviewed:

• WFP PRRO policy document: “From Crisis to Recovery” (WFP/EB.A/98/4-A)• WFP, Guidelines for Preparation of a PRRO• PRRO project document• WFP/RE resource summary table for the PRRO• Preceding EMOP or PRO document(s)

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

8

• Country Strategy Outline and Country Programme (if exists)• Programme Design Manual (2000)• Cost Containment Study (2001)• Emergency Needs Assessment Guidelines (1999)• Supplementary Feeding Operational Guidelines (1998)• Food aid in Emergencies (red book) Operational Procedures (1993)• Food and Nutrition Handbook (2000)• Gender Guidelines (2000)• Gender Checklist (1999)• Environmental Review Guidelines (1999)• ALNAP Annual Review 2001. Humanitarian Action: Learning from Evaluation• Previous evaluation summaries and full reports• Country or operation case studies• Documentation on UN CSF, CCA, UNDAF• WFP/OEDE thematic evaluation “Recurring Challenges in the Provision of Food Assistance in

Complex Emergencies”• Most recent JFAM mission reports (for refugee operations) and other assessment reports• Most recent WFP/FAO Food & Crop Assessment reports• Most recent Consolidated Appeal Process (CAP) documents• Any evaluations/reviews undertaken by implementing partners during the last 2 years• Mid-term review of the WFP Commitments to Women (if relevant)• Country Gender Action Plan

Key information to be prepared by the WFP Country Office

• Basic country data• Basic data on the country’s food balance, including its import and food aid requirement and local

commodity prices• Completed Part 1 and 2 of the new Country Office Report (COR), see attachments• Resource table for the PRRO with stock balance, information on losses and regularity of deliveries• Management structure of PRRO: list of staff, equipment, etc. for the PRRO• Outputs achieved vs. planned for PRRO (disaggregated)• Relevant comparative data (budgetary, output) for PRRO and its predecessor EMOP• Socio-economic status of beneficiaries broken down by gender, beneficiary category, location• Local monetary value of the ration/per beneficiary category and relation to household income• Description of logistics and security arrangements (maximum 5 pages each)

Prior to arrival of the evaluation team, the Office of Evaluation (OEDE) should forward the TORs to theCountry Office. A small task force of key stakeholders (composition to be determined by the CountryOffice) may be established to comment on the TORs and to host the debriefing at the end of the mission.

Phase II – The in-country evaluation (14 days):

To the extent possible, the evaluation team should meet with all relevant stakeholders, includingbeneficiaries, local and national government, key implementing partners and other development agenciesinvolved in the UNDAF and with any of WFP’s programmes.

Data collection will take place both in the offices of key stakeholders in the capital and in the field wherethe activities of the PRRO can be visited. Rapid Rural Appraisal (RRA) techniques will be usedwhenever possible. The mission leader, together with the Country Office, will determine the optimumbalance between mission time spent in the field and in the capital.

Key informant interviews to be carried out by the Team may be divided into different groups:

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

9

Group one: WFP full time and contract staff working on the PRRO;

Group two: Key institutional partners/actors involved at the national level in programming relief andrecovery assistance. These may include:• relevant government ministries,• relevant UN agencies (OCHA, UNHCR, UNICEF, FAO, WHO, etc.)• local offices of World Bank and regional development bank• bilateral donors with a significant presence in relief and recovery assistance,• NGOs with a mandate for relief and recovery assistance and a strong presence and reputation in food

aid and/or food security

Group three: Institutional partners engaged in WFP activities• national, provincial and local offices of agencies implementing PRRO activities• staff of national and international NGOs involved in the delivery of WFP assistance within thePRRO• staff of human service agencies supported within the PRRO such as nutrition rehabilitation units,MCH centres, etc.

Group Four: Participants and Beneficiaries

The mission members should visit households, and meet with key informants and groups which representeach category of PRRO beneficiary. Attempts should also be made by the team to make use of rapid ruralappraisal techniques (RRA) as a means of gathering qualitative data from beneficiaries.

Phase III – Report writing (5 working days team members, 10 working days team leader).

The team leader is responsible for coordinating data collection activities, written inputs by team membersand for preparing the Aide Memoire, evaluation summary and final report.

6.2 Timetable

Include in this section a tentative, detailed itinerary for the PRRO evaluation. An example:

Review/finalize evaluation TORs October 10, 2001 Planning/desk review Prior to 24 October 2001 (done by

consultants independently – 3 days perconsultant)

Briefing at WFP Rome 24 October 2001 Travel to Baku 25 October 2001 In Azerbaijan 26 October – 8 November Debriefing of Country Office 7 or 8 November Travel to Rome 9 November 2001 Deadline for Evaluation Summary 23 November 2001 Deadline for Final Evaluation Report 30 November 2001

6.3 Organization of the Mission

Role of the Team Leader: Will finalize the methodology and key issues for the evaluation. This will bedone in consultation with the OEDE Evaluation Officer. He/she will identify the role and input of eachteam member, including individual requirements for the Aide Memoire, Evaluation Summary and FinalReport. With assistance from the WFP Evaluation Officer, the team leader will define any preparatorywork required by the CO and/or local consultants prior to the mission (at least 4 weeks notice should be

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

10

given to the Country Office). The team leader will assume overall responsibility for the mission, andwill synthesize the inputs from all sources in order to produce the necessary outputs.

The Team leader is responsible for producing the following outputs :• an Aide Mémoire for presenting the mission’s early findings and recommendations at the final

debriefing in Azerbaijan;• a Final Evaluation Report; and• an Evaluation Summary Report for presentation to the Executive Board.

The team leader will present the team’s findings at all debriefings and will ensure that all deadlines aremet for the above outputs.

Role of the other team members : To provide technical expertise according to individual skill sets, and toprovide written inputs to the Aide Memoire, Evaluation Summary and Final Report under the guidanceof the Team Leader and WFP Evaluation Officer. The second international team member should alsoprovide to OEDE, in writing, structured and constructive feedback on module E of the Monitoring andEvaluation Guidelines prepared by OEDE.

Role of the WFP Evaluation Officer : He will provide support to the overall evaluation exercise asnecessary, which includes liaising between team members, relevant areas of WFP headquarters and thecountry office. He will also ensure compliance with the intended thrust of the evaluation, and that WFPHQ and the CO provide the necessary logistical support.

Role of the Azerbaijan Country Office : To advise on the timing of the evaluation to ensure that theevaluation outputs are available for the preparation of the next phase of the PRRO. To ensure that allnecessary documents required to plan the evaluation and undertake the desk review are provided in atimely manner. To assist with the identification and hiring of local consultants as required. To ensurethat any necessary preparatory work is undertaken in-country prior to the arrival of the evaluation team,and to facilitate the work of the team while in-country. Prepare and organize the mission in-countryitinerary, and organize the PRRO evaluation briefing/debriefing.

6.4 Products of the Evaluation

• Aide Mémoire for debriefing the Country and Regional Offices and HQ (maximum 10 pages)deadline : 6 November 2001

• Final Evaluation Report and Recommendation Tracking Matrixdeadline : 30 November 2001

• Evaluation Summary Report (maximum 5000 words) deadline : 23 November 2001

All reports will be prepared in English and must be written in conformity with the outlines in Annexes 1and 2. Draft versions of the Evaluation Summary Report and Final Report will be reviewed by the OEDEEvaluation Officer prior to being finalized.

The Evaluation Summary Report, technical reports and Final Evaluation Report must be submitted inhardcopy accompanied by an electronic version. If applicable, annexes should also be made available inWFP standard software (i.e., Microsoft package). For ease of processing, the Summary Report should besubmitted as plain, unformatted text only (no paragraph numbering, limited bold, underline, etc.).

The mission is fully responsible for its independent full report, which may not necessarily reflect theviews of WFP.

The evaluation shall be conducted in conformity with these terms of reference and under the overallguidance of OEDE.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

11

The Annexes to these Terms of Reference provide the evaluation team with some tools to beused in carrying out and reporting on the PRRO evaluation.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

Annex 2

Evaluation methods

The Mission would like to thank all the staff of WFP Azerbaijan and staff of WVI for their great help infacilitating the carrying out the following activities.

Positionality

The three person Mission were all Europeans, two male and one female. All had previous experience ofwork with IDPs. One is a full time member of WFP OEDE, one has had considerable experience ofworking for WFP, is an expert in social issues and in particular gender issues. One has had ten yearsexperience of evaluating WFP activities in natural and humanitarian disasters and post–disastersituations. One team member is a native of Azerbaijan, thus was able to speak directly with allbeneficiaries and other interviewees and was able to interpret culturally relevant phenomena for otherteam members. This member, an expert in gender, was also able to explain social, economic and politicalaspects specific to Azerbaijan.

Methods of work

Normally the Mission worked as a group, particularly for interviews with staff of agencies other thanWFP, allowing discussion among the Mission of their shared experiences and impressions. On mostvisits to the field the Mission split so as to allow contact with more beneficiaries. Each Mission membercarried out a number of interviews with individual IDPs, representatives of refugee committees, localpoliticians and ad hoc groups of beneficiaries. In total the Mission interviewed about 100 beneficiaries,either as individuals or in groups. Generally interviews with beneficiaries were unstructured and stronglyinfluenced by what the IDPs wished to say rather than by an outsider’s checklist, though inevitably it wasnecessary to direct interviews in order to cross-check information. It was difficult to avoid groupinterviews unless interviews were in the privacy of a house and even there it was unusual for only onemember of a household to contribute. Group interviews, however, do allow the possibility of cross-checking the information with knowledgeable people, and help in gaining an impression of the generalapplicability of the information or opinions being given. The main locations for field interviews were atdistribution sites, in IP offices but most were in or around IDP houses of several different types. In fact,the Mission were able to visit several examples of most types of living accommodation and were able tosee others. The mission visited all warehouses and EDP offices and witnessed three distributions. Travelto and from the field allowed opportunities for long discussions with WFP and WVI staff.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

Annex 3 – Checklist for Meeting Commitments to Women and Mainstreaming a Gender Perspective

Level of PRRO Coherence WithCommitments to Women andGender Mainstreaming PolicyEssential Elements of Commitments to

Women and Gender Mainstreaming PolicyDetailed Observations

VeryHigh

High Low VeryLow

Commitment I: Provide Direct Access to Appropriate Food for Women

♦ Does the PRRO make a real effort to getfood into the hands of women, e.g.through women’s ration cards?

WFP monitors and controls distribution of food assistance to women throughdirect implementation and WVI. Though WFP does not stipulate thatdistribution should normally be to women, in fact almost half of families arefemale-headed. Some 60 per cent of recipients of rations are women if they livenear the distribution points though this falls to 30-40 per cent if they live at agreater distance. The Food Economy Survey of 1999 suggests that 96 per centof food is managed by women and the WFP Gender Survey of 2001 suggeststhat 68 per cent of food was controlled by women, 17 per cent by women andmen together and only 15 per cent by men alone.

*

♦ Do the PRRO activities addressmicronutrient deficiencies amongstwomen and children?

The WFP salt ration is iodized, to reduce the incidence of goitre throughout thepopulation. Vegetable oil is enriched with vitamin A and wheat flour isenriched with iron, to reduce the high levels of anaemia among women. Noprovision is made for the particular nutritional needs of pregnant and nursingwomen.

*

♦ Do the PRRO activities consider localcooking and eating habits?

The CO has investigated local cooking methods in response to the suggestionthat the green peas supplied were not fully acceptable to beneficiaries. Inresponse, WFP have provided yellow split peas and have consulted localwomen on cooking methods to produce an information sheet for beneficiaries.Some beneficiaries have said that they would prefer butter to vegetable oilthough the latter is generally acceptable. Beneficiaries said that the wheat flourration has been suitable for local cooking methods.

*

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

2

Level of PRRO Coherence WithCommitments to Women andGender Mainstreaming PolicyEssential Elements of Commitments to

Women and Gender Mainstreaming PolicyDetailed Observations

VeryHigh

High Low VeryLow

♦ Have women been consulted indetermining the food basket?

WFP and WVI monitors have interviewed women on the food basket and theywere able to comment on the food basket components. The commentsmentioned in the previous question were elicited in this manner. Some womenrequested the provision of rice as part of the ration and extra sugar forpreserving fruit. The mission saw that sugar was used in this way. WFP believethat with reducing resources, the fine tuning of the diet is a lower priority.

*

♦ Are female-headed households givenspecial attention because of their greaterpoverty and time constraints?

WFP give no special attention to female-headed households on account of theirpoverty or time constraints. WVI and WFP collect information on gender andother aspects of beneficiary families and monitors are aware of the issues ofpoverty and time limitation, but the focus on FHHs is not motivated by thesetwo factors per se.

*

♦ Does the PRRO make an effort to reducethe security and/or health risks womenface when collecting food?

Where feasible WFP selects distribution sites providing protection fromadverse weather, for example under an open-sided shelter. Compared withother WFP operations there are few security risks for women. By trying tolocate distribution sites near to concentrations of population (a balancebetween effectiveness and cost of delivery) WFP reduces the distance thatwomen need to transport rations. In fact many beneficiaries were seen to hiretransport for delivery from the DP to homes. It was noted that at DPs someWVI staff ensured that women did not need to lift heavy weights.

*

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

3

Level of PRRO Coherence WithCommitments to Women andGender Mainstreaming PolicyEssential Elements of Commitments to

Women and Gender Mainstreaming PolicyDetailed Observations

VeryHigh

High Low VeryLow

Commitment II: Take Measures to Ensure Women’s Equal Access to and Full Participation in Power Structures and Decision-Making

♦ Does the PRRO address women’sstrategic needs, i.e., use an approach thatchallenges traditional gender roles andempowers women? Describe how.

The PRRO and Gender Action Plan place greater emphasis on the practicalneeds of women than on their strategic needs. The image that WFP and WVIpresent to the beneficiaries is gender balanced, with women in positions ofauthority, as team leaders and as distributors. Within both organizations thereis gender balance. To this extent, the PRRO does in fact implicitly, though notexplicitly, address strategic needs. WFP-AZE was assessed as excellent inrelation to gender in the 2000 Gender Action Plan Review.

*

♦ Does it address gender relations? Does itbring men into the dialogue around theissues of women’s status?

There is no specific organized activity. *

Commitment III: Take Positive Action to Facilitate Women’s Equal Access to Resources, Employment, Markets and Trade

♦ Are PRRO resources deliberately targetedto women and girls where there is a biggender gap, i.e. of 25%?1 What is done?

Because of the heritage of the FSU social relations, there is no large gender gapin Azerbaijan. The juridical rights of men and women are equal. In practiceWFP treats men and women beneficiaries equally.

Notapplicable

♦ Does the PRRO have incentive programsto address the gender gap in primaryeducation? What are they?

The only area of education where WFP resources are involved is the pre-school/kindergarten work with “war traumatized” children, who are in fact thechildren born and growing up in the restrictive environment of camps and withadults who may reasonably be described as war-traumatised. The children aregender balanced. All care providers are female. The Mission would encouragethe participation of WFP in school feeding and understand that WFP wouldwish to be so involved.

*

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

4

Level of PRRO Coherence WithCommitments to Women andGender Mainstreaming PolicyEssential Elements of Commitments to

Women and Gender Mainstreaming PolicyDetailed Observations

VeryHigh

High Low VeryLow

♦ Do women participate in FFW? Aslaborers or also as decision-makers? Dothey control the assets created?

All of the 254 care providers in the pre-school activities are women, as arehead teachers. As part of their social-reproductive realm, the Mission is of theopinion that women of the IDP communities will gain considerably from theassets (that is the improved/maintained psychological state of the children).Control is hardly an issue.

*

♦ Is there any opportunity in the PRRO forwomen to learn new skills through FFTfor greater development sustainability?

The joint UNCEF/WFP pre-school initiative has involved training of women inaspects of child development and child care. This is undoubtedly an asset forthe short, medium and long-term futures.

*

♦ Does the PRRO engage in advocacy onbehalf of women? For gender equity? Toleverage resources for partnership work?

Not specifically, though the frequent mention of women’s interest impliesadvocacy, or at least positive discrimination in favor of women.

*

Commitment IV: Generate and Disseminate Gender-Disaggregated Data for Planning and Evaluation

♦ Are the M&E systems used in the PRROsensitive to gender? Explain how.

Gender has been the subject of a specific WFP investigation during 2001, andWFP, in association with partners, has gathered gender-sensitive data inhousehold surveys. Detailed reporting of distributions and in post-distributionmonitoring routinely provide considerable data on gender, age and some onvulnerability. Field monitors are gender balanced.

*

♦ Is qualitative information sensitive togender also collected?

At distributions, the Mission witnessed several occasions on whichbeneficiaries raised issues relating to gender with WFP and WVI staff: it wasclear that this was not a rare event and the response of the staff suggested thatthey were aware that the topics discussed were of concern to femalebeneficiaries. Monitors gather such qualitative information in post-distributionmonitoring. Field monitoring is gender balanced so that women and men haveopportunities to make their opinions/feelings known.

*

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

5

Level of PRRO Coherence WithCommitments to Women andGender Mainstreaming PolicyEssential Elements of Commitments to

Women and Gender Mainstreaming PolicyDetailed Observations

VeryHigh

High Low VeryLow

♦ Does the PRRO look at inputs, outputsoutcomes and impact from a genderperspective?

The PRRO document considers gender explicitly only in relation to women.For instance six of the periodic monitoring indicators specify women: nonespecifies men per se. Commitments to women are included throughout thedocument, with inputs in capacity building and outcomes relating, for example,to minimum levels of women’s participation in activities and benefiting fromdistributions.

*

Commitment V: Improve Accountability of Actions Taken to Meet the Commitments

♦ Are WFP staff managing the PRRO heldaccountable for meeting the Commitmentsto Women and mainstreaming gender?How?

The fact that WFP has two gender focal points implies that the issue is takenseriously. WVI has detailed reference to women in its LoA with WFP.Reporting specifies gender disaggregation. At field level, the implementationof the PRRO by WVI and WFP presents a gender-balanced face to thebeneficiaries.

*

♦ Is the Gender Focal Point given sufficientauthority to influence decision makingwith regard to the PRRO? Support?

WFP, at present has two gender focal points. One of these, the newly-arrived(female) Emergency Officer is undertaking an increasing role in this respect.The two gender focal points are sufficiently senior to influence decisionmaking and activities in relation to this issue.

*

♦ Are implementing partners heldaccountable for meeting the Commitmentsto Women and mainstreaming gender?How?

The Letter of Agreement between WFP and WVI specifies in considerabledetail commitments to women in accord with WFP principles. What is inpractice, daily reporting by WVI includes gender-disaggregated data. In effectWVI follows the same approaches in relation to gender, as does WFP.

*

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

Annex 4

Mission Itinerary and Persons Interviewed

Date Day Activities

24 Oct Wed Rome HQ, initial briefing and meeting of two members of Mission. InterviewBeatrice Bonnevaux OD/WFP (Regional Bureau Programme Officer).

25 Oct Thurs Arrival Baku.26 Oct Fri Full Mission meet for initial discussion.

Briefing WFP CO. Meeting Cherif Zaher (WFP Representative), Hae-Won Park(Emergency Officer), Fuad Guseynov (Programme Analyst), Tahir Agayev(Senior Field Monitor), Rasim Hajiyev (Administration and Finance Officer).Interview Akif Saatcioglu, UNICEF Representative.Interview with and briefing by Ali Hasanov ( Deputy Prime Minister, ChairmanSCRIDP, Head of Republican Commission on International HumanitarianAssistance), Sadigov Gurban (Acting Chief of Department for Problems ofRefugees and IDPs Migration and Work with International HumanitarianOrganizations) and other members SCRIDP.

27 Oct Sat Full Mission field visit, Imishli; briefing by and interview with VidadiAllahverdi oglu Quliyev (Mayor of Imishli); visit WFP distribution site;interviews with beneficiaries; visits to beneficiary houses.

28 Oct Sun Mission discussions and perusal of documents.29 Oct Mon Mission discussions with CO staff, perusal of documents: interview Didier Laye

(UNHCR Representative); interview Ian Curtis (Country Director WVI), JunusDavid, (Senior Commodities Manager WVI), Seifu Tirfie (CommoditiesManager) and senior members WVI country team.

30 Oct Tues Mission discussions with CO staff; meeting with USAID staff: William McKinney (Country Director) and Khalid Hasan Khan; meeting, Mercy CorpsWilliam Holbrooke (Country Director) and staff of Mercy Corps.

31 Oct Wed One Mission member field visit to Masalli WFP warehouse and field office,interviews WFP staff then field visit to Saatli.Two Mission members field visit to Saatli. Field visit IDPs in rail wagons,interviews with beneficiaries: Yolchu Aliev (Representative of IDPs) and localrepresentatives: Gachay Shabazov (Deputy mayor), Nasir Gadirov (Head ofIDPs Department); visit Imishli WVI offices and warehouses; interviews JunusDavid and WVI staff. Discussions with accompanying WFP staff.

1 Nov Thurs Visit to pre-school programme at ECHO Camp 1, interviews head teacherTuygen Khudiyeva and care providers.Field visit Kurdamir WFP/WVI distribution sites; interviews with IDPs andWVI implementation staff, inspection of IDP housing and water accessing; fieldvisit Agjabadi IDP accommodation in dugouts; interviews with beneficiaries;interviews with Junus David (WVI), Seifu Tirfie (WVI), WVI office staff , WVIfield staff and accompanying WFP staff.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

2

2 Nov Fri Mingechevir. Field visit WFP/WVI distribution site. Interviews: WVI fieldstaff, monitors and beneficiaries.Field visit WFP/WVI warehouses and office, interviews WVI warehouse staff.Visit IDP living accommodation in unfinished building. Interviews with IDPsand IDP representatives. Visit Market at Mingechevir to check whether sale ofWFP commodities.

3 Nov Sat Return Baku. Report preparation.4 Nov Sun Preparation and writing of Aide Memoire.

5 Nov Mon Interview Marco Borsotti, UN Resident Coordinator. Discussions with CO staff.Report preparation.

6 Nov Tues Report preparation, discussions CO staff; debriefing Deputy Prime Minister.7 Nov Wed Report preparation; discussions with Formulation Team (Darlene Bisson

(Deputy Director, Regional Bureau for Eastern Europe) and Jon Bennett(Independent Consultant).

8 Nov Thurs Debriefing CO staff, further discussions with Formulation Team and CO staff.Report preparation.

9 Nov Fri Departure from Baku.

In total some 100 IDPs were interviewed either individually or as members of groups during the fourdays of field visits.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

1

Annex 5

Documents Reviewed and Consulted

• ALNAP (2001) Annual review: Humanitarian Action: Learning from Evaluation, ALNAP, London.

• Deng FA (1998) Letter to C Bertini reporting on IDPs in Azerbaijan. Headed Brookings Institution.

• FAO (2001) Assessment of Drought Damage and Priority needs for Assistance, Mission Report 6-15January 2001.

• FAO/WFP (1999) Special Report Crop and Food Supply Assessment Mission to Azerbaijan,FAO/WFP, Rome.

• GoA (1998) Plan for IDPs, Baku.

• GoA (2000) Strategic Directions Towards Development, Baku.

• GoA (2000) The Multiple Indicators Cluster Survey, Baku.

• Mercy Corps (2000) Annual Report, Portland.

• Mercy Corps (2001) Azerbaijan Humanitarian Assistance Program, Baku.

• Mercy Corps (2001) Economic Opportunities Bulletin, Baku.

• Morsy SA (1999) WFP Commitments to Women: Mid-term Review of Implementation, Case StudyAzerbaijan, Rome.

• OECD (1999) Guidance for Evaluating Humanitarian Assistance in Complex Emergencies. Paris.

• Schelhas B (1996) Evaluation of the WFP Emergency Relief Operation in the Caucasus (23 may-11June 1996) Consultant Report.

• Sphere Project (2000) Humanitarian Charter and Minimum Standards in Disaster Response,Geneva.

• UNDP (1998) Azerbaijan Human Development Report, Baku.

• UNDP/WFP (1999) Refugees and Internally Displaced Persons in Azerbaijan (statistical data) Baku.

• UNDP (2000) Azerbaijan Human Development Report. Baku.

• UN Azerbaijan (2000) Annual report of the UN Resident Coordinator, Baku.

• UN Country Team (2001) Common Country Assessment Azerbaijan, (Final Draft) Baku.

• WFP (1993) Food Aid in Emergencies, Rome.

• WFP (1997) Summary Evaluation Report on WFP Assisted Emergency Relief Operations in theCaucasus, Including the CLAU Operation, WFP Rome.

• WFP (1998) Emergency Food Assistance to Internally Displaced Persons and Other VulnerableGroups in Azerbaijan 5302.04 (EMOP).

• WFP (1998) PRRO Policy Document: From Crisis to Recovery (WFP/EB.A/98/4-A).

• WFP (1999) Donor Assessment Mission Report.

• WFP (1999) Emergency Needs Assessment Guidelines, Rome.

• WFP (1999) Letter of Understanding between WFP and GoA, Baku.

• WFP (1999) Guidelines for the Preparation of a PRRO, WFP, Rome.

Full Report of the evaluation of AZERBAIJAN PRRO 6121.00

2

• WFP (1999) Letter of Agreement between ADRA International and World Food.

• WFP (1999) Letter of Agreement between Relief International and World Food Programme(Khanlar District).

• WFP (1999) Letter of Agreement between Relief International and World Food Programme (Yevlakand Barda District).

• WFP (1999) Midterm Review of Implementation Commitments to Women, Baku.

• WFP (1999) Policy Issues: Enabling Development (WFP/EB.A/99/4-A), Rome.

• WFP (1999) Protracted Relief and Recovery Operation; Azerbaijan 6121.00, WFP/EB.2/99/5-B/2,Rome.

• WFP (1999) Report on Joint WFP-Donor Mission to Armenia, Georgia and Azerbaijan, Rome.

• WFP (1999) WFP Assistance to IDPs Country Case Study on Internal Displacement, Azerbaijan,WFP Rome.

• WFP (2000) Consultation on Humanitarian Issues, Situations of Displacement: Issues andExperiences, Background Paper 1, WFP Rome.

• WFP (2000) Environmental Review Guidelines, Rome.

• WFP (2000) Food and Nutrition Handbook Rome.

• WFP (2000) Gender Action Plan for Azerbaijan 2000-2001, WFP Azerbaijan.

• WFP (2000) Gender Guidelines, WFP, Rome.

• WFP ( 2001) Gender Survey in Lieu of Food Security, Baku.

• WFP (2000) Situations of Displacement: Issues and Experiences, WFP’s IDP Review: WFP-Reaching People in Situations of Displacement, Consultation on Humanitarian Issues, WFP, Rome.

• WFP (2001) Contingency Plan for Azerbaijan, Baku.

• WFP (2001) Household Food Economy Survey among the IDPs in Azerbaijan, Baku.

• WFP (2001) Letter of Agreement between WFP and WVI. Baku.

• WFP (2001) Mid-term Evaluation of WFP’s Commitments to Women, Baku.

• WFP (2001) Terms of Reference PRRO Formulation Mission, November 2001, WFP, Rome.

• WFP/RE (2001) Resource Summary Table for PRRO 6120.00.

• WFP/WHO (1996) Health and Nutrition Survey of IDP and Resident Population of Azerbaijan.Baku.

• World Bank (1997) Azerbaijan Poverty Assessment.

• World Vision (1998) World Vision Azerbaijan Food Security Survey, Horadiz, Fizuli District, WVI,Baku.

• World Vision (1998) World Vision Azerbaijan Household Food Security Survey of IDPs, WVI, Baku.

• World Vision (2001) Results of Cross-checking Exercise for Food and Micro-finance Programmes,Baku.

• World Vision (2001) Study of Food Distribution Tools of Measurement in Emergency Relief andRehabilitation Programmes.

1

MAP 1 – WFP Areas of Operation in Azerbaijan

1

MAP 2 – WFP Activities in Azerbaijan

1

MAP 3 - WFP/UNICEF Pre-School Centres