167
April 2013 Evaluation report Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS) Project- GCP /GLO/212/GFF Office of Evaluation Food and Agriculture Organization of the United Nations

Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

April 2013

Evaluation report

Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS) Project- GCP /GLO/212/GFF

Office of Evaluation

Food and Agriculture Organization of the United

Nations

Page 2: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

Food and Agriculture Organization of the United Nations

Office of Evaluation (OED)

This report is available in electronic format at: http://www.fao.org/evaluation

The designations employed and the presentation of material in this information product do not imply the

expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United

Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its

authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or

products of manufacturers, whether or not these have been patented, does not imply that these have been

endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.

The views expressed in this information product are those of the author(s) and do not necessarily reflect the

views or policies of FAO.

© FAO 2013

FAO encourages the use, reproduction and dissemination of material in this information product. Except where

otherwise indicated, material may be copied, downloaded and printed for private study, research and teaching

purposes, or for use in non-commercial products or services, provided that appropriate acknowledgement of

FAO as the source and copyright holder is given and that FAO’s endorsement of users’ views, products or

services is not implied in any way.

All requests for translation and adaptation rights, and for resale and other commercial use rights should be

made via www.fao.org/contact-us/licence-request or addressed to [email protected].

For further information on this report, please contact:

Director, OED

Viale delle Terme di Caracalla 1, 00153

Rome, Italy

Email: [email protected]

Page 3: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

iii

Acknowledgements

The evaluation team likes to extend its thanks to the many individuals who spent time answering questions and exchanging information about the GIAHS initiative. While most people answered evaluation questions during face to face meetings, a fair number of people were also approached by phone to respond to specific evaluation queries. To name them all here would certainly take up too much space and the list of people met will have to substitute for this purpose. Special mention of a few individuals is, however, in place as their role was central to the evaluation exercise. Special thanks therefore to Dr. Mary Jane de la Cruz, global project technical officer, who has been the main interlocutor for the evaluation on the side of the project. Thanks to her patience and sincere efforts to shed light on a project period stretching close to 10 years1, the evaluation feels that it has been better able to grasp the historical perspective. Special thanks are also due to Dr. Parviz Koohafkan (Global Project Coordinator). Despite his busy schedule as Director NRL, he has managed to spend substantial time with the evaluation team for conceptual exchanges regarding GIAHS. This was especially important as it shed light on GIAHS conceptual origins and the possible roadmap along which it may further evolve. Lastly, the evaluation wants to thank ADG NR, Mr. Alexander Mueller, for his time, management views and personal insight extended over a total of 4 meetings throughout the course of the evaluation.

Evaluation Team

The MTE team was composed of an international Team Leader and 3 regional/national consultants recruited for 4 out of 6 countries in which the project operates directly. The evaluation work for the Philippines and Chile did not involve national consultant inputs. Mr. Ali Ferchichi took responsibility for the national evaluation inputs covering Tunisia and Algeria. Mr. Mario Mancini acted as the national consultant for Peru, and Mr. Yexu Wang shouldered responsibility for the China national evaluation. The international team leader position was filled by Bart Dominicus. He acted as the liaison between the national consultants to ensure conformity of evaluation approach and to facilitate exchange of initial evaluation findings among countries. Country visits were conducted in all 6 participating nations under the GEF GIAHS project. Only Algeria was not covered by a visit of the evaluation Team Leader (TL). The TL maintains overall responsibility for preparation of the global consolidated evaluation report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried out by the national consultants is presented in the form of self-standing and independent documents.

1

When including the project PDF-A and PDF-B preparation phases.

Page 4: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

iv

Acronyms

ADG Assistant Director General AM Adaptive Management BH Budget Holder C&R Conclusions and Recommendations CBD Convention on Bio-Diversity CDD Community Driven Development CEO Chief Executive Officer CMP Conservation Measures Partnership COAG Committee on Agriculture COP Conference of Parties CPF Country Programme Framework DAC Development Assistance Committee DENR Department of Environment and Natural Resources FAOR FAO Representation / Representative FO Functional Objective FSP Full Scale Project GDP Gross Domestic Product GEF Global Environmental Facility GIAHS Globally Important Agricultural Heritage Systems HQ Head Quarters HR Human Resources LEGA (FAO) Legal Office / Branch LF Logical Framework M&E Monitoring and Evaluation MoA Ministry of Agriculture MoE Ministry of Environment MTE Mid Term Evaluation NGO Non-Governmental Organization NIAHS Nationally Important Agricultural Heritage System NR Natural Resource NRL Natural Resources Land (Division) OED Office of Evaluation OP Operational Programme PDF Project Development Fiche PES Payment for Ecosystem Services PM&E Planning Monitoring and Evaluation PRODOC Project Document RAF Resource Allocation Framework SAC Scientific Advisory Committee SC Steering Committee SO Strategic Objective SWOT Strength Weakness Opportunities Threats TL Team Leader UN United Nations UNDP United Nations Development Programme UNEG United Nations Evaluation Group USD United States Dollar

Page 5: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

v

Table of Contents Acknowledgements ............................................................................................................. iii Evaluation Team ................................................................................................................ iii Acronyms ............................................................................................................................ iv

Annexes ............................................................................................................................... vi Executive Summary ........................................................................................................... vii 1 Introduction ................................................................................................................. 1

1.1 Evaluation background .........................................................................................1

1.2 Structure of the report ...........................................................................................2

1.3 Evaluation purpose ...............................................................................................2

1.4 Evaluation scope ...................................................................................................3

1.5 Evaluation methodology .......................................................................................3

1.6 Constraints to evaluation .......................................................................................6

2 Project context and description ................................................................................... 6

2.1 Origin and context ................................................................................................7

2.2 Project description ................................................................................................9

2.2.1 Stated project outcomes ...................................................................................... 10

3 Analysis of project relevance, concept and design .................................................... 13

3.1 Relevance of project objectives ........................................................................... 13

3.2 Project concept and design .................................................................................. 13

3.2.1 Analysis of project concept and logic .................................................................. 13

3.2.2 Adequacy of timeframe and resources ................................................................. 14

3.2.3 Selection and choice of initially designated pilot sites ......................................... 14

3.2.4 Choice of project national counterparts .............................................................. 15

3.2.5 Stakeholders’ participation ................................................................................. 15

4 Analysis of project implementation........................................................................... 16

4.1 Budget and expenditures ..................................................................................... 16

4.2 Project management, structure and steering mechanisms ..................................... 20

4.3 Technical backstopping....................................................................................... 22

4.4 Design of M&E system ....................................................................................... 24

5 Project Results and contribution to stated objectives .............................................. 24

5.1 Achievements under Outcome 1: An internationally accepted system for recognition of GIAHS is in place (Global) ............................................................................ 26

5.1.1 Achievements against Output 1.1 ........................................................................ 28

5.1.2 Achievements against Output 1.2. ....................................................................... 29

5.1.3 Achievements against Output 1.3 ........................................................................ 33

5.1.4 Overall rating of results for project outcome 1 .................................................... 34

5.1.5 The way forward ................................................................................................. 35

5.2 Achievements under Outcome 2 ......................................................................... 36

5.2.1 Achievements against Output 2.1 ........................................................................ 37

5.2.2 Achievements against Output 2.2 ........................................................................ 40

5.2.3 Overall rating of results under Outcome 2 .......................................................... 41

5.2.4 The way forward ................................................................................................. 43

5.3 Achievements at local level, under Outcome 3 .................................................... 45

5.3.1 GIAHS and Adaptive Management ...................................................................... 45

5.3.2 Achievements against Output 3.1 ........................................................................ 48

5.3.3 Achievements against Output 3.2 ........................................................................ 48

5.3.4 Achievements against Output 3.3 ........................................................................ 49

5.3.5 Achievements against Output 3.4 ........................................................................ 50

5.3.6 Achievements against Output 3.5 ........................................................................ 51

5.3.7 Overall rating of results under Outcome 3 .......................................................... 52

Page 6: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

vi

5.3.8 The way forward ................................................................................................. 53

5.4 Achievements under Outcome 4 ......................................................................... 55

5.4.1 Achievements against Output 4.1 ........................................................................ 55

5.4.2 Achievements against Output 4.2 ........................................................................ 55

5.4.3 Achievements against Output 4.3 ........................................................................ 56

5.4.4 Achievements against Output 4.4 ........................................................................ 56

5.4.5 Overall rating of results under Outcome 4 .......................................................... 56

5.4.6 The way forward ................................................................................................. 57

5.5 Gender equality mainstreaming ........................................................................... 58

5.6 Capacity Development ........................................................................................ 60

5.7 Sustainability ...................................................................................................... 61

5.8 Impact ................................................................................................................ 63

6 Global Project Performance Rating .......................................................................... 66

7 Conclusions ................................................................................................................ 69

7.1 GIAHS and FAO corporate decision making ...................................................... 69

7.2 GIAHS and GEF Project Implementation............................................................ 69

7.3 GIAHS at National Level .................................................................................... 69

7.4 International instruments ..................................................................................... 70

7.5 Systems analysis ................................................................................................. 70

7.6 FAO Advisory Role to Governments .................................................................. 70

8 Recommendations ...................................................................................................... 71

Recommendations to FAO Corporate Management .............................................................. 71

Recommendations to FAO/NR Management ......................................................................... 71

Recommendations to GIAHS Project Management team ....................................................... 72

Annexes

1 GIAHS MTE ToR 2 Generic ToR for National Evaluation Consultants 3 Itinerary 4 List of people met 5 Evaluation Matrix 6 Co-Financing Details GIAHS (July 2012) 7 Budget overview (June 2012) 8 Objectives by project phases 9 Analysis of the Logical Framework 10 Extended MTE Findings 11 Background to Project Consultancies 12 List of Project Publications 13 National Reports of Algeria, Tunisia, China and Peru

Page 7: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

vii

Executive Summary

ES1. This Mid Term Evaluation (MTE) was conducted as a mandatory requirement for Project GCP/GLO/212/GFF “Conservation and Adaptive Management of Globally Important Agricultural Heritage Systems”. The project started in 2008, with a 5 year duration until June 2013. The MTE started in April 2012, well beyond the mid-point of the project, because there had been a lack of tangible process earlier, due to adjustments made at the start of the project and delays in implementation. A GEF funding reduction by about half of the originally planned amount was one of the principal factors necessitating project planning adjustments and re-budgeting. Considering its late timing, the MTE also looked beyond the project’s scheduled lifetime. ES2. The GEF funded global project consists of 4 components divided among four project outcomes. Project outcomes deal with the Global (1), National (2), Local (3) levels and Cross Cutting Lessons Learned, M&E, Documentation and Information Management dimensions of the GIAHS Initiative (4). National and Local project interventions took place in 6 countries: Algeria, Chile, China, Peru, the Philippines and Tunisia. Initiatives promoting the GIAHS concept at National and Local level in other countries were initiated as well. The MTE findings draw from information collected both at FAO HQ level, and in the countries where local GIAHS initiatives were supported. National and international consultant carried evaluation missions in all 4 pilot countries. Stand-alone national consultant reports are presented in annex to this report. ES3. In common with all GEF funded projects, two steps involving identification and preparatory phases preceded the full scale project (FSP). The preparatory (PDF-B) phase was drawn out from an original 14 months to a 4 year period. Subsequent FAO executed project preparation steps were initially supported through the UNDP-GEF funding window. Project preparation, by 2007 resulted in formulation of a complex multi-tier FSP. Given the importance of the formulation process on the project, the evaluation felt necessary to provide a comprehensive view of the overall GIAHS initiative from the project’s origins. ES4. The evaluation judged the project purpose of preserving knowledge and agricultural biodiversity as relevant, in particular with respect to obtaining global recognition of the GIAHS concept. The project design is complicated due to the multiple tiers (global, national, local) in which it is expected to operate. This complexity was underestimated from the outset. A major challenge was the project mandate to create a permanent entity to represent GIAHS at the global level, including a secretariat and sustainable funding. ES5. At the national level, the project expected to leverage the formulation and adoption of multi-sector national policy, legal and regulatory frameworks in support of GIAHS. At local level, the project aimed to support specific agro-biodiversity conservation and adaptive management targets, some covering substantial geographical areas. The project however was found not to have sufficiently defined the actual means for such interventions, raising concerns regarding the realism of the project design. In addition, In the evaluators opinions, by combining the responsibility of creating a global GIAHS governance structure with local implementation and national lobbying, the project set-out with non-compatible purposes. ES6. The evaluation also questioned the appropriateness of allocating the largest budget share to local level interventions and questioned the underpinning logic according to which these may be a principal drive for global GIAHS recognition. The project indeed argued that as a new concept, GIAHS calls for a convincing demonstration to gain recognition which justified supporting localized actions in selected sites. Nonetheless, the institutional set-up

Page 8: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

viii

further compounded the project structural complexity , and project resources (human-financial) dedicated to the global tier were not sufficient. ES7. A key implementation issue that affected effectiveness relates to decision-making authority. Project management decisions were dependent on the availability of the Division Director of the lead technical unit. The decision not to hire a CTA to manage the global project affairs has contributed to weakening management presence and impinged on project effectiveness. The project’s institutional arrangement should be adjusted in a way to dedicate appropriate support to global, national and local tiers. ES8. By focusing all project management and decision-making functions at the central level, local level interventions indeed suffered from cumbersome coordination arrangements and top-down decision making, eventually achieving more limited results. This is quite different from a relationship where the global entity acts as a sounding board for local and national initiatives, but allows independent decision making at these levels, which would have allowed for more conceptual dialogue, innovation and learning. ES9. Conversely, technical backstopping suffered from the limited availability of the main technical officer, whose time was largely used for administrative matters. In addition, opportunities to draw on Organizational capacity were limited by the reduced institutional anchorage of the project. ES10. The fact that global project management dealt with local operational issues has also proven detrimental to furthering the national and global institutional agenda of the project. With one staff combining the roles of global project coordinator and budget holder in addition to regular divisional management duties, the time available to devote to the initiative and placing GIAHS on the international institutional map was limited. This also did not play-out for opening up the institutional platform for GIAHS within FAO. The evaluation has found however that there is considerable buy-in for the concept among a diverse range of FAO technical staff across several divisions. ES11. The project did not allow either for decentralized decision making at Country level, therefore failing to make the most of capacities available in FAO Representations, in particular for advocacy aspects of the GIAHS initiative. To avoid a similar scenario under a potential future GIAHS programme status, a role could be given to Representations. This could entail for GIAHS to find its way in country programme frameworks (CPF), as a unifying concept for production, food security enhancement and conservation interests. ES12. The MTE has examined project achievements to date to assess how project outcomes may have contributed to increased global visibility and exposure of GIAHS as a concept. Project targets were judged to be unrealistically high, particularly in view of available funding. This may explain why achievements have been limited compared to the plan, in addition to being slow in coming. Priorities should therefore be established for the remainder of the project life and beyond. ES13. Despite the above-mentioned limiting factors and the fact that insufficient resources were allocated to the global tier, which limited the project capacity to engage with the national policy and global level, the evaluation recognized that international profile of GIAHS has nevertheless expanded considerably, with a some promising work carried out to represent GIAHS at the global level and raise its profile. Still, the current FAO institutional arrangements for GIAHS faces functional limits, related to the little institutional foundation it has been granted. The project has failed to create a wider institutional buy-in over the conceptual relevance of GIAHS to the Organization. The project has correctly concluded that

Page 9: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

ix

pursuing a formal treaty or convention development for GIAHS will not pay off. Therefore, the best guarantee for institutional sustainability lies in elevating the GIAHS to a programme level, while the Initiative has now reached a key moment after a 10 year nurturing period and given the explicit demands of participating Member States to FAO to urgently lend international credibility and permanency to the Initiative. ES14. The evaluation found that there is potential to support the elevation of GIAHS as an FAO programme. The remainder of the project period, perhaps stretched by a budget neutral extension or possibly a new phase, must be devoted to harnessing the strategic momentum for national and global aspects of the GIAHS initiative. First and foremost this requires creating an institutional arrangement within FAO to bring GIAHS champions together into a single platform. The same platform could act as the basis for a cross-departmental taskforce once the programme actually comes to fruition. Project past experience should be built upon in shaping a future programme and in particular seems to suggest that, the support to local initiatives has better be managed separately from the creation and management of a global programme that will requires direct and regular guidance by departmental management, and calls for an inter departmental dialogue at FAO HQ level on the role of GIAHS. ES15. The GIAHS concept has proven to have potential to act as a catalytic element for the conservation and transforming use of bio-diversity and culturally rich agricultural systems and offers a fresh take on conservation, combined with (niche) market oriented production. Systems can be self-sustaining and help to curb important trends of rural out-migration, provided the rule based governance system surrounding it is cognizant of its specific values and management requirements. This is where the GIAHS initiative can provide considerable added value through analysis and (re) distribution of information and knowledge. Ultimately the chance for GIAHS to grow depends on enhancing recognition and linking systems in a meaningful way to the market, for products and services they produce. International conventions and treaties which are already in existence may be more easily captured through a GIAHS provided umbrella function, while the same applies for national and international certification schemes adding value to production. ES16. The national project tier is supposed to link the global GIAHS governance and the local individual GIAHS sites. The selection of national counterparts is of key importance. In the majority of cases the national focal point was stationed under the national authorities dealing with environmental matters rather than with agriculture. The environmental institutions are not necessarily well placed to deal with the complexity of policy, legal, regulatory and operational issues facing GIAHS. Even when environmental agencies are fully committed to the idea behind the GIAHS initiative, they are hard pressed to act as a fully functional national counterpart for the initiative. Often this is because environmental ministries do not have sectoral objectives. Their mandate thus stands apart from the immediate institutional and technical concerns confronting GIAHS. The evaluation did not find convincing evidence of mainstreaming of GIAHS in sector and inter-sector policies and local level ownership does not appear to have transformed into wider national buy in. ES17. Despite the resources invested into local support, the evaluation cannot attribute much change to the project. Pilot projects may have been too small scale to act as drivers of change. Much of the potential national strength of the GIAHS derives from well thought-out multi-stakeholder arrangements and well selected national institutional leadership, which will be key to furthering the principal issues at stake. Success stories in some of the project’s pilot countries demonstrate that a balance can be found between the engagement of agricultural ministries and other stakeholders. In such cases, project managers have showed a capacity to clearly define facilitating and regulatory roles for local authorities, distinct supporting and

Page 10: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

x

advisory functions for academia, as well as the facilitating, employment, capacity building and advocacy involvement of private and/or civil society sectors. ES18. In the remaining project period, lessons should be drawn from currently supported local initiatives, by documenting the success stories, for the benefit of future GIAHS programming. Project experience should be used to contribute to improved understanding of stakeholder dynamics (between global, national and local players) and the sort of partnerships the future GIAHS programme must continue to forge. ES19. There exists also a serious need to research the complex of economic drivers which determine GIAHS systems. Apart from the bio-diversity, cultural and technical aspects, this forms one of the central premises that can help determine what type of external assistance to the system is meaningful, cost effective and sustainable. The evaluation has not found much evidence that such an approach has been followed so far. In part this may explain why the project has been lacking in vision and capacity to further define local intervention support. ES20. Lessons learned, M&E, documentation and publications of the project under outcome 4, as well as the case-study output of outcome 3, are by definition cross-cutting and serve all three other project tiers. These should be fully integrated into a future Programme, supported by a fully-fledged programmatic think-tank, while ensuring at the same time the independence of evaluation of a future global programmatic entity. It is also important is to ensure a formal representation of Member States as part of the evaluation function. ES21. Here below, are listed the recommendations formulated by the evaluation. Recommendation 1: GIAHS and FAO Corporate Decision Making

If FAO is to meet its commitment to a permanent status for GIAHS, it is recommended that FAO corporate management takes specific note of growing demand among member states regarding the need to take a clear position on the future status of GIAHS as an initiative within FAO. With the shortest possible delay this must lead to an unequivocal corporate management decision to support and take formal governance steps, which could lead to GIAHS programme elevation. The Evaluation recommends strengthening the M&E capacity under such a follow-up program initiative.

Recommendation 2: GIAHS at National Level

It is recommended to FAO corporate management through the appropriate committees and governing bodies, to undertake formal process steps which: 1) will squarely put GIAHS programme formulation on the Organizational governance agenda; 2) carry out up-stream programme preparation work under guidance of the NR departmental head; and 3) commit to put GIAHS programme establishment to a vote during the 2013 FAO conference. The combined steps requested from FAO must insist on specific commitment of departmental resources, allocation of consultant support as required, close coordination with the GEF GIAHS project to ensure optimal benefit from lessons learned and capturing its remaining financial and human resources. Programme preparation must however go well beyond issues covered by the GEF project to date, thereby actively distancing itself from the local project emphasis and pulling in corporate technical/ normative and diplomatic resources in the broadest sense.

Page 11: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

xi

Recommendation 3: International Instruments

As part of a concerted drive towards GIAHS programme establishment, it is recommended that the NR Department define and make public what is its strategy to facilitate / broker member state interest to accede to existing international instruments, as part of adding content and value of GIAHS recognition. In order to achieve this important goal of turning GIAHS into an umbrella facility, it is essential that NR management prioritizes information exchange and negotiation with partner organizations like UNESCO (cultural heritage), ILO (labour related issues), CBD (bio-diversity aspects) and UNFCC (climate change mitigation and adaptation)

Recommendation 4: System analysis

To match global, national and local capacities and resources for future GIAHS support it is recommended that GIAHS selection takes active recognition of system (national and regional) representativeness. Moreover thorough and systematic technical as well as economic system analysis must be introduced in order to prioritize support needs on basis of system recovery / development and economic integration2 potential.

Recommendation 5: FAO Advisory Role to Governments

It is recommended that, for the remaining project duration, the focus of attention be put on mapping and qualitative analysis of existing national policies in relation to GIAHS, as well as the influence / impact of legal and regulatory instruments. It is essential that the project as well as an emerging FAO programme structure for GIAHS acts in an advisory role to National Governments. The principle of introducing systematic gap analysis prior to the formal start of GIAHS selection designation and eventually certification must be adhered to if FAO wants to live up to its Policy and Normative mandate towards National Governments.

Recommendation 6: GIAHS and GEF Project Implementation

During the limited remaining project time3 GIAHS project management is recommended to focus remaining project resources on optimal documentation and transfer of accumulated insight regarding global and national GIAHS organizational mandates. To the maximum extent the project must solicit and structure information inputs from participating national governments as well as FAO Representations in participating countries. Such information as made available by the project, must feed into GIAHS programme establishment, when initiated by corporate management and as agreed by the FAO governing bodies. Local level project initiatives should be continued, but realistically re-scaled on basis of what is considered feasible in terms of available financial and human resources as well as remaining project implementation time. This will require active participation of work plan revisions by both national counterpart agencies and FAO Representations. Limited budget neutral extension for the project may be considered in agreement with the GEF and NR departmental management. Any steps directed towards, project based continuation of the GIAHS initiative must meanwhile be discouraged, as there is no evidence that additional project initiatives will bring programme prospects closer within reach.

2 Integration in terms of opportunity to generate sustainable self-financing in support of system continuity. 3 The project is due to end by mid 2013

Page 12: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

1

1 Introduction

1.1 Evaluation background

1. The Mid Term Evaluation (MTE), of which this report, its annexes, and associated national evaluation reports are the output, was conducted as a mandatory requirement under the funding agreement between the Global Environmental Facility (GEF) and the Food and Agriculture Organization (FAO) of the United Nations (UN). 2. The evaluation main work extended from April to September 2012, with field missions in May and June. The majority of interviews with people involved with and knowledgeable about the GIAHS initiative were conducted during the same period. Field follow-up meetings in FAO HQ took place in July while the last phone interviews were completed in September 2012. Report writing was stretched over a longer period than planned, as the institutional and organizational complexity of the evaluation proved to necessitate additional feedback, fact checking and testing of evaluation findings (preliminary draft circulated in October, and final draft in November). 3. Following an extended preparatory phase from 2002 to 2007, the 5 year project (known as Full size Project or FSP) started mid-2008. According to the agreed schedule, the project is due for completion by June 2013. The MTE thus took place well beyond mid-point of the project. Such late scheduling was prompted by apparent requests from participating countries with GIAHS pilot projects. The principal justification is that the project itself was experiencing a slow and gradual start, mainly due to its complex design and management structure. More specifically this resulted in only limited local “ground” level activities by the time the project was reaching mid-phase. Towards the end of the project, a final project evaluation is mandated by the project document. The MTE itself includes ideas for making the most effective use of this final project evaluation. Particular timing circumstances and overall stretched duration of the MTE led to some adjustments pertinent to the current evaluation results. To the extent possible the MTE took advantage of the progressed project status to substantially focus on post project scenarios. The evaluation period was used to have exchanges regarding such GIAHS programme options. 4. Because of the limited time for project completion, it was agreed with the responsible Department (NR) that emphasis should be put on clear priority-setting of expected results against scheduled outcomes. This approach also meant looking at priorities beyond the project’s scheduled lifetime rather than addressing issues for the project sake within the limitations of its own framework. Such an evaluation approach which is broader than what is usually expected from a project MTE is also logical in view of the high level global institutional and organizational targets set by the project under its outcome 1. These targets are aiming to lead to permanency of the GIAHS Initiative and demand from the evaluation that it maintains a programmatic rather than a project view. Accordingly the same thinking has to permeate into the evaluation conclusions and recommendations, to make these truly meaningful. 5. Thus the evaluation has looked at the post project programme scenario and the specific and most crucial inputs expected from the project to ensure that this GIAHS global outcome (1) can be achieved. There is general agreement, among all project stakeholders that GIAHS as an initiative finds itself at an important cross roads, where chartering an (institutional) way forward has become fundamental to the future of the initiative.

Page 13: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

2

1.2 Structure of the report

6. Chapter 1 presents a brief general introduction, describing the evaluation background, report structure, evaluation purpose and scope and methodology. Under this section the analytical approach followed for the MTE, as well as longer term forward looking stand of the evaluation are explained and justified. 7. Chapter 2 deals with the project context and description. This sections leans substantially on analysis of the project design process and the manner in which this design process evolved over a period of 6 years into the FSP. Review of the project logic forms an integral part of this part of the evaluation. In support, a detailed project logical framework analysis is presented as an Annex4

to this chapter. Elements of the predictive findings / values resulting from this project design analysis are systematically revisited in chapter 3 on Main Project Findings. 8. The analytical parts of the report are contained in Chapters 3 and 4. Chapter 3, analyses the result of the project preparation process as well as the influence of the project design on the FSP implementation. Only the key findings having a fundamental bearing on the project achievements are dealt with in this chapter. Chapter 4 presents the systematic assessment findings for each of the project outcomes and outputs. Unlike the preceding chapter on main findings which follows project themes of a general and cross cutting nature, it adheres to the structure provided by the three principal project tiers (global, national and local). This chapter looks at the balance and cohesion between these three levels of project interventions. Actual project results are listed and held against the project framework. For each outcome, an overall performance rating is proposed, in line with GEF evaluation practice. Last, and in consideration of the aim of this Mid-term evaluation to provide practical suggestions to direct project stakeholders for moving forward until the project ends, outcome-bound recommendations are also offered in a “way forward” sub-section. 9. Chapter 5 presents the report conclusions and recommendations (C&R), which are drawn from the preceding chapters. True to their original purpose as a MTE, the C&R are forward-looking and management and operationally-oriented. The most important difference, as mentioned, is that the C&R do exceed the project life and purposefully considers the future programmatic character of GIAHS.

1.3 Evaluation purpose

10. The purpose of this evaluation is to measure the projects progress and provided an assessment of its achievements using the UNEG / DAC evaluation criteria in relation to each of the individual outputs of the project framework falling respectively under 4 distinct outcomes (objectives). These outcomes deal with the global, national and local tiers of the project as well as the lessons learned from its engagement at these 3 levels. Based on the detailed evaluation of project outputs an achievement / project status rating is attributed following GEF standards. 11. Since the MTE for this project takes place well beyond mid-point, the evaluation report not merely plays a role in advising on the completion of the current project phase, but also looks at strategic requirements for the creation of a more permanent entity occupying itself with GIAHS at global level. As the creation of such an entity is contained in outcome 1, this also fulfils the

4 Annex 9

Page 14: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

3

purpose of providing accountability for the achievement of this particular GEF objective. The evaluation also advises on a re-balancing of the relationship between respective project tiers in a manner that will enhance realism and achievability of the project goal and that of a potential ensuing programme; in order to make maximum benefit from the project experience for a future GIAHS programme the GEF objective expressed under project outcome 4, is further scrutinized and recommendations are provided to link this outcome more effectively with programme design and content definition

1.4 Evaluation scope

12. In the first place the evaluation scope is defined by the Evaluation ToR. As mentioned, beyond the mandate provided by the ToR post project scenarios were however also taken on board. The advanced stage of the project implementation period and specifically the importance of outcome 1 (global) justified and prompted this expansion. Key elements covered by this evaluation included: i) the assessment of project design and concept: relevance of project objectives; appropriateness of project concept, design and logic; adequacy of timeframe, resources, pilot sites and counterparts; ii) the assessment of project implementation efficiency: budget and expenditures; project management, structure and steering mechanisms; technical backstopping design of M&E system; and iii) project results: attainment of outcomes and outputs, impact and sustainability. 13. While the main MTE report will refer to specific national project components and achievements, it will not cover each and every detail in a comprehensive manner. Full details of the national components can be found in the national evaluation reports, which are issued as self-standing documents with the main report. The main report will to the maximum extent capture the common denominators surfacing from analysis of the national project components. It is thus the aim of this MTE report to consolidate the overall global experience and achievement of the GIAHS initiative. 14. The scope of national evaluations was defined by a generic ToRs. Aside from harmonization of the evaluation methodology and cross referencing of evaluation hypothesis between countries, these generic ToRs also served another purpose. A strong element of data gathering on institutional, policy and legal framework issues, allowed the evaluation to match its own efforts with those of the project itself. Such national level evaluation contributions resulted in substantial amounts of fresh information, and helped to establish the depth of engagement and priority setting of the project in the preceding project period.

1.5 Evaluation methodology

15. The evaluation relies on the use of evaluation criteria (OECD) and performance standards (UNEG) to assess the rate of project progress and feasibility of attaining its stated outcomes and outputs. Overall rating terminology for each of the 4 project outcomes derives from the GEF guidelines for project evaluations.5 16. At the very start of the MTE, an Evaluation Matrix (see Annex 5) was prepared strictly following the project framework and posing generic evaluation questions. The majority of these questions are tuned towards establishing incremental benefits accruing from GEF involvement

5 Guidelines for GEF Agencies in Conducting Terminal Evaluations Evaluation Document No. 3, 2008

Page 15: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

4

with the project. The matrix questions have served a guiding purpose during the evaluation and played an important function not just in relation to GEF’s role but equally to sustainability issues that are central to the project and a future programme. 17. The GEF funded project meant to support all global aspects of the GIAHS initiative. These include both national and local interventions planned under the project itself as well as parallel funded national initiatives in countries other than the 6 selected by the GEF project. The prime focus of this evaluation was on the globally significant aspects of the initiative. A methodological review of how project design and implementation has either benefited or hindered this global dimension is therefore in place. The design review was mainly based on desk analysis of the project development process leading to the FSP. 18. To enhance understanding of the conceptual origins of GIAHS and their evolution over time, the Evaluation Mission devoted ample attention to documentation and review of the overall project design process. This process has turned out to be important in its own right, as highlighted by the gradual conceptual shift taking place between PDF-A, PDF-B and FSP. Whereas the original concept focus was on global incremental benefit delivery, the later design iterations introduced a more traditional project implementation mode, aiming at linking global, national and local levels. The explanatory value/power that comes from the followed approach makes it possible to point at the long term effects / consequences of early conceptual choices and formulation preferences. This insight is considered useful as a learning exercise which may benefit a future continued GIAHS initiative. 19. Any attempt to base findings on design factors may easily be considered a projection of knowledge in hindsight. The mission therefore wants to state categorically that it does not pretend that all issues arising from design could have been accurately foreseen. Obviously there exist varying levels of predictability, based on project and programme experience accumulated within the FAO field programme and its broader corporate environment. It is, however, useful to establish and point out cause and effect relationships which can be traced back from the earliest design phase. 20. The combined6 country specific GIAHS review visits served the purpose of streamlining the overall evaluation methodological approach and creating a cumulative body of evaluation references or evidences drawn from specific country and GIAHS site environments. The sequence of international TL’s visits7, in a way, determined the manner in which such experience accumulated and evaluation hypothesis8 were formed. A different sequence of country visits might have resulted in slightly different emphasis on issues, but would ultimately have produced a very similar evaluation outcome. The unifying elements in the form of FAO HQ based stakeholder

6 National consultancy based data gathering and pre-evaluation, combined with joint national- international

evaluation interaction with national GIAHS stakeholders. 7 1) Philippines,2) China, 3) Tunisia, 4) Peru, 5) Chile (with Algeria visited last by the national consultant only) 8 One hypothesis assumed that the manner of site selection remained relatively ad hoc and subjected to shifting and

often incomplete selection criteria, with in particular the economic aspects driving GIAHS undervalued. A further hypothesis holds that the project incorrectly banked on local level interventions with a so called specific GIAHS conservation approach. This while the development of a coherent global selection and classification structure remained underdeveloped and the project had relatively little to offer in terms of concrete policy support at the national level. A last hypothesis states that the centralized and globally based project management structure has been less that conducive as well as costly in maintaining a regular, realistic, and operationally relevant dialogue with national and local stakeholders. This is at least in part past on ignoring modern scientific approaches for the facilitation of multi-stakeholder processes.

Page 16: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

5

interviews and document review, in any case, provided the basis for forming a coherent evaluation view in regard of the GIAHS initiative as a whole, and the project in particular. 21. The evaluation also engaged in several face to face and telephone interviews with those involved in GIAHS project interventions outside the 6 GEF specific locations and countries. This included interaction with the evaluation TL and evaluation manager of a recent Kenya, Tanzania GIAHS project evaluation. It also consisted of interviews with individuals associated with GIAHS related initiatives in Japan, India and Italy. 22. All country visits were involved a combination of structured meetings with national counterpart agencies, and direct affiliated executing partners9. These meetings were hosted by national counterpart agencies, and these allowed them to present their views regarding the project. Following formal presentations, the evaluation team requested follow-up sessions with individual representatives with semi structured interviews. Institutional interviews were complemented and cross-referenced with GIAHS site specific interviews, held with a variety of stakeholders10. 23. On average, the joint evaluation visits lasted for about 5 days in each country. Upon return from GIAHS site visits11, the evaluation team provided debriefings with counterpart agencies. These meetings served to present initial team impressions and verify these for accuracy with the immediate stakeholders. The same meetings were also used to share information with national counterparts regarding the status of FAO international initiatives with respect to GIAHS. During debriefings, the opportunity was also taken to discuss potential actionable country initiatives to advance the international GIAHS recognition process. 24. With progressing evaluation insight, the mission developed further working hypotheses related to individual outcomes. Hypotheses covered the following key project elements / concerns: 1) level of advancement of GIAHS programme formulation, creation and future role of a statutory secretariat (global); 2) refinement of GIAHS selection mechanism and formal (national / international) accession process (national, global) ; 3) desirability and function of GIAHS specific policy, legislation and certification (national) ; 4) development of accurate GIAHS economic assessment and system valuation methodology (local, national, global); 5) desirability of direct GIAHS programme involvement in site implementation initiative (local, global); 6) current and future GIAHS programme function in scientific GIAHS follow-up, information preparation and distribution (global). 25. Evaluation hypotheses were repeatedly tested by means of “second round12” interviews with national counterpart representatives and FAO HQ staff13, international GIAHS stakeholders14. Interviews with FAO HQ staff with past and current involvement in GIAHS were considered

9 At GIAHS site level. In most but not all instances national counterpart agencies did accompany the evaluation

team to site level, however meetings with different national/ local project implementation actors were held separately.

10 Including local officials, farmers, regional elected representatives, local academia, NGO and private sector representatives (the latter including chamber of commerce, food processing, hospitality industry and tourism)

11 One GIAHS site was visited in each country, except for Peru, were geographical diversity and size of the GIAHS “transect” merited multiple site visits.

12 Follow-up with same respondents 13 Including but not limited to project management and budget holder. 14 Mainly through teleconferences, and including: FAO GEF focal point, (former) GEF officials involved in project

preparation / negotiation, GIAHS project consultants (including the evaluation consultant for a parallel German funded GIAHS project), Representatives of International Organizations and Academia, Representative of the Global GIAHS steering committee.

Page 17: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

6

particularly useful to obtain a fair picture of the strength and weaknesses with respect to the GIAHS initiative to date. Based on the testing and hypothesis adjustment, the mission shared its initial key findings, conclusions and recommendations (C&R) with project management, the budget holder and FAO corporate management. This was done with the aim to reach an early agreement on prioritization of the main outstanding issues, and the general way forward.15

1.6 Constraints to evaluation

26. Implementation results for this global project are to a fair measure open to interpretation. This is quite normal for complex evaluations dealing with considerable policy goals as well as field based implementation. Complexity and variability of results is only increased by the fact that a project like the current one deals with highly diverse stakeholder interests16. The complexity of the global project setting does to and extends offers additional freedom to the evaluation to highlight or down/play specific project achievements. The evaluation has opted not to systematically follow the project framework and tick off what the project has done and not done, without showing any preference or relative importance. As agreed, the evaluation looked beyond the project framework and elected an evaluation approach, which accommodated a focus on both tangible and less tangible project results. Incidentally this still follows the general spirit of the project framework, as both categories of outputs are contained by project. 27. The Log Frame of this project introduced an important number of hard geographical and quantitative targets That the evaluation judges as unrealistic in view of the fact that there are, to the evaluators’ knowledge, no tools and/or practical modalities to accurately measure achievements against such indicators (e.g. area based conservation targets or number of species to be conserved). 28. Moreover the physical achievements that could be reached in the short period opened by this project, such as short-term bio/diversity conservation 17, is considered by the evaluators as of relative importance in comparison to the question of the GIAHS initiative’s sustainability. The evaluation thus looks at the project initiative first and foremost as a policy and institutional effort, with more focus towards the less tangible global and institutional project results, rather than those it cannot quantify or considers of a lower programmatic importance/relevance at this stage.

2 Project context and description

15 Varying by respondent, the MTE received differing levels of feedback in terms of accuracy and fundamental

nature of responses. This may be seen as somewhat regrettable as the overall spirit of MTE recommendations focused on the way forward. Certain responses could however be seen more in the light of defending individual positions, rather than availing of the opportunity to have a joint critically constructive look aiming for the benefit of GIAHSs future’.

16 Apart from local custodians of GIAHS, these include, local and regional governments, national ministries, NGO´s, private sector representatives, as well as International Organizations.

17 Even if it was actually possible to measure such achievements based on solid baseline data and in presence of a well-developed monitoring system, which was already demonstrated to be absent for the current project itself. Part of the project strategy has been to contractually outsource such monitoring, but the evaluation has not found evidence of a systematic approach among participating countries to conduct such studies.

Page 18: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

7

2.1 Origin and context

29. The project deals with “Globally Important Agricultural Heritage Systems” (GIAHS), a term coined within FAO, used to describe remarkable land use systems and landscapes which are rich in globally significant biological diversity evolving from the co-adaptation of a community with its environment and its needs and aspirations for sustainable development. It can interchangeably also be used for the programmatic concept of stakeholder relationships between the system and the outer world to begin with the larger eco-system in which the GIAHS is embedded. 30. The project finds its origin as a partnership initiative of FAO launched at the World Summit on Sustainable Development (WSSD) in Johannesburg, 2002. The GIAHS concept originally emerged in the then FAO Sustainable Development Department and, together with its main proponent, was transferred to the Land and Water Division within the Natural Resource Department (NRL/NR) with the last restructuring of the Organization. 31. Mainstream recognition of the GIAHS Initiative within FAO has so far been elusive. During 2012 references to it have been made in COAG, Programme Committee and Council Reports as a candidate for incorporation in the Organization’s programme of work. This indicates that the Initiative has maintained and expanded its relevance as part of the international discourse among Member States of the Organization. Such relevance is generally said to be attributed to the innovative and holistic take that GIAHS offers on conservation, in combination with broadly recognized global agricultural production issues18. At the early stages of the Initiative in 2002, this discussion was dominated by the notion of multi-functionality in agriculture. This entry-point and the manner in which it dominated the perception of GIAHS led to a situation where it was difficult to reach consensus about the potential role and value of the initiative for FAO and its membership constituency. 32. It is in this spirit that the GIAHS Initiative set off19 with the development of a recognition framework, based on a set of selection criteria. The benefits from engagement with GIAHS were implicitly understood to lie in their unique bio-diversity, cultural and technical value, which co-evolved with specific environments in which systems are located. 33. The PDF-A project document (2002) states20: “The aim of this project is to identify and mobilize recognition and support for the conservation and sustainable use of globally important agricultural heritage systems and landscapes and their associated agricultural biodiversity and knowledge systems”. The concept for the project is therefore directly linked to the CBD Agricultural Biodiversity Work Programme and the GEF Operational Programme (OP13)21: “Conservation and sustainable use of biodiversity of importance to agriculture”.

18 This is based on the multitude of roles and functions accorded to the GIAHS concept, as explained in the FSP and

subsequent informative and promotional materials produced by the project, national initiatives and global project secretariat. The evaluator gives a separate description of what it sees as the principal GIAHS advantages at the start of Chapter 6, project implementation.

19 The elements of recognition, identification, selection and certification was already contained in the PDF-A. It evolved and was simplified over time, but can as yet not be considered as a set of final selection criteria and procedures.

20 The current GCP/GLO/212/GFF project and its 2 preparatory phases20 have been on-going since 2002. 21 CBD COP 5 Decision 5, May 2000 and sustainable use of biological diversity (decision 24), and the knowledge

innovations and practices of local and indigenous communities (decisions on Article 8j).

Page 19: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

8

34. The PDF-A document also states that: “although there is increasing ad-hoc recognition of the value of GIAHS, through for example, scientific media, CBD and CCD, this is not translating into a widespread acceptance and coordinated support on a worldwide basis”. Thus there is an argument for “modest but critical interventions that promote the maintenance of these alternative systems and maintain their viability”. So far “only ad hoc support has been directed to sustaining such ingenious agricultural systems”. Modern mono-cropping and high external input based agriculture combined with “in some cases, opposition to tradition” has led to inadequate recognition or attention for “the important knowledge and agricultural biodiversity they maintain”22. 35. The message that can be taken from the above statement is that the project and an ensuing programme are supposed to provide critical but limited interventions within a systematic and permanent global framework. The precise nature of interventions at various levels (global, national and local) was unevenly defined among countries at the early stages of the project. However the purpose of such interventions were always intended to create conditions for self-sustenance of GIAHS sites and the associated conservation of the bio-diversity and cultural values contained in them. The pathway envisioned for this was seen as the promotion of adaptive management and co-adaptation. Hereby individual GIAHS should take advantage of the catalytic environment created by global project / global programme. 36. The original scope of the GIAHS project as reflected in the PDF-A terminal report mainly spoke of its catalytic information and knowledge based relationship with the international environment. This shifted later on more towards active intervention involvement within local GIAHS sites, of an organizational, planning, facilitating, investment, research and consultancy based nature. 37. Ensuring that knowledge contained in GIAHS and conservation of agricultural biodiversity takes place is seen as important. This is because it may provide valuable alternatives to modern agriculture to help face the various crises this sector has experienced. “Scientific

evidence showing that GIAHS can be viable and sustainable alternatives to systems that rely on

mono-cropping, exotic breeds and high external inputs, is increasing “ Or “valuation techniques

have shown the comprehensive advantages of such systems in food production in the medium and

long term”. Despite this, there is inadequate recognition of, or attention to the value of GIAHS. Increasing pressures on such systems are resulting “in serious gaps in transmission of this

important global heritage, constraining farmer/herder innovation, and potentially blocking the

evolution of domesticated species”. 38. The project intervention is presented as an innovative contribution to the “dynamic conservation of selected viable ingenious systems on a demonstration basis, diagnosis, documentation and dissemination of knowledge and best practices”. Moreover, a further reason to pay special attention to this intervention is the observation that GIAHS “is not perceived as being of high national importance in many countries where such systems exist, due to competing development priorities”.23

23 This underscores the presently loosing nature of the GIAHS advocacy argument in the face of main stream agricultural development.

Page 20: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

9

2.2 Project description

39. The project concept has undergone a perceptible shift from focus on global governance, national facilitation oriented entity, into putting considerable efforts in creating its own diversified implementation pilots over the 3 continents. Annex 8 (Objectives by project phase) points at this through a systematic comparison of the evolving project objectives and outcomes between the PDF-A, PDF-B and FSP phases.24 40. The original project concept as set in PDF-A presents project objectives which reflect characteristics of a standard GEF global approach. The key idea is to provide / leverage incremental global value in relation to national and local GIAHS initiatives. Such added value comes in the form of research and knowledge enhancement (at global level, Objective 1, PDF-A), recognition and advocacy and policy (at global and national level, Objective 2), and capacity building initiatives (at national and local level, Objective 3). 41. Influenced by a workshop held at the end of the PDF-A, and coinciding with the global First Stakeholder Workshop and Steering Committee Session (Rome, 5-7 August 2002), the project concept underwent a perceptible shift. This is presented as: Enhanced GIAHS understanding and mobilization of recognition (categorization), including provision of financial support and policy incentives (at global and national level, Objective 1, PDF-B) 25. Generally, an increased focus on pro-active operational engagement at site level can be noted: “demonstration of

dynamic conservation”. This entails direct project interaction with selected GIAHS to feed experience into (national) policy initiatives and to improve GIAHS categorization (local, national and global level, Objective2, PDF-B). 42. Following the framework of expected PDF-B outputs, 5 different pilot demonstration systems in 6 countries were selected.26 All of these were eventually adopted under the FSP. Aside from the 5 selected systems, other potential systems were “studied and identified for further

expansion and networking of GIAHS (systems)” (PDF-B Terminal Report, 2008). 43. Key papers were commissioned to build the overall GIAHS methodological framework, and to guide pilot participatory development interventions. Work took place in selected (candidate pilot) systems and countries to “design Pilot Frameworks for each pilot system through a fully

participatory process”. This covered a period of “almost 3 years” from 2004 to 2007. Direct on site participatory engagement, following a multi-stakeholder approach, was considered to cover the full project planning and preparatory requirements of FSP GIAHS pilots. 44. Active leverage of global and national support for GIAHS was another component of PDF-B, and resulted in development of GIAHS programme development strategic path, including the preparation of papers examining GIAHS context in existing multilateral instruments (S.R. Harrop, August 2005) and analysing the drivers of change in GIAHS farming systems (notes prepared by F. Dévé FAO, 2004). In this context, it is mentioned that national funding support from country-specific biodiversity allocations may be used for GEF co-funding support.27 A

24 Starting with the introduction of FAO’s 2007 revised guidelines for project formulation 25 To individual GIAHS 26 These are : 1) Andean agriculture in Peru (Cusco-Puno) , 2) Chiloé Agriculture, Chile, 3) Rice-Fish systems in

China, 4) Rice terraces of Ifugao, The Philippines , 5) Oasis of the Maghreb, (Algeria and Tunisia) 27 This was based on endorsement by GEF operational focal points in each pilot country, PDF-B terminal Report

2008

Page 21: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

10

further component of PDF-B was the elaboration of a communication strategy and plan. This encompassed the launch of the GIAHS website (2006), production and distribution of hard copy GIAHS information materials, and preparation of the FSP communication plan and strategy. 45. At site level, efforts were made to enhance “awareness” regarding GIAHS among “vulnerable stakeholders”.28 Lastly, the PDF-B phase saw the preparation of the full FSP document which was submitted to the GEF CEO for endorsement. The previously mentioned national project frameworks for the selected countries were produced in addition to the project document. Furthermore, two major events designed to provide additional direction to the GIAHS conceptual framework took place prior to the formal start of the FSP in 2008. 29 46. Following the extended preparation phase30, the FSP was eventually launched in mid-2008. The FSP maintained most specific elements for the generation of global incremental benefit. The most striking element of the project was, however, to emphasize forging of direct project links between local site interventions, support to national policy, legal framework contributions and the global governance aspects of GIAHS. Here, the most important unifying concept is contained in a vision of bottom-up generated influence by project-specific GIAHS site interventions over national level policy and legal framework adaptation. The project design makes a clear distinction between global, national and local level interventions foreseen. 47. Resulting from FAO changes in its project formulation format, the FSP was left with a single remaining objective: to promote conservation and adaptive management of globally significant agricultural biodiversity harboured in globally important agricultural heritage systems or GIAHS. The four project outcomes as formulated are considered to substitute for the project objectives earlier presented during the PDF-A and B preparation phases. The project outline presents a clear budgetary allocation for each outcome. 48. From a broader budget allocation perspective, it should be noted here that the actually allocated GEF resources were cut by half from the original estimates for the FSP during project preparation. This however was not reflected in a proportional reduction of the project implementation ambitions for the project itself. These remained relatively unchanged and increased in scope where local interventions are concerned. Of course the GIAHS initiative of which the GEF funded project under evaluation forms the core, also knows other project funded components, outside the 6 countries on which GEF itself focused. These components and interventions are reflected in the total international co-funding of the initiative, Annex 6. The project funded secretariat and international staff dealt with all these other national GIAHS initiatives.

2.2.1 Stated project outcomes

49. Outcome 1 reads: “An internationally accepted system for recognition of GIAHS is in place (Global) (Total cost: USD 1,375,001; GEF: USD 375,445; Co-financing: USD 1,000,556)” This outcome has 3 outputs contributing to it: 1) international public endorsement of the GIAHS concept; 2) creation of long term GIAHS programme support structure and statutory secretariat

28 Terminology taken from the project framework, and interpreted by the evaluation to mean equitable access to

information. 29 Second International Steering Committee Meeting, 7-9 June 2004; International Forum on GIAHS: experiences on

dynamic conservation, 24-26 October 2006. 30 PDF-B

Page 22: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

11

negotiated through an inter-governmental process; 3) a sustainable financing mechanism in support of the former.31 50. This outcome covers the global GIAHS aspects and the long term programme ambitions for the concept. It is supposed to bring GIAHS recognition to the next level, carrying the concept beyond the project stage and ensuring its future expansion. Linkage with existing multilateral instruments is discussed as part of this outcome. However, the approach to follow in order to achieve such formal linkage is left open-ended “in order to accord international importance to

GIAHS there is a need for developing a supportive policy declaration exclusively dedicated to the

concept, by building on the positive reinforcement of the concept in existing international

instrument”. 32 51. From the quoted funding figures for this outcome it may be noted that it foresaw in allocation of substantial co-funding: “While GEF resources are being requested to complement co-financing for catalyzing this system, future funding of this will be negotiated under the FSP from other sources.” 52. National level interventions are covered by Outcome 2 and its underlying two outputs: “The conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National), (Total cost: USD 1 878 661; GEF: USD 534 441; Co-financing: USD 1 344 220. 53. The first output aims to assert a (positive) influence on (inter-) sectoral policy, legal and regulatory development at national country level. Proposed tools are workshops, development of policy proposals and guidelines dealing with issues from national level GIAHS (protection and cultural) designation, environment, tourism, land-tenure (adjustments), indigenous peoples rights, and national resource access. An embedded approach is advocated by seeking to conduct “lobby

and awareness raising activities, (including) through the identification of GIAHS champions”.

The second output focuses on national level institutional capacity building in support of policy mainstreaming and largely covers the intervention areas of the first output. Tools include “training

sessions on the legal and policy requirements for the conservation of GIAHS”, workshops, and policy briefs. 54. In terms of the GEF funding contribution, less than one third of the cost foreseen for this outcome is covered, with the remaining costs to be supplemented by counterpart funding. Aside from the GEF funded interventions of this kind, co-funding-based interventions were to support similar activities outside the six countries directly covered under the project33. These include initiatives in Japan, India, Kenya and Tanzania. The latter two countries are covered under a

31 These outputs as well as subsequent ones are rephrased and shortened for the purpose of report readability, using

original key words and their original meaning. 32 A footnote in the FSP documents reads: “A multilateral convention would be the ideal solution to securely

establish the GIAHS concept, but it seems unlikely that this would be feasible in light of the time it would take to negotiate and put in place”. On the one hand this indicates the desire to have such a separate instrument as it is considered that the full breath of the concept is not covered under existing multilateral instruments . It does however also indicate that no such instrument will be pursued during the project, which leaves the multilateral status of GIAHS relatively open ended, of limited specificity in terms of its definition.

33 Face-to-face and phone-based interviews with individuals associated to these parallel initiatives were carried out by the evaluation about these additional sites.

Page 23: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

12

German funded project, which is also mentioned as part of the GEF co-funding and therefore a formal part of the baseline. 55. From the support scenario for the national level, Outcome 3 moves to the local level; “Globally significant agro-biodiversity in pilot GIAHS is being managed and sustainably used by empowering local communities and harnessing evolving economic, social, and policy processes and by adaptation of appropriate new technologies that allow interaction between ecological and cultural processes (Local). Total cost: USD 8,491,906; GEF: USD 1,108,152; Co-financing: USD 7,383,754. The key premise of this outcome is that adaptive management (AM) should effectuate change and develop novel political, social and economic processes governing GIAHS. This is achieved in a way that maintains or improves the basic characteristics defining the GIAHS. Hereby “the outcome will address the obstacles for long-term sustainable management of GIAHS”.

56. The five proposed outputs under this outcome are: 1) the establishment of collaborative public-private-civil sector stakeholder set-ups; 2) process monitoring and analysis to enable the enhancement of positive impact and community empowerment; 3) screening testing and introducing management and productivity enhancing technologies in GIAHS; 4) design and implementation of alternative and supplementary livelihood programs to absorb excess agricultural labour within GIAHS; 5) preparation and dissemination of case stories covering local level management and support to individual GIAHS. 57. Foreseen global governance and management of GIAHS is based on an international networking model, linking and exchanging locally and nationally gained experience. The project aims to facilitate this through its outcome 4: “Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion and up-scaling of the GIAHS in other areas/countries and creation of the GIAHS network (Global, National, Local) Total cost: USD 5,126,650; GEF: USD 1,172,742; co-financing: USD 3,953,909. 58. Aside from its networking role, this outcome is centered on analysis and documentation of GIAHS’ best practice and lessons learned. Four (4) outputs are formulated to achieve outcome 4; 1) Implementation of the project M&E plan to inform and allow the project to adapt as needed to achieve formulated outcomes; 2) A global GIAHS publication making available distilled lessons learned and best practice on key management aspects; 3) Publication and dissemination of scientific reports based on project work; 4) Maintaining the GIAHS website as a forum for global exchange.

Page 24: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

13

3 Analysis of project relevance, concept and design

3.1 Relevance of project objectives

59. Considering that a principal objective (outcome 1) for the GEF-funded project under evaluation and other associated GIAHS projects is to put the subject and its importance on the global ‘map’, the mission agrees with the project’s key underpinning that recognizes the importance of having elevated targets to act as a driver of the global and national GIAHS recognition process. The evaluation also acknowledges the ambitious purpose at national level dealing with national policy and legal frameworks (outcome 2). 60. The PDF-A34 project document (2002) states: “The aim of this project is to identify and mobilize recognition and support for the conservation and sustainable use of globally important agricultural heritage systems and landscapes and their associated agricultural biodiversity and knowledge systems”. The original GIAHS idea as reflected in the PDF-A terminal report mainly spoke of its catalytic information and knowledge based relationship with the international environment. The initial objectives underpinning this project and the global level impact it was aiming to reach, were judged as entirely relevant by the evaluation mission. Ensuring that knowledge contained in GIAHS and preservation of agricultural biodiversity takes place is seen as important. 61. From the PDF-A to PDF-B, the project concept has undergone a perceptible shift from focus on global governance, national facilitation oriented entity, into putting considerable efforts in creating its own diversified implementation pilots over the 3 continents, shifting more towards active intervention involvement within local GIAHS sites of an organizational, planning, facilitating, investment, research and consultancy based nature. 62. According to the evaluation, GIAHS as a concept holds a high level of relevance and the relevance of this project was largely stemming from its global drive, towards international recognition. For that matter, the questioning, early on the project, of the realism of the global objective aiming for a formal international Instrument covering GIAHS as an agricultural category, created an unbalance within the project objectives. The subsequent shift of the project thrust away from its global aim therefore is can be seen as having affected its overall relevance.

3.2 Project concept and design

3.2.1 Analysis of project concept and logic

63. The mission takes into consideration the long and difficult project gestation process and the initial setbacks this resulted in for the launch of the GIAHS initiative. The formulation process has affected the project design clarity and in no minor way this resulted in high level of complexity. 64. The logical framework (LF) for a complex initiative involving multiple stakeholders such as GIAHS if of considerable importance to provide clear directions. The GIAHS initiative as it has

34 The current GCP/GLO/212/GFF project and its 2 preparatory phases34 have been on-going since 2002.

Page 25: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

14

developed with support from project GCP/GLO/212/GFF demonstrates that investment in LF preparation should be prioritized. Obviously, this should be carried out as a joint process and the early phase of the project should allow for systematic revisiting and adjustment of the LF. The preparation of a LF for project GCP/GLO/212/GFF appears to have been carried more as an afterthought or administrative requirement of project formulation. The MTE argues that devising a LF at the start of the project would have resulted in superior quality and value. As it is, the current LF has served the purpose of providing a reporting framework and but does not go beyond that. This missed chance, has immediate consequences for the project’s capacity and openness to self-assessment and critical stock taking of the direction it has taken, let alone to the capacity it has to change course, when needed. 65. The evaluation has found evidence that the complexity of dealing with local, national and global stakeholder dynamics35 has been underestimated. This has had a limiting impact on the manner in which natural stakeholders have engaged, been able to complement each other and move towards sustainable economic inclusion of individual GIAHS and opportunities for generation of incremental benefit36

3.2.2 Adequacy of timeframe and resources

66. Considerable demands have been required from the project teams during the initial roll out stage of such new global initiative as GIAHS. A human resources (HR) crunch seems to have weighed on the project’s global management and coordination but also was felt through the limited opportunities to flexibly draw on external capacity or FAO staff member. Moreover by combining the responsibility of creating a global GIAHS governance structure with local implementation and national lobbying, the project design was “saddled” with a variety of responsibilities and interest that were difficult to articulate.37 67. Inclusion of over-ambitious and often unrealistic project targets in relation to secured project resources for local, national and global outcomes have made achievement of several outputs a tall order. An example of this is the large bio-diversity conservation areas, directly resulting from the choice of project pilots.

3.2.3 Selection and choice of initially designated pilot sites

68. Efforts to develop GIAHS selection criteria have been progressing over time. Early on, efforts focused on capturing GIAHS’ composite38 nature through a broad set of selection criteria39. During the early stages of GIAHS conceptual development, selection criteria appear to have been applied in a seemingly contradictory manner40. This has now been eliminated however. GIAHS purposely does not opt for narrow selection definitions, although these would make it easier to

35 Despite sufficient forewarning about the fundamental importance of this aspect during the extended formulation

phase. 36 For their immediate custodians. 37 International structure creation and rule setting, combined with execution along these rules as well as monitoring

itself and others in abiding to the governance structure. 38 Composite in the sense that it focusses on the broad range of characteristics defining the agricultural system, which

as a result also leads to a higher level of complexity in dealing with selection criteria. 39 Encompassing, historical, physical, technical, cultural, institutional and economic dimensions 40 Selection criteria on the one hand ask for historical ‘prove’ of GIAHS resilience in the environment of their

existence, while threat to the system survival forms another criterion.

Page 26: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

15

pigeonhole the concept. This more holistic and complex process to describe, classify and give practical meaning to GIAHS must be lauded. 69. A simplified set of selection criteria would surely make it easier to place GIAHS under existing conventions and treaties. However, seeking “easy” alignment with a specific existing classification would not do justice to the contemporary, complex and dynamic nature of GIAHS. Rather than equating GIAHS with existing international instruments, this leads to the justified view that it must rather be considered as a broad umbrella concept. Specific affiliations and certifications provided through existing mechanisms may take place under it and may be assisted by GIAHS selection and recognition. 70. The selection of GIAHS sites has increased in effectiveness and efficiency over time by reducing process complexity. The project, however, still remains affected by a past selection process which was lacking in transparency and which, in some cases, led to the inclusion of sites where sustainability of project efforts are difficult to guarantee. There are strong arguments for working towards a further honed selection process. The role of an established scientific committee, operating independently, should be central to efficient selection of GIAHS sites. 71. When the evaluation mentions further honing and setting of a universal selection framework, it does in no way imply the creation of a heavy and bureaucratic selection and accession procedures. However clearly there is the need for a more formalized selection process based on a set of transparent and proportionally weighed criteria. Such a system is crucial to enhance the level of credibility of the selection process, its global recognition and GIAHS branding. Currently the establishment of such a mechanism is not (yet) sufficiently in evidence. Investment to define and articulate the role for national pre-selection as a first stage is required. This stage is to be followed by accession to the global selection mechanism. The overall internal dynamics of this two- step approach, including binding requirements and responsibilities are to be further clarified and elaborated.

3.2.4 Choice of project national counterparts

72. A GIAHS programme, giving full justice to its composite nature should be integrated in the agriculture sector and its surrounding economic environment. National GIAHS linkage to the environmental sector, just because of its agro-biodiversity aspect, only covers a single, albeit, important GIAHS component. 73. It was pointed out systematically in all visited countries, that because of the sector nature the national interlocutor of choice for GIAHS is the Ministry of Agriculture. The evaluation is in full agreement with these expressed views. Meanwhile the evaluation is aware about the institutional process along which the project came into being, and can therefore express understanding about the difficulties, at the time of engaging with MoA’s due to their more limited affiliation with GEF, more productivist orientation and less ecological inclination. Still, by not placing GIAHS institutionally where it belongs, the effectiveness and efficiency of institutional engagement has been jeopardized. This fact directly contributes to a lowered impact, and institutional sustainability of efforts to date.

3.2.5 Stakeholders’ participation

Page 27: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

16

74. Strategizing stakeholder engagement needs to be matched to available resources, not least of which is time. Prioritizing stakeholder interventions (from local to national and vice versa) ensures that quality prevails over quantity and the chance of long term sustainability over short term visibility. Custodians of GIAHS, namely local farmers / communities will then be the first to recognize and agree that the cause of GIAHS is better served by structural public and private sector recognition and support, rather than by one off project interventions. By disproportionally focusing and engaging at the local level, the project has rapidly lost steam insofar as the more elevated and more strategic stakeholder processes are concerned. The absence of a scientific approach towards building stakeholder platforms and communication arrangements has resulted in reduced project effectiveness at the institutional level. 75. In the above context the evaluation makes reference to the articulated (social) scientific approach towards institutional stakeholder dynamics, introduced during the PDF-B phase41. There is evidence that such advice was not internalized by the GIAHS initiative (project). This is because of an absence of a specific project strategy to engage along the lines of advanced understanding of institutional and social dynamics. The project strategy in this respect does not reach further than making oblique reference to participatory consultations of stakeholders, without specifying of how this takes place in practice. The latter, as well as interviews carried out by the mission suggests that means of consultation and level of participation was largely locally determined without clear and specific guidance from the global project level.

4 Analysis of project implementation

4.1 Budget and expenditures

76. The total budget for the project is USD 16,872,218 and is made up by contributions from GEF and co-financing, broken down in detail at the activity level for each of the 4 project outcomesas follows: 42 Table 1: Combined GEF and co-financing figures (USD)

Outcome Total GEF Co-financing

1 Global 1,375,001 374,445 1,000,556

2 National 1,878,661 534,441 1,344,220

3 Local 8,491,906 1,108,152 7,383,754

4 Lessons Learned Global, National and Local 5,126,650 1,172,742 3,953,909

77. The project design effectively called for country funding contributions within the GEF Resource Allocation Framework (RAF). In most instance, where and when such RAF allocations were used for project purposes, these also went into meeting of locally directed project expenditures. Each of the participating pilot countries negotiated co-financing commitments from

41 This took place in the framework of technical assistance provided through the Ministry of Cooperation of the

Netherlands, Dr. Frank van Schoubroeck et al, 2007 42 China USD1,200,000 – Chile USD990,000 – Peru USD1,600,000, Philippines USD1,000,000, Algeria USD

100,000 and Tunisia USD 100,000 (combined cash and in kind contributions as per commitment letters)

Page 28: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

17

the aforementioned RAF. (Re)-negotiated co-funding amounts were reflected in commitment letters and annexed to each project country frameworks. 43 78. Total co-financed sums do not fully correspond with the FSP document figures. This may be accounted for by the fact that some co-financing sums from third party sources, did not actually materialize. Main third party funding of the GIAHS initiative have focused on national and local interventions (outcome 3). National co-funding contributions have been variable in terms of their importance and influence. Especially in case of China the national co-financing sums have far exceeded the original planned amount. This is a reflection of the national adoption of the GIAHS concept and it being rolled out in several new locations. For now the experience of China remains unique among the 6 pilot countries. 79. In relative weight, the global component (outcome 1) relies for about 75% on GEF funding, the national component 25%, local about 12 % and 80% for component 4 dealing with lessons learned, communication and promotion of results. The higher percentage wise contribution thus shows up in the global and specifically cross-cutting project components, though the absolute amounts allocated and the impact the actual budget distribution have resulted in different project priority settings. From this point of view, the overall budget allocation, including the leveraged co-funding amounts for each component has considerably steered the project towards a local level implementation. 80. No budget revision took place during the implementation period to-date. Table 2: Expenditures (as per BH figures)

Country

Resources Allocation Framework (USD)

Actual Expenditures (USD)

Total 2008 2009 2010 2011 2012

Algeria 200,000 30,436 17,950 2,027 50,413

Chile 600,000

1,286 101,339 87,442 18,224 208,291

China 500,000 140,815 110,707 152,154 36,010 439,685

Peru 600,000

24,908 25,812 73,685 66,005 190,410

Philippines 500,000 44,088 77,639 53,572 32,942 208,241

Tunisia 100,000

23,592 13,975 32,911 1,618 72,096

HQ 1,000,000 13,296 179,114 213,586 331,363 115,784 853,144

Total 3,500,000 13,296 444,239 561,008 731,127 272,610 2,022,280

81. The table above points at under-spending of the budget since the project started. Up to date the delivery of the project is about USD 2,022,280 including commitments leaving a balance of about USD 700,000, resulting in under-delivery with the NTE in June 2013. However, Oracle Financial Statements as of June 2012 show a difference of about USD 150,000 with a commitment level of USD 2,172,843; actual reported expenditures of USD 1,987,18444. These amounts should soon reflect amendments requested to the BH by the GEF Coordination Unit.

43 The reason to speak of re-negotiated is that during the lengthy course of project preparation the levels and actual

commitments to funding and co-funding shifted over time. So final figures had to be agreed upon when the FSP started in 2008.

44 Source: GEF Coordination Unit

Page 29: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

18

82. A possible explanation of such under-spending overall is that the project faced initial delays, quite significant in some countries, attributable to internal administrative FAO requirements with regard to the mandatory signature of Government Cooperative Programme Agreements by each individual participating country (late signature, in Peru for example). A duly signed GCP Agreement is required to be able to declare a project operational and implement activities in the country. Also the translation of the project document into Spanish was needed. Indeed, the low rate of expenditure, below 1% in 2008 and around only 15% in 2009 and 2010 also evidences a slow start. Table 3: Overview of expenditures (in USD) by project components, as of June 2012 (as per BH figures)

Component Indicated Budget Actual Expenditures Expenditures Status

(A) (B) (B ÷ A)

Component 1 $ 374,445 $ 198,352 53%

Component 2 $ 534,442 $ 251,309 47%

Component 3 $ 1,108,152 $ 749,509 68%

Component 4 $ 1,172,742 $ 684,150 58%

Component 5 $ 310,220 $ 138,961 45%

$ 3,500,001 $ 2,022,280 58%

83. While the distribution of expenditures according to project components (ref. Table 3) has been relatively balanced; it is striking to see that an analysis of expenditures according to the project location evidences important discrepancies between countries and with Headquarters. The latter has captured 42% of total expenditures while planning had allocated 29% of the budget for HQ-related expenses. As for national and local implementation, in line with findings presented in section 5, expenses show that China has captured a larger part of the budget than planned, while others have so far under-spent compared to initial budget allocations, as per table 4. Table 4: Expenditures by location

Country / Location

Expenditure by location

planned actual

Algeria 6% 2%

Chile 17% 10%

China 14% 22%

Peru 17% 9%

Philippines 14% 10%

Tunisia 3% 4%

HQ 29% 42%

84. The responsibility for the overall operational execution and financial management of the GIAHS project is performed by the Budget Holder/Global Coordinator in NRL45. The duties and responsibilities of the Budget Holder for the project are defined in a memorandum from the Assistant Director-General of the Technical Cooperation Department (TCD) to the Assistant Director-General of the Natural Resources Management and Environment Department (NRD)

45 The NRL Director retained the functions of Global Coordinator and Budget Holder

Page 30: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

19

when the project was declared operational. Subsequently, the Budget Holder was nominated by the Assistant Director-General of NR. Expenditures in the field are authorized by the Budget Holder to field offices through field disbursement authorizations that specify the purpose(s) and the level of the authorized disbursement(s). 85. The FAO GEF Coordination Unit in TCID is responsible for the implementation of overall supervisory functions that include periodic in-depth reviews of project expenditures as well as the timeliness and quality of progress reporting with regard to both operational as well as financial chapters as required by progress reports. If and when the FAO GEF Coordination discovers during financial reviews discrepancies or charges that require explanations, the FAO GEF Coordination Unit contacts the BH to correct and revert any questionable charges or to provide justifications that are acceptable to the FAO GEF Coordination Unit considering the relevance of the charge made in the context of the agreed scope and goals described in the project document. If not acceptable the BH is instructed to correct any such charge. Table 5: Expenditures as of June 2012

Account Description Indicated Budget (USD)

Algeria Chile China Peru PhilippinesTunisiaHQ Total

Contracts 840,020 18,052 119,135 359,040 64 105,195 29,893 37,003 668,383

Professional

Salaries 1,027,500 - - - - - - 425,730 425,730

Travel 262,982 12,146 31,364 52,187 38,888 48,622 14,858 173,659 371,724

Consultants 509,566 - 51,139 6,789 107,246 8,247 1,995 164,702 340,118

Training 561,150 5,079 6,519 6,618 18,196 24,492 25,233 2,000 88,138

GOE 247,032 2,455 134 5,959 1,754 12,487 117 38,470 61,375

Expendable Procurement 2,250 508 - 5,505 19,421 1,121 - 1,842 28,396

Non Expendable Procurement 49,500 12,172 - 3,587 4,840 4,636 - 477 25,713

OVH/Chargeback 0 - - - - - - 9,261 9,261

Locally cont labor 0 - - - - 3,364 - - 3,364

Hospitality 0 - - - - 77 - - 77

86. The levels of expenditures (ref. table 5), both over and under, reflects globally the level of implemented activities as described in this report. According to the GEF Coordination Unit, some of these figures however need to be reviewed, particularly on travel and procurement budget lines. 87. Looking at the structure of expenditures to-date, two lines show over-expenditures: actual travel expenses indeed appear above the original budget by 141%, and expendable procurement by over 1000%. The latter probably reflects a mis-allocation as this type of expense should be, as was planned, very minimal for such project. As mentioned, corrective measures are being undertaken. The travel line may have been underestimated at the project onset, but the overspending appears somewhat in contradiction with some countries’ lament of too limited exchanges with FAO project staff. Other budget lines appear as largely under-spent for such final phase of the project, as compared to their initial budget allocation, such as training (16% of planned budget spent) or general operating expenses (25%).The latter’s sub-spending could eventually compensate for the over-spending on expendable procurement budgets. 88. Notwithstanding the above, comparing the expenses as a share of the total expense to date against the budget lines’ weight of total budget at project planning stage (ref. Table 6), no strikingly high discrepancy arises, aside from the travel budget line.

Page 31: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

20

Table 6: compared weight of each account as share of the initial budget vs share of expenses to date

Account Description % against total budget% against total expenses

Contracts 24% 33%

Professional Salaries 29% 21%

Travel 8% 18%

Consultants 15% 17%

Training 16% 4%

GOE 7% 3%

Expendable Procurement 0% 1%

Non Expendable Procurement 1% 1%

OVH/Chargeback 0% 0%

Locally cont labor 0% 0%

Hospitality 0% 0%

89. According to the Project Management and GEF Coordination Unit, the financial management of the project is performed at different levels that guarantee a clear distinction between the day-to-day operations and administrative management by NRL and the overall supervision and controls which is carried out by the GEF Coordination Unit and the Budget Group in the Investment Centre Division. 90. It should also be noted that as part of the FAO-GEF Fund, the GIAHS project, like all FAO GEF projects, are subject to a biennial audit. Beginning in 2013, the FAO-GEF Fund will be audited annually.

4.2 Project management, structure and steering mechanisms

91. The global project management structure consists of the Global Coordinator, on a part time basis, and one technical officer. Both are supported by secretarial and clerical staff from the NRL division whom carry out such tasks as regular support staff to the division and its Director. The management structure at country level differs. In all cases national coordination is (formally) provided by counterpart staff, with a highly variable level of intensity of involvement among countries. In some countries regional coordination staff and technical project coordination staff at local level are FAO hired on a local consultant basis. 92. The project management structure for the project must be qualified as too lightly staffed in relation to the requirements of the project. Too many responsibilities were handed to a relatively small global project, even taking into account the negotiated co-funding. The HR crunch has applied to global management and coordination but is equally felt in the limitation to flexibly draw on external capacity, FAO staff member, and consultancy based and otherwise. 93. The only staff with permanent responsibility for the project is the assigned technical officer, who however is not scaled at a level allowing for independent project decision making. This is despite the intention of the project to field a full-fledged CTA to manage the global project affairs. Instead the formal global coordination responsibility was carried by the initiator of the GIAHS initiative, the BH, which is considerably occupied with other matters, being a divisional director. Moreover by combining the responsibility of creating a global GIAHS governance

Page 32: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

21

structure with local implementation and national lobbying, the project design was “saddled” with a variety of non compatible responsibilities and interest. 94. In reality, the technical officer has had to rely on the availability of the project Global Coordinator/Budget Holder for most of the fundamental project management decisions and all of the decisions concerning budget expenditure. Combined with the characteristics of a project where the global presence is already thinly spread, this has led to a weakening of management presence at not just the local and the national level, but also in terms of regularity of the global project’s exchanges and communication with national institutions and formal counterparts. 95. The overall management situation has further been compounded by the centralized budget control and decision making structure which has been maintained thus far. Although, not entirely impossible to manage a global project this way, transaction cost are automatically increased in function of the multiple and delicate needs for coordination between global, national and local actors. Besides, and this is borne out by the current project complex coordination arrangements can function in theory, but entail a risk factor and an additional management burden when they do not actually perform in practice 46. 96. Despite this a lot of visibility for the project has been achieved at global and national level through special events and workshops. Such visibility can however not be equated with hands on management presence to act in capacity of a regular sound board for national and local initiatives and activities. 97. The project, at this stage, appears to lack a formal and explicit linkage with the Organization’s overall Strategic Plan. This causes the project to operate in relative isolation as the inherent value of the project to the wider programme environment is not an established given. (Potential) project benefits must depend on project advocacy47. The above is partly a chicken and egg dilemma: a project with a novel and pioneering function automatically risks standing at the periphery of the Organizational structure, and moving it to a more centrally recognized position requires means to increase the linkage with the Organizational Objectives or adapting the latter in recognition of the importance of the initiative. Neither approach is easy to realize without sustained diplomatic and political effort. 98. Logically the project could have relied more strongly on the role of FAO Representations to extend its outreach at national and local levels. Several combined factors however prevented this from happening on a regular and satisfactory level. Budgetary delegation through the creation of baby projects never materialized. Even if, during past years the conceptual buy in from FAO Representations and their Representatives may not always have been optimal, the majority of interviewed Representation staff indicated that they could have played a more fundamental role in the project if so entrusted. Roles adopted by representations are uneven, however all have been affected by the fact that there was no specific budget allocation foreseen or budget control transferred for national and local support initiatives. Furthermore the adoption of a FAOR role in respect of GIAHS has also been affected by the aforementioned relative isolation of the project as a pioneer initiative, not fully anchored in the FAO programme structure or FAOR country programming frameworks. The evaluation has specifically been impressed with the articulate FAO staff responses in China, Tunisia, Peru and Chile.

46 One simple reason may be found in the earlier argument of an imbalance in involvement between national and local level, which prevents needed national level inputs from arriving timely at the local level. 47 Also within the Organization.

Page 33: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

22

99. Then there have been issues of capacity and leadership at the individual FAO Representations, having to deal with the GIAHS initiative. In some cases a project-financed national coordinator was appointed. In other cases the formation of such positions was rejected because the global project management considered it a national counterpart responsibility to pull the initiative, without much more than communication with the global coordinator. 100. When and where the project has attempted to strengthen its presence through the appointment of a dedicated coordinator, this has generally resulted in an intensification of information gathering at the local and national level. Project implementation decision making has not been affected by such arrangements as the budget control remained unchanged and the upper coordination arrangements still suffered from the same low level of empowerment. 101. The project steering mechanisms were duly established in the form of national and global steering committees. Depending on the pilot country national steering committees have played a more or less pro-active role in project follow-up and guidance. In instances were national governments have been more actively engaged with the GIAHS concept, such as China and Chile, the involvement and visibility of the national steering committee has also been more pronounced. 102. The evaluation has reviewed the role of the global steering committee as well. Especially during the conceptual and design phases the global steering committee has taken an active part in defining the scope and role of the GIAHS initiative. The evaluation has interviewed the chairman of the global steering committee to assess its more recent involvement. From this interview it became clear that the impact of the global steering committee on the global advancement of the GIAHS concept has been limited in concrete terms. 103. The global steering committee has contributed to the global profile of GIAHS, but it has not achieved in stimulating a more solid programmatic operational basis. Some of the instruments available to the global steering committee, such an established scientific committee focusing on an improved GIAHS selection mechanism have remained relatively stagnant during the project life. During an interview with the chairman of the global steering committee, one of the main forward looking recommendations was to put the selection of sites at national and global level on a stronger footing. This should be done using established scientific committees as part of structured National Important Agricultural Heritage System (NIAHS) pre-selection, which proposed sites for global classification by a GIAHS programme structure. The evaluation fully endorses this approach as practical and moreover country driven. 104. The chair of the global steering committee, himself a strong proponent of GIAHS in India, believes that it is FAO’s corporate responsibility to move GIAHS to the next stage of a formal programme. He considers this an essential step to make it really feasible and implementable to deal with the fundamentals, technical and administrative requirements of the global GIAHS initiative.

4.3 Technical backstopping

105. The section analyzing project’s efforts with regard to capacity building somehow may provide complementary information with respect to technical support. 106. In line with the findings presented above, the evaluation found that the coordination arrangements shortcomings somewhat affected the effectiveness of the project’s technical

Page 34: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

23

backstopping. As mentioned above, the actual decision making power of the project’s main technical officer has been limited due to the supervisory role of a high-level manager over the project, who by his function could not be engaged enough at the field level. This implied that the technical project staff also had limited time for technical backstopping due to heavy workload on day to day project operational issues. This project governance arrangement may be a cause for the project having lacked concrete management presence to provide a sounding board for national and local initiatives and activities. Stakeholders in pilot countries indeed regretted that the useful exchanges with the global project coordinator had been too sporadic. 107. The project’s limited explicit institutional anchorage also had an impact on the options available to project management to draw on broader Organizational capacity. Effectively this has been an impediment to the project’s stature and internal networking possibilities. It thereby remains partly locked out of the technical and normative function structure of the Organization. 108. The proposal to establish a statutory secretariat goes in the sense of improving the technical support available to provide practical and meaningful inputs into GIAHS selection or designation. One can expect that the initial organizational, networking and technical demands on the secretariat will be substantial, and may need to be supported through a taskforce structure. 109. Relative to the project’s objective (outcome 2) to “mainstream conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS in sectoral and inter-sectoral plans and policies in pilot countries”, the evaluation noted that a few technical consultancy missions have made attempts to touch upon sub-sector specific national policies. Specifically this could be mentioned for tourism related investigations in the Philippines. 110. By and large, technical support has mainly been provided at local level. However use of nationally and locally recruited temporary assistance has led to variable results, with in some instances pre-project interventions using the national consultants involved before. In that sense FAO as an institutional repository of technical knowledge has played a less significant role. 111. The involvement of FAO representations in visited project countries was judged to be limited, FAO representations themselves justifying this by a lack of capacity, though they also all expressed a sense of under-valuation and under- utilization of their national/ local insights and potential service support. 112. With respect to the local-level objective48, the evaluation has not come across convincing evidence of actual systematic screening and testing of “new” technologies, though some small NGO type technical interventions has been witnessed. Unfortunately in the majority of instances these appeared neither technically innovative nor economically viable. The evidence collected led to think that too often applied project interventions were insufficiently taking into account the fundamental socio-economic and environmental drivers affecting GIAHS. This leads the evaluation to conclude that there has been an insufficient level of coherence in the underlying analysis, possibly mirroring the absence of strong technical backstopping.

48 To “sustainably manage globally significant agro-biodiversity by empowering local communities and harnessing

evolving economic, social, and policy processes and by adaptation of appropriate new technologies that allow interaction between ecological and cultural processes”

Page 35: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

24

4.4 Design of M&E system

113. All in all, the MTE Mission considers that too much of the M&E effort has been left to the global technical officer, who already had more than full plate dealing with day to day project management issues. This makes it difficult to estimate which percentage of the overall budget allocation has been spent on M&E related activities. 114. Moreover this project M&E arrangement puts into doubt the required independence of the system. The absence of better structured (SMART) baseline data and an adequate LF impeded implementing qualitative satisfactory M&E efforts (thereby affecting its potential impact). This is mainly worrying as the project was considered a learning exercise (outcome 4), to feed into a full scale GIAHS programme. The implicit message here is that M&E as a function of the global programme needs to be re-enforced. 115. At this late stage, any attempt to be initiated for strengthening project specific M&E capacity would come too late. The principal project outcome that should be advanced by the end of the project is a full GIAHS programmatic structure (outcome 1). Consequently the Evaluation is not in favor of leaving the creation of a more structured M&E capacity to the project, or to specifically plead for project extension to attain this. Rather systematic strengthening of M&E aspects as part of a continuous Planning – Monitoring and Evaluation (PM&E) cycle must become one of the priorities of an emerging programme structure. This is a concept that accords well with the principles of adaptive management central to the GIAHS initiative.

5 Project Results and contribution to stated objectives

116. Mostly the less tangible project results, such as the success in creating international exposure stand out as achievements. Exposure is a tool towards recognition and subsequent formal acceptance of GIAHS as an agricultural category and as a tool it does not fit exactly in the project framework as an outcome or output. Still it has had a positive effect on the overall viability of the GIAHS initiative. 117. GIAHS as it stands in 2012 provides a much better starting point for a broad based international programme, than existed 10 years ago. This is a considerable achievement against the strategic hurdles met and implementation constraints faced. What has become an urgent matter now is capturing the created momentum on the basis of systematic arguments as to why the programme structure is worth pursuing. 118. The limited and as yet not formalized international recognition resulting from the project is as such only part of the road to be travelled. For the preferred scenario of turning GIAHS into a permanent FAO programme, broad based member state acceptance through Council and Conference agreements is essential. Before that, basic corporate issues such as making budgetary reservations for the next biennium must also be agreed upon. 119. There are several reasons to plead in favour of a programme status. First the original argument stands solid: GIAHS as a concept deals with real, unique and rich eco-systems which a

Page 36: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

25

valuable to preserve.49 Beyond that, and as stressed by many interviewees, GIAHS provides a refreshing combination of entry points and a holistic take on dealing with important global issues in relation to agriculture and rural development. FAO´s focus on such systems as an excellent opportunity for extending corporate advocacy and trans-disciplinary cooperation at national and global level50. Valuable lessons of do’s and don’ts can already be learned from the current project initiative in this respect. 120. The concept does so by adding conceptual value to the development of sustainable rural economies, and their improved integration with the overall national economy. GIAHS recognition will also lead to the promotion of farmer to consumer trade linkages and support to the development of niche markets for GIAHS products51. 121. Other direct benefits may accrue from GIAHS contributions to enhanced regional and local food security52. Defining combined productive and alternative service aspects which GIAHS53 environments may provide has the potential to sustainably strengthen the rural employment basis for at least part of the rural population thereby mitigating rural urban migration.54 122. Whether these combined traits can generate sufficient momentum to halt and reverse decline of such systems is not a certainty. To begin with GIAHS global support could ensure a more accurate valuation of such services as the basis for decision making, development of conducive government and investment policies. An important aspect must also be comparison of GIAHS support with alternative strategies of managing a continuing outflow of the rural populations from most GIAHS and other traditional agricultural zones. 123. By taking on board the importance of production and services in a self-sustaining manner and linking these were possible with local private sector initiatives the GIAHS approach does certainly offer the potential to go well beyond classical development-oriented and subsidy driven interventions. 124. From the FAO corporate perspective, GIAHS can be an important unifying programmatic driver. As argued in this report, GIAHS has started to demonstrate that it has the ability to link economic production interest with conservation, and that this has an impact both inside and outside the agricultural domain. 125. The fact that GIAHS may include a wide range of production environments of immediate interest and concern to the Organization lends the character of GIAHS as a microcosm of FAO’s mandate. The global and relatively practical nature of GIAHS allows it to be a ‘centre of condensation’ or backbone structure for normative, technical, legal, diplomatic and operational support inputs.

49 From the point of view of diversity including Bio-diversity, which once lost will be difficult if not impossible to

replace. 50 Including the possibility of enhanced cooperation among UN Agencies. 51 Among others relevant for the Gastronomy and Organic produce segments of the market. 52 Maintaining agro/biodiversity having a direct positive correlation on agricultural production stability by spreading

risk to crop failure as a consequence of climatic unpredictably. 53 Tourism including local accommodation and catering, Payment for Eco System Services PES, and Rural

Retirement Services 54 This is not to suggest that traditional employment levels of GIAHS may be maintained. However through

conglomeration of land holdings and appropriate levels of mechanisation a more acceptable and viable rural employment basis may be supported.

Page 37: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

26

5.1 Achievements under Outcome 1: An internationally accepted system for recognition of

GIAHS is in place (Global)

126. At the present time an internationally accepted GIAHS system is not yet in place. This is despite a substantial amount of debate, mainly within the project environment, which has taken place concerning the shape and merits of such a system, as well as tireless efforts by the global coordinator to promote the idea of an elevated FAO-GIAHS status with corporate management. 127. Part of the discussion involved weighing the merits of aiming for a GIAHS specific International Legal Instrument. Early on in the project and as explicitly stated in the FSP document, it seemed that the creation of such an International Legal Instrument was viewed as desirable. The likelihood of achieving a GIAHS international instrument by the end of the project was however questioned. 128. At project management level there has been increasing expression of doubt about the actual value of aiming for a formal international Instrument covering GIAHS as an agricultural category. It is argued that because of the long negotiation process involved, and the often tedious operational mechanism behind such a convention or treaty, added value to specific GIAHS will be difficult to demonstrate. The evaluation agrees with the above management view as the process of obtaining such an instrument, apart from consuming a lot of time and effort would likely also lead to a more bureaucratic engagement between a GIAHS programme and the individual sites. This is not desirable in view of the flexible umbrella support which is required to boost the conservation and adaptive management of individual sites 129. An intermediate solution would thus be, as mentioned earlier, to have a permanent FAO GIAHS programme with a statutory secretariat. This arrangement would lend a lighter but still formal touch to the GIAHS initiative and will not prevent individual sites from acceding to existing international instruments, national and international certification schemes. Thus elevation of GIAHS to a FAO programme status, is the logical choice as it status would formally embed GIAHS in the FAO programme of work, and would ensure continuity of the initiative based on Organizational core funding. 130. The evaluation was provided with several indications from the side of FAO corporate management that the Organization is now ready to consider the formal creation of a GIAHS programme. Although coming late in the life of the project which was supposed to be instrumental in leveraging a similar process, this is certainly a positive development. Meanwhile based on the project track record to date, the evaluation estimates that its ability to lead the process of elevating GIAHS to programme level is limited. The currently involved technical project staff is too tangled-up in daily management issues and lacks authority to act on behalf of the Organization during high level meetings with member states. Similar the accorded status for the technical officer position make it unlikely that this staff can effectively negotiate the merits of programme elevation within the Organization. 131. Additional dedicated HR inputs, both within the Organization and through consultant support will be required during the remainder of the project life55, in order to provide a fair chance of making a successful transition from the current project status to a new FAO programme. While

55 Appointment of a staff member focal point

Page 38: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

27

the suggestion for the need of another project to achieve programme status within FAO is a hard sell, a budget neutral extension of the project may be useful to guarantee the continuity of the initiative until programme status is achieved. This may require the strategic re-thinking and prioritization of remaining project budget resources. Allocation of additional strategic HR inputs to internally leverage the conditions for GIAHS programme creation must transcend the current project level as it will require regular and direct access to FAO corporate management for the duration of programme establishment56. 132. Aside from the views of member state and internal negotiation results, which are impossible to predict in advance, prospects to aim for a GIAHS programme status within FAO seem positive. Consultations with National Government representatives in countries visited by the evaluation mission, have with varying degrees of urgency all expressed their opinion that a FAO GIAHS programme creation acts as a major condition for their continued interest and participation in the initiative. Participating member states share legitimate reasons to value formal and long term international GIAHS recognition. 133. International recognition acts as a condition in the process of demonstrating and advertising GIAHS to the world by lending it credibility. As long as the GIAHS initiative is basing its status on being part of a collection of associated projects, this will be difficult to achieve. International recognition is also directly linked to the mobilization of national and international funding and technical resources required in support of conservation and adaptive management of these systems into the future57. 134. GIAHS when nationally integrated and managed as a viable sector category can develop into an effective international branding tool. GIAHS holds the potential ability to showcase overall (agri)-cultural diversity and richness of a participating member state. This can be remarkably effective in providing a more positive image to the national agricultural sector as a whole. GIAHS can in this sense be a flag bearer for national policy efforts, which want to demonstrate a balanced approach between ecosystem preservation and development. Demonstrating a conscious attitude towards this trade-off is important in developed and developing economies alike. 135. In the course of the MTE, and especially towards its end, the prospect for the establishment of a full-fledged permanent GIAHS programme had however dramatically improved. FAO institutional capability to provide international and national support to GIAHS also has gained a more favourable outlook. Unintentionally the MTE may itself have contributed to this shifting prospect. Specifically some of the higher level ministerial interviews in various participating countries entered into the subject of GIAHS conceptual and institutional sustainability and potential strategies to arrive there. The principal trigger can however be attributed to demand by member states for up scaling and continuation of GIAHS (as a programme). This has set into motion a train of events within FAO itself, leading it to take formal process steps for GIAHS programme elevation by 2013.

56 It is key that at the earliest possible date a de-facto cross-departmental taskforce for GIAHS be created. While the

official status of this task-force can only be guaranteed after programme recognition, its services are required to set the stage for this programme by drawing in broader corporate inputs for elaboration of programme design.

57 Funding is however usually not quoted as a main concern per se. Currently international funding is more seen as a substitute and temporary replacement for international recognition, which is so far absent. The approach of the Chinese Government to GEF for financing of a national GIAHS initiative must also be seen in this light. The approval of such a project will act as a temporary safeguard for continuation of the initiative under an international umbrella, once the current project ends.

Page 39: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

28

5.1.1 Achievements against Output 1.1

136. Output 1.1 was formulated as follows in GIAHS Programme Development: Public endorsement of the GIAHS concept, definition and criteria by key international institutions and pilot country governments. 137. The evaluation notes first that the output is not formulated in the correct output specific terminology; a remark that applies more generally to the whole project framework. In addition, the concept of “Public Endorsement” is difficult to specify and measure as an output. It is not stated whether public endorsement is a formal act of recognition and acceptance and what are the conditions attached to such a formal recognition. It might be that at the time of project formulation there was insufficient clarity about what was intended with public endorsement. If this were the case, it might however be expected that at the current stage of the project more detail would have been supplied about the intended achievements for this output. The evaluation has not retrieved any evidence that this is actually the case. 138. Still it is safe to state that this output has not been concretely achieved. It is clear from the project’s own analysis under PDF-B that GIAHS is a concept that first and foremost needs to be carried by FAO as an Organization. To date, and as part of the project impact and efforts by the global coordinator there is mention of GIAHS in forum such as the CBD58. References made in the past are difficult to take for granted in the future, without making the initiative more permanent and formally established as a part of FAO. Obtaining endorsement and (formal) recognition by international organizations, conventions or treaties outside those governed by international bodies other than FAO, simply requires formal FAO endorsement and corporate support. 139. Although not impossible for an individual project to leverage the emergence of a new FAO programme must be considered as a feat of serious magnitude. At present there is the requirement for a programme joining the Organization’s diverse normative, operational and diplomatic capacities to strategically link and add value to several of the Organizations Strategic and Functional Objectives59, using the GIAHS initiative as a focal point for action. To obtain international endorsement / recognition the sharing of such a vision by FAO corporate management is a first crucial element that counts. The latter will act as a driver for process facilitation. Beyond that it lies however with the governing bodies, their membership / country representatives and other concerned international institutions to agree on formal recognition. 140. The establishment of a permanent programme structure would have required more intensive lobbying within the FAO corporate structure, based on an effectively coordinated strategy carried by programme champions within the Organization. In reality those efforts directed towards establishment of formal programme structure capacity, including political level exchanges with member states, potential donors and FAO corporate management were almost exclusively carried out by the Global Coordinator. A broader platform of corporate engagement and public advocacy is required however to secure a stable status for the GIAHS initiative. The current MoU’s with other organizations are helpful in the context of the project, but will lose their weight in future as FAO is the asking party for wider partnership arrangements. Partnerships and their continuity tend to be as strong as the level of common interest that binds parties. For this to occur sustained corporate management advocacy of partnering benefits springing from the GIAHS is required.

58 COP-10 59 List SO and FO that are Relevant

Page 40: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

29

141. HR limitations mentioned in the preceding paragraphs have played a great role in limiting the spread of the GIAHS concept outside the NRL Division60. The assigned technical officer lacked the formal status and mandate to carry the initiative effectively into other parts of the Organization. The Global Coordinator might have been able to do so, had there been opportunity to spent more dedicated time on this effort and on convincing others that GIAHS is a potential unifying instrument for the field programme. In house interviews confirmed that the GIAHS initiative was sensed as strongly being personally owned. Nevertheless, the same FAO interviewees all distinguish what is considered an obstacle of political nature and the conceptual and technical merit of the GIAHS initiative61. 142. It is important to note that there is considerable buy in for the concept while at the same time the project has not yet been able to create a structured basis of corporate support. This makes the challenge less technical, less confined to a single division and more promising in the face of a dedicated corporate and member state lobbying effort. 143. A linked cause for the lack of programme development can be traced to the fact that over the last 10 year period GIAHS has always been pitched at the level of a project. This includes the GEF initiative itself. When the initiative manages to receive timely support from corporate management this will no doubt lead to de-blocking the current stagnation in elevating the GIAHS initiative to programme level. Project level initiatives lack in formal means to draw in structured corporate assistance. This presents a clear catch-22 situation. In order to introduce programmatic thinking and action one first requires acceptance of the programmatic value within the Organization. To gain acceptance however, it is important to be able to act as a programme from the start, and this possibility is most of the time not evident because no fixed resources have been committed for this by the Organization. 144. From interviews, a consensus emerged around the establishment of an inter departmental taskforce structure as conditional for the creation of a FAO GIAHS programme and from a technical point of view essential to capture the broad sub-sector characteristics and adequate technical involvement for GIAHS, by the Organization. Because of the overall organizational and individual dynamics within FAO the necessary momentum for a GIAHS programme has not come together or rather has not been marshalled. The evaluation considers it crucial that this step is now taken towards the end of the project. Even when the outcome of efforts cannot be predicted with certainty. It is the only way forward for the GIAHS initiative to effectively engage with the FAO corporate structure and Member States. 145. FAO should without further delay start on a systematically planned course (roadmap) to draft the contents for a GIAHS programme and statutory secretariat. It should do so in close consultation with current national proponents as well as other regional and inter-regional partners. The latter is required to ensure that the GIAHS programme is linked to all relevant international conventions and treaties, rather than itself embarking on the development a separate legal instrument.

5.1.2 Achievements against Output 1.2.

60 Including the lack of strategic use of available personnel 61 Across the organization

Page 41: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

30

146. Output 1.2 was formulated as follows in GIAHS Programme Development: Establishment of interim GIAHS Secretariat with a statutory mandate and Scientific Advisory Committee, as well as articulation of a process for designating agricultural systems as GIAHS. 147. The evaluation through its discussions with national counterparts and related ministries, conclusively established that a permanent and formalized international status of GIAHS is considered to be a key interest of participating Member States. Such status and a statutory secretariat are seen as conditional requirements to sustain national support aimed at furthering GIAHS conservation and adaptive management strategies and implementation. 148. Important secretariat tasks, as seen by member states include its empowerment and capacity for global GIAHS designation and setting in motion a formal certification mechanism. In this context frequently quoted roles would be to provide assistance for the preparation and processing of GIAHS for accession to existing convention and treaty instruments as well as various forms of product certification.62 149. To date, a project secretariat has been created, however insufficiently empowered to advance project ambitions for global GIAHS recognition and the establishment of a statutory secretariat. The understaffed and under-capacitated nature of the global project secretariat can partly be attributed to project budget reductions just prior to the time of its launching. A cut-back in GEF funding was instrumental to this despite earlier anticipated secure financial commitments. The budget allocations for staff FSP planned and actual spent can be found in Annex 7 (budget overview) 150. The establishment of a statutory secretariat itself would be a direct functional result of elevating GIAHS to programme status. Since, under the project, the latter process has not been completed, concrete steps for the creation of the statutory secretariat have not been taken or its future functions properly defined either. During the course of the project FAO’s legal branch (LEGA) looked into the pros and cons of a number of modalities, but is not conclusive on a specific choice. Thus far the discussion of how to mandate a statutory secretariat has mainly centred on the option of an Article XIV body under the FAO constitution. 151. Output 1.2 clearly refers to designation process for new GIAHS, which presumably can only take place once a formal programme has been established63. More general the evaluation has posed the question how the current project has prepared for FAO’s functions within the framework of an established secretariat. This question leads back to the overall programme function as the secretariat shall act as the principal channel of expressing these functions. 152. From discussions with the Global Coordinator, the evaluation learned that in essence the role of a future statutory secretariat should be considered along similar lines as those currently fulfilled by the project secretariat. It was also confirmed that involvement of field level implementation in GIAHS should be considered part of the future secretariat role. This tends to indicate that only limited thinking has gone into defining the differences between a programmatic and project role and that there exist a clear need to prioritize functions for the programme and its secretariat. Careful reflection regarding roles is essential to ensure that the secretariat will not

62 Including services 63 Designation and processing of GIAHS recognition certificates, as has taken place under the project is relatively

meaningless. The “certifying” agents are the BH and the chair of the global project steering committee. The certification does not imply any official International Recognition as it is issued in function of a temporary project.

Page 42: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

31

engage in duplication of national and local roles and shall act in a catalytic rather than an implementing role. 153. Following ten years of project preparation and implementation, the evaluation sees the need to bring the concept to a next level of maturity. The evaluation looks at this issue in the spirit of the original project design and true to the usual role of a global programme. In this context the secretariat role would be to deal with the truly global issues, which means aiming for programme harmonization, a universal structure for its labelling and generation of agreed and allowable system benefits. Inter alia the secretariat would have to emphasize the topics of system classification, conceptual, scientific / technical advisory and international networking inputs. Herewith, according to the evaluation the next programme phase of GIAHS should not need to get involved in direct local level implementation involvement. A link between global governance aspects for GIAHS and field operations clearly exists. However this must not translate into a single management mechanism covering global, national and local aspects all at the same time. The local level implementation experience by the project provides ample evidence about absence of comparative advantage for a FAO HQ based unit to deal with such interventions from a distance. What effectively happened in case of the current project amounts to a classical confusion of mixed roles and functions. In other words acting in capacity of the legislative, executive and audit / monitor all at once. What is needed instead is a functional separation of powers. 154. Clearly this includes the possibility for national FAO programmes to involve with GIAHS through independently managed64 GIAHS support initiatives overseen by FAO Representations. Such projects, as any other project support source would act in a complementing role to the global initiative. Simply, what is important is that the global programme structure does not get engaged in national and local level implementation. This automatically diverts its attention away from higher level strategic, institutional and global governance issues, and puts the initiative in an impossible self- regulating and self-policing position. Function wise, and as a contribution to the formal designation process, one of the initial roles of a statutory secretariat is thus to prioritize work on practical and meaningful international as well as national GIAHS definitions and explanation of its diverse benefits in the global, national and local contexts. Despite the accrued pilot experience, the body of experience for systematic designation of GIAHS remains limited and incomplete in its scope. This means there is a high priority for a statutory secretariat to mobilize a scientific advisory committee to give guidance to refinement of GIAHS selection, designation criteria and allowable external support processes. From the view point of technical and coordination requirements a secretariat needs to house or be able to tap into the basic HR skills required for such a mobilization. It is reasonable to expect that allocated secretariat staffing alone will be insufficiently equipped for this task. 155. This is especially the case, because the establishment of a full-fledged scientific advisory structure for GIAHS remains in its infancy. The initial organizational, networking and technical demands on the secretariat will be substantial. 156. FAO may house the open ended GIAHS programme and its secretariat. This organizational choice is not only justified because the initiative was started by FAO staff, but as discussed before GIAHS clearly fits and complements in the Organizations normative and technical mandate. By extension it is (more than) logical that the secretariat shall also be able to

64 As laid down in Country Programme Frameworks (CPF)

Page 43: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

32

draw on available FAO staff expertise and networks to help establish a stratified scientific structure65. 157. This structure should be designed in a stratified manner, drawing capacity from national scientific committees in GIAHS countries. While ultimate decisions of GIAHS designation will have to be taken at the global level, there would be no merit in centralizing the pre-designation and selection processes, which is better handled at the national and regional level. As the current project approach to GIAHS designation is still relatively arbitrary and does not measure the comparative importance of applying systems or weigh selection criteria in terms of relative importance there is an immediate need to re-visit and hone the presently used selection criteria. Aside from the evident bio-diversity, technical and cultural aspects, there is need for a robust methodology of GIAHS economic service valuation. 158. At the core of selection there must be a more solid overall system analysis than is currently the case. As mentioned the most crucial element which is missing, despite reference to this in the early project preparation phase is proper economic valuation of products and services produced by potential GIAHS. Factoring in such criteria for selection of GIAHS candidate systems starts with a more fundamental understanding of the economic drivers explaining the current system status, as these also provide an important albeit partial indication about future potential and role of the system. As well as the requirement for external support to maintain and/or improve this status. 159. This means that apart from improved assessments of the current economic status, the selection must also take on board the evolution trends of growth or decline of potential GIAHS. The latter clearly count as potential system threats but depending on the level of advancement of decline should also be a consideration in their selection or not. 160. Wider FAO support to secretariat functions must be generated through a taskforce structure. A structure which shall continue to be required at later stages of a full-fledged GIAHS programme. Support needed for the secretariat’s overall coordinating role on technical and scientific matters justify an explicit and continued taskforce support role. In the medium term such assistance is needed to formalize steps that follow the GIAHS designation process. More specifically this will include the development of GIAHS specific certification including the definition of specific benefits that will be the result of certification66. This is by itself a threat to the creation of a formal FAO GIAHS programme and statutory secretariat. To avoid a lengthy and potentially inconclusive debate early on in the process the evaluation perceives there is a need to address such concerns openly and seriously. In fact the flow of external support into GIAHS must be an integral part of the ensuing debate over GIAHS selection and designation. 161. By taking on board legitimate concerns over indirect or hidden objectives behind the international elevation of the GIAHS concept becomes a key element for preservation of programme viability. The issue of National and International funding support to individual GIAHS must be a central element of the programme development debate67. Other measures that may be

65 National and global scientific committees, with a clearly defined advisory mandate. 66 There is widespread demand among participating GIAHS member states for a form of certification. However there

is little agreement about the type of certification that might be required. Where some argue in favour of certification at the level of individual producers, others see more benefit from a system level certification with an umbrella function. The evaluation is inclined to be in favour of the latter, as it is less demanding a system and still allows individual producers to apply for more specific national and international certification accreditation.

67 Which in part may stem for GIAHS linkage to existing international conventions and treaties

Page 44: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

33

introduced are setting of clear boundary and monitoring conditions. These will help to create consensus about the fair use of GIAHS designation e.g.:

• limitation of GIAHS to an agreed (limited) percentage of the overall national agricultural domain or capping at a pre-determined allowable level of national financial support for GIAHS services based on a ratio of the annual agricultural GDP

• inclusion of a clear upper limitation for the individual scale of agricultural operations, based on economic productivity thresholds.68

162. Beyond such thresholds public financing will not be authorized for other aspects than independent system research and documentation.

5.1.3 Achievements against Output 1.3

163. Output 1.3 was formulated as follows in GIAHS Programme Development: Establishment of a sustainable financing mechanism and institutional support for consolidating and expanding the GIAHS approach as a long-term open-ended program. 164. As for the establishment of the Statutory Secretariat, a sustainable programme financing mechanism can only seen in function of the establishment of the long-term open ended programme mentioned under this output. The same applies for institutional support for consolidating and expanding the GIAHS approach. The evaluation questions the logic of linking such institutional support directly to the sustainable financing mechanism as is the case under this output. It sees institutional support as a function directly associated with the permanent global governance of GIAHS (or programme) and its executive vehicle the statutory secretariat. As such institutional support should not be directly associated with the funding aspect. Funding while important is the fuel and lubricant of the mechanism not it’s content. 165. The evaluation will therefore not dwell on the institutional support element under this output. This has been sufficiently covered under the assessment of outputs 1.1 and 1.2. The only cautionary remark the evaluation wants to contribute in this respect is that in future secured funding should not be equated with content based matters, as is strongly suggested in the phrasing of output 1.3. 166. The only way to predict the potential for sustainable programme financing the evaluation can go by is by assessing the project’s capability to draw in additional funding for baseline initiatives. This ability must be judged as increasingly positive. During the lifetime of the project GIAHS has seen a steady growth of funding commitments from both originally included sources69, as well as fresh initiatives.70 The role of the BH must be commended in this respect, as the success of generating fresh interest for the GIAHS initiative must almost exclusively be attributed to him. The individual nature of fund-sourcing is at the same time a threat to the future of programme funding, since no alternative champions within the Organization have been identified that can take over this role upon his imminent retirement.

68 Limiting the inclusion of GIAHS designated areas in function of economic productivity thresholds, avoiding that

large already economically viable and highly productive areas could become eligible for additional national support streams.

69 E.g. China and Chile 70 India, Japan

Page 45: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

34

167. The evaluation is aware that fresh funding proposals for the GIAHS programme itself are being considered by a number of nations.71 Time is opportune to request that the commitment and delivery of such commitments shall be tied to the establishment of an actual FAO programme. The evaluation urges both member states and FAO corporate management to ensure that funding contributions are pitched at the correct (program) level to set an example for the future.

5.1.4 Overall rating of results for project outcome 1

168. Insufficiently mandated managerial and coordination resources under the current project have effectively limited its capacity to engage with the national policy and global level. The continuation of this situation cannot be afforded in the period of formal programme establishment. Continuation would certainly lead to a lack of efficient preparation of participating and prospective GIAHS member states in relation to establishment of a formal FAO programme structure. Preparedness on part of FAO would also be negatively affected if project led process would continue. 169. Discussions with project staff and FAO management have highlighted that both the 2012 FAO Council Meetings and 2013 FAO Conference act as significant benchmarks to measure prospects and progress for the upgrading of GIAHS to FAO Programme status. Both occasions present a concrete test-case to judge the level of international support, among a broader range of Member States. 170. However the degree to which the project has been able to turn such international buy-in in an effective programme strategy is mixed. The fact that internally within FAO the GIAHS programmatic process has started late can somehow be seen as a blessing. It can be argued that only now the time is ripe to take GIAHS to a more permanent status. Also it may be seen as strategically favourable and positive for sustainability that this takes place as the result of a demand driven process rather than a strong institutional push by FAO. 171. To date, there is a lack of convincing evidence that a coherent FAO cross-departmental platform has emerged in support of GIAHS. The level of FAO preparedness to establish a new programme is judged as relatively low. This applies both to matters of conceptual content as well as resource allocation, all of which has to take place within a limited time frame72. In that respect, there is no time to be lost to proceed with the next steps of formalization of GIAHS. 172. It is clear that the international momentum for GIAHS programme formalization is opportune and may not be easily revisited once the present, near end of project, opportunity has passed.73 Pro-active involvement of other interested FAO member states is of utmost importance for the practical shaping of a permanent GIAHS programme. This must continue to take place in the context of the FAO programme committee, council and conference. Direct involvement of member states in discussing the programme scoping is equally important to make the final programme optimally relevant for national application and distillation of international benefits. At present GIAHS international support is based on a still relatively loose network of stakeholders. This limits the effectiveness of international mobilization for wider GIAHS support. Partly this is a typical “chicken and egg” dilemma, as the overarching global structure has not yet been

71 China and Japan 72 Remaining 73 Aside from the member state demand there appears to be a typical confluence of ideas and involved personalities

as well as a conducive management environment in FAO explaining this current situation

Page 46: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

35

formalized. This also results in uneven progress of national GIAHS recognition among participating countries. The same variable impact applies among GEF project pilot sites and other associated FAO baseline projects. 173. There is a real risk in relation to this relative narrow institutional base of GIAHS within FAO. The project for a variety of reasons has been unable to shed the image of GIAHS, being a limited (FAO) divisional concept. This means that, in a formal sense, the wider conceptual relevance of GIAHS to the Organization has not been sufficiently carried over. This may form a potential obstacle towards a broad corporate initiative to effectively elevate GIAHS to a programmatic level and formalize international recognition. Although at present potential corporate support is neither effectively nor efficiently harnessed, the evaluation found articulate individual support for GIAHS as a concept. This assessment is based on a wide range of individual interviews with FAO staff74, both new to GIAHS and with past association to it. It remains however mostly the presence of a set of strong external drivers which lends credibility to the prospect of an efficient push for the creation of a FAO GIAHS platform75. 174. Based on GEF Evaluation rating the project results under outcome 1 can, as of present, be rated as Moderately Satisfactory. This rating is justified by the fact that despite delays in achieving the project outcome at the global level, there still exist opportunity to redress this situation towards the end of the project lifetime. The interaction between outcomes, mainly that between outcome 1 and 4 ensures that the visibility of GIAHS as a concept has been considerably enhanced at the global level, leading to the setting of positive conditions for an Organizational institutional breakthrough.

5.1.5 The way forward

175. Generating FAO internal momentum requires a systematic corporate exercise to re-visit and re-cast GIAHS’s (potential) role in relation to mainstream issues of the FAO corporate agenda e.g.: family-farming, food security, agricultural diversity and climate change risk mitigation. Both elements: managing external demand and initiating broad based corporate reflection on the GIAHS concept are essential ingredients to successfully leverage GIAHS permanent programme status76. FAO corporate resources must be strategically mobilized to refine the arguments and give content to formalized FAO GIAHS programme. A GIAHS programme must draw on the collective FAO technical, normative, legal, diplomatic and communications capacity. This process requires concerted well managed inter-departmental inputs. 176. Cross cutting managerial and operational themes to be addressed are: 1) updating of the GIAHS strategic agenda77, 2) outlining diplomatic and negotiation processes with current and aspiring GIAHS programme countries78, 3) delineation and preparation of a formal GIAHS programme mandate and plan or operations79.

74 See Annex 4. List of people met across various departments. 75 Mobilized demand from participating member states. 76 Of a critically constructive nature 77 Including far more systematic research and including introduction of economic valuation criteria and tools for

GIAHS, able to cover a broad range of GIAHS environments and scenarios. 78 Despite the fact that GIAHS has developed selection criteria and has refined these over time, the accession

mechanisms is still loosely organized, based on personal connectedness and insufficiently transparent for countries to take the initiative of joining GIAHS.

79 Specifically boiling down to the role of the – to be- statutory GIAHS secretariat and proportional to the estimated funding resources for such a secretariat.

Page 47: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

36

177. The inter-departmental agenda associated with GIAHS elevation to programme level will, as usual translate in the established of a taskforce, subdivided in specific working groups. It is recommended by the evaluation to establish these working groups already on a temporary basis as part of the programme design process. This may also act as a basis a more permanent taskforce structure beyond formal establishment. Suggested working groups are those for: a) GIAHS Application, Selection and Designation Criteria, b) Enhancement of GIAHS Economic Valuation Instruments, c) Definition of GIAHS Economic Boundary Conditions, d) Private Sector Role for GIAHS Production and Services, e) Value Added and Retailing (including product certification), f) Legal Definitions, g) GIAHS corporate communications management. 178. In order to maintain process focus on preparation of GIAHS programme status. NR Department Management is recommended to mobilize incremental, dedicated but time bound HR Resources Funding may draw on remaining cumulative project funding for GIAHS80. Concerned FAO staff member having direct access to NR management, and acts as the Institutional focal point for GIAHS programme preparation81. Apart from anchoring GIAHS programme development within the NR Department the Evaluation advocates use of external, stand by process facilitation and management support inputs. The latter must be consultancy based and work in direct liaison with the corporate institutional focal point. External facilitation serves to add agility and flexibility to the programme preparation process and will diminish risk of slow-down of the interdepartmental dialogue. The latter might more easily occur when only FAO internal HR is relied on. 179. The global programme component and future secretariat certainly has an important role to play in further defining and elaborating the conceptual framework and analysis requirements for GIAHS designation. To optimize the position of FAO in terms of its conceptual, normative and operational roles it is recommended that these respective roles are better distinguished between and defined in terms of centralized and decentralized responsibilities. Concretely this means that the programme should occupy itself with the first two tasks, but stay away from direct operational involvement. 180. It is clear that with a programme framework in place the local interaction with National Counterparts and GIAHS sites needs to be delegated to the FAO Representations. When applied this recommendation will only help to enhance the desired national ownership aspect.

5.2 Achievements under Outcome 2

181. Outcome 2 was formulated as follows: The conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National). 182. Besides two high quality studies on the international policy and legal dimensions related to GIAHS82 the evaluation has not come across substantial project documentation engaging in

80 Not just the GEF project but other resources dedicated to the advancement of GIAHS. It is understood that several

donors are interested to commit fresh resources into the process, and this is an excellent occasion to do so. 81 This function is distinct from the current project secretariat, which will continue in the intermediate period with its

current project role. Close coordination with the project secretariat is however essential to ensure sharing of project experience for the permanent programme.

82 Also mentioned under the chapter project findings.: Harrop and Mr. Leandro Moura da Silva, 2012

Page 48: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

37

systematic analysis of existing national policies, legislation and regulations of relevance to GIAHS. 183. Neither did it find convincing evidence of project impact on mainstreaming of GIAHS in sector and inter-sector policies. There are examples of specific local level decrees pertaining to GIAHS. These do however not count as part of a mainstreaming process. Such decrees are more of an administrative necessity for the release of local government funding contribution. Because the achievements demanded under outcome 2 can simply be considered unrealistic and over-ambitious, the evaluation is not surprised about the general result for this outcome. Although common practice in many development projects actual policy, legal or regulatory influence by small individual projects is relatively unheard of83. Therefore much of the further evaluation assessment of this outcome leads back to project design rather than implementation. A basic informed view regarding domestic political, institutional / administrative processes could avoid any undue pretence of external actor influence over policy development. 184. In the main expressed areas of concern under this outcome are conservation and adaptive management. However extension of GIAHS as a subject all such policy and legal elements as spatial planning, land tenure, agriculture, trade, marketing, product certification must also be covered. Central to the work of the evaluation’s national consultants has been an investigation of the body of existing policy, legal and regulatory material pertaining to the above fields. This exercise was introduced in consultant ToRs for two reasons:

• One: Initial desk study prior to country visits revealed that there was little country specific information in regard of national policy environments. For this reason the evaluation felt it important to acquire sufficient detail, which would allow it to grasp the complexity of the national policy, legislative and regulatory environments84. Listing of organized relevant uncovered material in the national consultant reports, in this respect speaks for itself.

• Second: The evaluation wanted to conclusively demonstrate that before launching policy ambitions, even if these considered unrealistic, should be preceded and based on systematic stocktaking of what is already there, and importantly, that such investigations can be carried out within a relative limited time span, provided such work is based on clear directions and carried out by well versed national consultants.

5.2.1 Achievements against Output 2.1

185. Output 2.1 was formulated as follows: Drawing on PDF-B assessments, identification and implementation of specific measures through which sectoral and inter-sectoral policies and regulations can be improved to support conservation and adaptive management of GIAHS. 186. The advocated approach of workshops and working through National GIAHS champions has actually been attempted by the project in all participating countries. Unfortunately the project design has failed to recognize that the proposed project tools are no substitute for normal national policy and legal development, let alone that project guidelines would take the weight of national regulations. Moreover the bulk of the proposed intervention areas, such as environmental management, rural development, product development and marketing, education have much broader implications than those sought for GIAHS specifically as is the case here. Because of

83 GEF project resources per country rarely exceeded USD 600,000 84 Although this complexity varies from country to country, in all cases it involves multiple ministries / agencies:

Including those covering Environment, Agriculture, Forestry, Rural Development, Interior, Finance, Justice, Trade and Commerce

Page 49: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

38

these wide implications and the required broad national institutional involvement, what is proposed under the project design, stands far from national policy and governance reality. 187. There is no doubt that PDF-B assessments have correctly identified the complex range of policy and legal issues impacting on GIAHS environments. Therefore on this point there is no need for discussion regarding the complexity of issues affecting GIAHS. However the issue of (lacking) realism immediately comes to the forefront when talking about the ambitious result expectations for the current output. The following paragraphs will outline the why behind this statement in more detail. 188. The PDF-B carried out (institutional and local) participatory processes to identify among others policy, legal and regulatory bottlenecks for GIAHS conservation and adaptive management. This does not mean that such exercises went in each case to the core of each influencing subject. In addition it can be argued that the PDF-B did not manage to exhaustively deal / engage with all relevant stakeholders involved with the broad range of policy and legal processes. Quite the contrary exercises were either facilitated by international or national consultants whom were not an integral part of national policy environment. The main national institutional interlocutors were Environmental Ministries / Agencies and not as would have been logically required for the range of subjects addressed players as the Ministries of Agriculture, Economic Affairs, Trade, Education, Culture and national Heritage, just to name a few. 189. When it came to FSP implementation the same institutional affiliations continued for the majority of participating member states85. Based on discussions with the Project Management, the evaluation wants to state that the institutional choices, made at the start of the project, may to an extend have been inevitable. It should however also not be excluded as a possibility that FAO invested too little in the negotiation of more appropriate institutional affiliations. Factors that defined the choice of national counterpart agencies were simple; such as the national GEF and bio-diversity focal points residing in Ministries of Environment (MoE) and initial ambivalence about the GIAHS concept in Ministries of Agriculture (MoA). 190. With certainty it can however be stated that MoE’s although having a policy interest in the GIAHS subject are not well placed to exert influence over the broad range of policy domains. Typically MoE’s deal with specific environmental issues86 in a cross cutting fashion and in “watch dog” rather than an implementing capacity. This makes these agencies are not well suited to deal with policy and legal aspects of territorial land tenure issues, economics of agricultural production, processing and marketing or most non-production related agricultural services. 191. On the project side a range of other obstacles did present themselves, many of them standing in the way of basic National level policy interaction. Key issues worth noting are: a) low frequency of interaction with authorities, b) lack of a focused policy and stakeholder approaches at national level, c) limited structural involvement of FAO Representations in policy debate and d) insufficient mandate and empowerment of immediately involved project staff to deal with fundamental policy issues.

85 China and Chile are the main exceptions. In case of China the Ministry of Agriculture (MoA) became the formal

national counterpart for the project, while in case of Chile the Ministry of Environment (MoE) handed over the de facto counterpart responsibility to the MoA

86 Environment of different colours (Green, Blue, Brown, Grey)

Page 50: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

39

192. The above resulted in a relative cumbersome interaction mechanism between the global project and national counterpart ministries. Not of the least importance, strong emphasis in the project design towards local interventions has effectively competed for time and HR with higher level national policy engagement. 193. On a more positive note the evaluation can note that project management and coordination realizes the importance of more regular and pro-active strategic engagement with national governments over the subject of GIAHS. 194. Evidence from country discussions also taught that while national level policies have rarely been affected by the project, over the last ten years, the dynamics surrounding GIAHS in many countries have taken a positive turn. It is in most instances however hard to substantiate that the project outcome 2 has played a role of significance in this. Outcome 4, dealing with communication / publications and lessons learned has been of certain influence. Equally the dedicated personal attention and high level exchanges, for which the BH has been an instrumental agent, have had a positive contribution. 195. Aside from this, they are mostly varied national developments, often independent from the project87, that have asserted the most fundamental change in attitudes pertaining to GIAHS. This is especially the case for key ministries in relation to GIAHS, but not necessarily those that acted as national counterparts. These influences are diverse and among countries uneven in scope. Moreover these have not yet resulted in new policies or legislation88. Such developments are however no less important as what the project originally set out to do under outcome 2. 196. An increasing and focused interest on the subject of GIAHS can be recorded in China. In recent years the balance between infrastructure development and eco-system conservation has gradually shifted in favor of the latter. GIAHS as a concept has directly benefitted from this in terms of institutional attention. Another main interest by the Chinese Government is using GIAHS to help re-adjust the global perception and image of Chinese Agriculture. The diversity of unique remaining GIAHS in the country forms the most important asset in this respect. 197. The rapid rise of the gastronomy sector as a defining part of Peruvian cultural identity holds good commercial prospects for better integration of producers in Peru’s GIAHS locations. This is a market driven approach which could have been more highlighted by the project, and can still form an aspect of facilitation under a future FAO programme. 198. In Chile, unlike China, there has been an increasing realization by policymakers that agricultural diversity as it remains today is important to preserve. Generally the country has been considered a model of agriculturally driven development, product placement and international marketing. However it is realized that in the process much of the regional uniqueness of the country’s agriculture has diminished. As an international platform, to stimulate national conservation of remaining assets GIAHS is positively appreciated.

87 E.g. important trends of elevating Andean Gastronomy to new heights, making use of unique local products

contained in and produced by GIAHS 88 An interesting initiative such as the Charter of the Oasis in Tunisia holds potential of influencing legislation, but is

so far mainly based on the work of a single legal specialist in the MoA of Tunisia. This individual pleads for increased international leverage by FAO to help move the legislative process at national level ahead.

Page 51: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

40

199. In Tunisia, despite the very small scale of the project, there has actually been an initiative linked to the key Ministry, MoA89. The drafting of the Charter of the Oasis, with much wider implications than the project pilot area, extending more in general to the Maghreb region, is an example of creative use of project resources. Given the post-revolution turbulence and overriding social issues such as high (youth) unemployment the Charter may however not have an immediate policy impact. Nevertheless this initiative is the basis of further future dialogue. At the same time it is quite clear that in order to increase the priority of GIAHS as a concept there needs to be sustained International engagement and support to help facilitate internal National processes.90 This was stressed several times by the principal author of the Charter. 200. In case of the Philippines, there is little evidence that inside or outside the project sphere of influence there has been much impact. Aside from the firm hold of the Environmental Department91 on the project, the complex relationship between the global project and the FAO Representation, the essence of this issue is that GIAHS has been treated as any other project, with very limited concern for sustainability of the approach.

5.2.2 Achievements against Output 2.2

201. Output 2.2 was formulated as follows: Development of capacities of national-level institutions to mainstream GIAHS in sectoral and inter-sectoral plans and policies. 202. Under output 2.2 national capacity building appears to be almost immediately equated with training. This is interesting as capacity building, and certainly institutional capacity building, has long been known to be much more than training of individuals92. 203. The core element of the training: “carrying over the GIAHS concept” is moreover an example of externally driven content matter. After all, even if GIAHS as systems have been around for long, the current proposition to engage with these systems in novel ways will, for many civil servants and others, be new and foreign. A second observation for output 2.2 is that it suggests use of more or less the same tools i.e. workshops, policy briefs as the preceding output. This begs for an explanation about the actual distinction between the two. 204. The weak level of differentiation between outcome, outputs and associated tools, make it on the other hand unnecessary to address the bottlenecks and conceptual flaws of the proposed approach. This was after all addressed under the previous section covering output 2.1. It merely requires reiterating that the proposed pathway and instruments are unlikely to result in fundamental changes of views, individual attitudes, policies and legislation. A concerted strategy to mobilize national lobbying including a well-defined role for Representations in such an effort is in demand.

89 A legal expert within the MoA was commissioned with drafting of the Charter. 90 This is very different from identifying and implementing policy initiatives through a project, it means taking a

catalytic role not driving the process. 91 DENR 92 To build institutional capacities sound capacity need assessments are required. Capacity building needs to be an

integral and focal aspect of national policy, it must be reflected in the prospects for individual career development through appropriate incentive structures. This is needed to build the capacity of the right people in the system. Capacity building is also known to be a long duration enterprise, not an interventions easily pulled off by a project.

Page 52: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

41

205. Nevertheless it should be stated that the project within its limited means has made an attempt to providing training and the creation of exposure to new ideas of individuals in the Government administration of participating countries. The diligent listing of project provided workshop events and training is a testimony to this. It would take too far for the evaluation during the limited time it has spent in each country to carry out a full scale training impact assessment. In any case it would be expected that the project, would have carried out such exercises. This is not the case. 206. Training impact assessments are however, not what is immediately needed to establish the project impact under this output. The overall success in leveraging national policies can act as a good proxy indicator. At that level the evaluation can establish that project training has had no discernible institutional impact. Simple matching of project resources, even if the GEF budget had not been reduced by half, would have been sufficient to show that under no circumstance it would have been possible to deliver on the full range of subjects under this output. As mentioned at the start of the discussion of outcome 2, the evaluation does attribute the inconsistencies related here not to the project implementers, but to design flaws93, which against better judgment available went unchanged. 207. The evaluation attributes the failure to carry out adequate reality checks on the project output delivery mechanism to a sometimes insular approach to project design. The evaluation has among others come across documentation produced by the end of PDF-B which outlines requirements for institutional learning94, including stakeholder analysis and dynamics. Although not directly focused on institutional capacity building many relevant aspects for the same are covered. 208. In case of more delicate issues such as national policy dialogue and efforts towards capacity building the evaluation opinion is that, internal FAO dynamics of this kind, can be ill afforded. Especially when, based on in-depth evaluation interviews several Representation staff have demonstrated sincere interest and commitment to furthering the GIAHS concept.

5.2.3 Overall rating of results under Outcome 2

209. National project activities have contributed to increased clarity regarding the global system selection criteria for GIAHS. Part of the national experiences has been positive others less so, both experiences count as lessons learned. Although the impact for the project itself tends to the negative, the impact for a future programme is neutral to positive. Negative impact in this case means that weakness of national institutional buy in has not been beneficial for increasing local level ownership or up-scaling local experience into the wider national arena. 210. National project interventions, and the manner these have drawn on information from local pilot systems have fed into the project communication strategy under output 4. The globalization of this information has at least partly fed back into the national experiences and helped in a rather non quantifiable way to raise profile of GIAHS at that level.

93 Thorough institutional capacity needs assessment, capacity resource matching, capacity programme design,

monitoring and regular updates of the same were not foreseen by the project. 94 This concerns the same project publication as mentioned earlier in connection with a scientific approach to

institutional dynamics: Developing governance mechanisms for agricultural heritage conservation, December, 2007, by Frank van Schoubroeck, Luohui Liang, Qingwen Min, Arend-Jan van Bodegom

Page 53: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

42

211. Through systematic interviews with Government officials in all visited countries as well as FAO personnel, the evaluation has established that the central role of environmental agencies have been a constraining rather than supporting factor for sectorial mainstreaming of GIAHS. On the one hand this can be seen as regrettable and pointing to low efficiency of resource use with corresponding limited effectiveness, impact and lack of institutional sustainability. Investment in the right institutional structures at national level would have been positive for institutional sustainability. Conversely the opposite applies when this is not the case. This opinion can be voiced when keeping the project to its expressed intentions of providing a sound basis and enabling environment for the future conservation, adaptive management and evolution of GIAHS. With a certain amount of goodwill the experience may also be considered as part of a normal institutional development process. In such processes it often takes more than one step to arrive at a well-timed and structured engagement involving the correct institutionally empowered national agencies. 212. The important difference between these two evaluation outcomes lies in the fact whether the collective experience is internalized in an effective manner and translated in measures having a concrete impact on the future identification, recognition and designation process for GIAHS. As long as the latter takes place the FSP experience amounts to considerably more than a missed project opportunity for advancement of national GIAHS recognition. 213. The FAO project management and coordination set-up and proportional under-staffing has resulted in an unnecessarily complex and layered involvement on part of the Organization. The first casualty of this has been limited effectiveness in communication between project layers. At times this has led to contradicting messages regarding FAO’s positions and views. This has had an especially adverse effect on project outcome 2, as this outcome serves as the hinge between the global project and local project aspirations. 214. The relative limited role played under the global project by FAO Representations has had further negative impact on advancing GIAHS as unifying programmatic factor of FAO’s country delivery. The careful protection of GIAHS as a globally managed entity has according to the evaluation missed the point of even testing Representations in their capacity to be a trusted strategic partner. 215. As a result GIAHS is now primarily seen as yet another self-standing FAO initiative. This is a missed opportunity to enhance institutional sustainability at the Organizational level. In few instances GIAHS is perceived to be the valuable agricultural sector entry point (read family agriculture, food security, and agricultural applied research) which it really is. During the evaluation many references were recorded about the operational hurdles which had to be taken to get to where the project is today. This does put a stigma on a project compared to a smooth running exercise. It is sad to note that the otherwise positive view was thus blurred as a consequence of often unwarranted fall out of cumulative project operational experiences. 216. As a consequence of lowered communication effectiveness, and too limited empowerment global project coordination options to effectively re-direct the strategy for outcome 2 have diminished over time. When project aims for national policy and legal issues are not within reach95, it is only natural that this leads to even further emphasis on the local aspects of the project, which is already prescribed in the project document and directed by the budgetary balance.

95 Incidentally this applies also to an extent for the global project outcome, as was corroborated by several

interviewees.

Page 54: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

43

217. However because of the inter-dependency of project levels, this then implies that local interventions will increasingly take place in isolation from the national policy structure. Resulting imbalances and disconnection between tiers are passed on to the output and activity levels. This has a two way effect; Local initiatives will not be informed by national policies and there will be very little in terms of an efficient upward flow of local experience into the policy domain. 218. It has mainly been the international and national consultancy inputs to study and document GIAHS which have enhanced visibility. Especially it has been the larger nationally organized and extensively reported events with strong FAO representation, not least that of the Global Coordinator which has had certain impact to move GIAHS more towards mainstream thinking. This is quite different from the intended nationally led approach. It is also true that such events have only been limited and far in between. 219. The overall logic behind the way of putting GIAHS on the policy and institutional map is doubtful and certainly not a truism. The implicit assumption that experience with national level project initiatives under outcome 2 would further be able to inform global strategy and approaches for rolling out GIAHS at a wider scale are also not effectively demonstrated. 220. The fact is that different countries apply a variety of implementation models for their national GIAHS. Differences vary between close involvement of the national counterpart96 and / or (delegated) government implementation to hardly any involvement whatsoever97. There are sites where direct FAO implementation through consultants plays an important role, and lastly implementation which relies on non- state actors (NGOs). 221. Applied project implementation (and institutional) mechanisms, an implicit variation in the institutional and political anchoring of the GIAHS initiative make it almost impossible to come to a relevant comparison of national institutional experiences or the effective translation of these to the global level. Still some limited experience can be distilled from the national level experience. It can hardly be seen as coincidental that in those instances where MoA’s were either the formal counterpart (China) or the counterpart, in all but name (Chile) the width and depth GIAHS national institutional partnership structures is more profound and better institutionally anchored. This can be considered a proof of elevated efficiency and effective institutional ownership over the initiative. 222. From a pure mechanistic perspective the evaluation would be inclined towards a negative GEF rating of this outcome. This rating would be based on simply ticking of the concrete achievements against the LF. This would however not allow for a weighed analysis and the consideration that parallel developments have taken place in relation to the area of influence of the project. The fact that there is evidence of increasing national ownership over GIAHS in a number of the core participating countries as well in the extended baseline elsewhere prompts the evaluation to rate this outcome as moderately satisfactory.

5.2.4 The way forward

96 China, Chile 97 Tunisia

Page 55: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

44

223. If the project were in a less advanced stage the recommendation of the evaluation would be towards more or less radical redesign of the outcome. Alternatively the output in its current form could be dropped altogether. In this advanced stage of project implementation it is recommended that during the remainder of its time the project takes renewed stock of existing national policy, legal and regulatory frameworks pertaining to national GIAHS. Inter alia this should also include possible instruments for system classification and certification. 224. This exercise should be carried out in close coordination with policy mandated national ministries, relevant legislative bodies, and national academic authorities. Process facilitation should be provided by FAO hired National consultants. This work should be carried out on a priority basis and should not merely focus on the national policy and legal framework matters. Equal attention should be given to how, within the legal framework cohesion between local level and national level interventions can be enhanced. 225. The integral output of this work should feed into and systematically expand the process already started by the Evaluation to document enabling policy and legal related to GIAHS. From a global perspective this information management process must be seek common denominators among countries rather than list country specific policy and legal issues. 226. A future statutory programme secretariat should have an advisory function regarding the key areas of national intervention which are important to comply with GIAHS designation requirements. This means that a future statutory secretariat shall not engage in pro-active measures for the adaptation of a wide range of national policy and legal frameworks. 227. It is not in the interest of programme efficiency and sustainability if a secretariat takes the role of actively helping to generate the base policy, legal and regulatory conditions for global GIAHS designation. In this context it is important that the burden of proof for preparedness be transferred to national governments. This should be based on expression of legitimate interest and fulfilment of a set of clear policy criteria to which the Government will commit over a fixed time frame. 228. It is however recommended that any future permanent secretariat shall positively respond to request for facilitation of on demand external assistance to review and analysis existing national policy, legal and regulatory instruments of participating member states. This type of service might be carried out on basis of official request of such members states against a nominal contribution to the GIAHS programme funding mechanism (to be established). This work by the secretariat should entail a gaps analysis based on set GIAHS designation criteria and written advice to member states for steps required during the further GIAHS accession process. In this context too little time is remaining for the current project secretariat to devise such a service structure. However the views of staff should count importantly in terms of advice. 229. It is further recommended that truly global instruments available to a future GIAHS secretariat, such as the convening of participants at a global level, exchanging country to country experience and the setting of selection and global governance mechanisms for the GIAHS initiative form the mainstay of its engagement. 230. It is also recommended that all matters of national interaction pertaining to contacts with national counterparts and GIAHS sites are delegated to the FAO Representations. When applied this recommendation will help to enhance the desired national ownership aspect.

Page 56: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

45

231. By no longer mixing roles FAO global, through the statutory secretariat can focus on delivery of the equally needed external analytical perspective for GIAHS. This re-vitalized and structured FAO role is acutely needed to provide a sound-board function for national and local GIAHS conservation and adaptive management initiatives. Apart from the aforementioned, the future of a strong GIAHS programme lies with an articulated global advocacy role of the statutory secretariat. This is key to help generate broad based national buy which in turn has to be facilitated by FAO representations under their regular programme of work. Even if current capacities of FAO Representations are not always ideally suited to convey the GIAHS message, this route must be taken and considered as the logical institutional pathway and an opportunity to build up delegated and decentralized capacity. 232. The evaluation recommends against making institutional changes for the remainder of the project duration. This would result in a rather disruptive and counter-productive exercise. Effectively it would risk landing the project in a new trap of bureaucratic hurdles. The Evaluation has used the opportunity of opening a direct discussion over appropriate institutional linkages at country level. It should be seen as a matter of strategic dialogue and continued discussion with current and prospective member states involved with GIAHS to get institutional affiliation right in the future. This will require solid negotiation on part of FAO. 233. The evaluation proposes that while the autonomous choice of member states to select national counterpart agencies should be respected, FAO on its side should at least engage in functional assessment of the proposed arrangements. Also the discussion over these arrangements must be made a full part of the GIAHS designation discussion. Incidentally such an institutional review must not be limited to the national level functions, but must equally cover the decentralized arrangements a country proposes for engagement with individual GIAHS sites.

5.3 Achievements at local level, under Outcome 3

234. Outcome 3 was formulated as follows: Globally significant agro-biodiversity in pilot GIAHS is being managed and sustainably used by empowering local communities and harnessing evolving economic, social, and policy processes and by adaptation of appropriate new technologies that allow interaction between ecological and cultural processes (Local).

5.3.1 GIAHS and Adaptive Management

235. As a point of departure for evaluating outcome 3, the MTE decided to look into the (practical) application of Adaptive Management (AM) within the project. AM acts as the principal conceptual backbone for the GIAHS approach. This includes the overall GIAHS intervention strategy as applied by the project. Discussing and assessing the role of AM as a conservation and management methodology is relevant. 236. AM as a naturally inbuilt trait, acts as the “evolutionary and co-adaptive” driver for GIAHS management98. However AM is also described as a specific conservation concept99. This conceptual approach is directly associated with eco-system conservation and management. Within the project the imprint of both the original GIAHS and the application of the formalized

98 Reference is made to the original status of agro-ecosystems as they have long been present prior to the project. 99 http://www.conservationmeasures.org/wp-content/uploads/2010/04/CMP_Open_Standards_Version_2.0.pdf

Page 57: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

46

conservation management concept/ approach is expected. If recognizable100, the visibility and impact should be mostly seen at the local project implementation level.101 Applied to the conservation setting, AM follows individual process steps, as listed below: 1) conceptualizing, 2) action planning102, 3) implementation103, 4) analysis of results and using these for strategic adaptation104 and 5) capturing and sharing for learning purposes. This iterative management process ultimately must result in changes of local GIAHS. 237. Conceptually AM is remarkably close to “action research”. Action research also owes much of its strength to a continuous feed-back mechanism for empirical research, adoption of research outcomes and re-evaluation of the resulting response. AM’s open standard is geared towards increasing system sustainability, based on gradually evolving elements of the system as broader changing conditions demand. This evolutionary principle makes AM a relevant, efficient and effective mechanisms for a living system such as GIAHS. 238. An essential key feature for successful adoption of AM is the presence of sufficiently established capacity to repeatedly conduct (critical system) analysis. Furthermore flexibility to adopt and internalize analytical findings with the objective to increase system robustness is required. Therefore understanding if conditions exist among immediate project stakeholders to act responsively to new information and up to date knowledge management, is an important measure to predict formal AM’s success. That AM’s enabling environment would play a major role as a determinant for its success was to be foreseen from the outset. AM as the effective empirical tool it is, is known to generally thrive well in academic, international development and (even) private sector environments. Its results are more sketchy in the context of Government led / driven operations. 239. The above is confirmed by the evaluation of the current project. Variability in composition and role national stakeholders has had much influence on how well AM has practically functioned105. Not surprising such variations do rely a lot on choice of the national institutional counterpart and lead executing agencies, and how these agencies have been able to reserve operational space for academia, NGO’s and the private sector. When and where academic / research, local private sector representatives have been brought together and have worked in a complementing fashion to the function of Government services, this has had a positive effect on the application of AM process, leading to increased local implementation effectiveness. 240. Still overall the evaluation has established that much more effort will be required to create and maintain diverse stakeholder platforms for the benefit of sustainably managing individual GIAHS. Not least it is important to clarify role definitions and clarify the individual stakeholder stakes (incentives) under such public-civil-private sector collaboration. This discussion is clearly not specific to GIAHS as such forms of collaboration play a generally important role in the way societies function and improve the effectives of interaction among stakeholders. The upshot is

100 Self evident 101 As AM has its focus direct on the functioning, enrichment and continuity of the systems themselves, not

necessarily on associated national and global processes of institutional adaptation. 102 And monitoring 103 of planned actions and monitoring of this implementation 104 of the approach 105 See discussion under the chapter on project implementation and results.

Page 58: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

47

however that many limitations need to be removed before true application of AM can emerge and lead to a win-win situation for individual stakeholders106. 241. In those instances where the empirical internalization component provided by AM was lost the project has been rapidly affected by common standardized, regulation driven Government execution. Under such conditions local ownership over the GIAHS initiative was diminished and the public role was treated as any other one based on externally provided funding source. 242. Another important influencing factor regarding the benefits of AM stems from steps 2 and 3 of its open standard. This concerns development and implementation of actionable content. Especially in the earlier stages, but certainly continuing during the FSP, there has been a lack of clarity of the sort of project interventions which could be considered meaningful and legitimate for GIAHS support. Again scarcity of well attuned local project intervention content is more in evidence when project execution was mainly influenced by traditional top down governance. Often this has led to application of standard development approaches, which include non-sustainable use of input subsidies and project funded productive inputs. The evaluation found indications that those executing local project interventions have frequently seen the project presence in the light of solving GIAHS “problems”. The evaluation is somewhat wary of such assumptions as it has taken the project further away from its (originally) intended catalytic role. Instead project (or for that matter future programme involvement) should focus on AM based mechanisms to influence “the” GIAHS enabling environment. The latter, by contrast, is not relying on a troubleshooting concept. 243. The prevailing global project management structure, by adding and additional distant bureaucratic layer107 has, although not intended, acted as a disincentive rather than a catalyst for frequent and direct interaction with local stakeholders / counterparts. The low frequency of project “sparring” on concepts and issues between the global project and the local implementation tier is partly to blame. The already existing limiting impact on unfolding of an AM based approaches as a result from local governance conditions has not been effectively addressed or countered by the global project layer108. Local counterparts and stakeholders were conceptually and otherwise largely left to their own devices. 244. In principle, the global project layer should have provided a conceptually innovative and harmonizing influence109 on local implementation. However practically, added local level benefits resulting from global interaction and the promotion of AM as a tool have, in the majority of cases insufficiently materialized. 245. Apart from resource limitations preventing the project to effectively extend its global AM role, the evaluation also sees a more philosophical dimension to this issue. Main project proponents repeatedly expressed the idea that GIAHS AM capacity should foremost be found in the systems own innovative capacity, based on ingeniousness of local farmers themselves. Creating the opportunities for optimal information exchange and innovation is paramount in this. It is also this juncture where the presence of a global programme in relation to local initiatives is of

106 The limitations apply to the national and local levels within pilot countries, but equally to FAO’s capacity to deal

with adaptive management issues from the level of the project. 107 Government like. 108 The question needs to be posed whether this role can be at all be directly fulfilled by a global project or

programme, in principle it should be the empowerment of nation al and local GIAHS advocates that drives such change. However in the emerging stages of a programme a more pronounced role by the global project would have been welcomed. Provided a clear exist strategy had been defined from the beginning.

109 The typical incremental benefit expected from this layer.

Page 59: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

48

most use. The external AM role required from the national and global layers is to help strategically reduce conceptual isolation at the local level.

5.3.2 Achievements against Output 3.1

246. Output 3.1 was formulated as follows: Establishment of appropriate stakeholder set-ups at the site level that brings together customary, state and non-government institutions (including private sector actors) that will support local farmers to engage in collaborative management and promotion of GIAHS. 247. By and large, the project has succeeded in this output. This took place through the establishment of formal and informal working groups at regional and local level. Among participating countries, considerable differences exist in terms of activity level, functioning and effectiveness of groupings. In instances where the state (national) has taken a more active interest promoting the GIAHS concept this has had a positive influence. A more active organizational role in convening stakeholders benefited the formation of local level stakeholder set ups, and the manner in which these have been consulted. The importance of such a constructive and facilitating role in local organization is logical. This role indicates that GIAHS as a concept is valued at the higher institutional level, and thus considered as worth investing in. This must also be seen in function of local expectations for rising non material and material support, acting as an incentive to invest in local organization.

5.3.3 Achievements against Output 3.2

248. Output 3.2 was formulated as follows: Identification and monitoring of political and socio-economic processes that impact biodiversity and cultural values in GIAHS in order to enhance positive effects and empower local communities with knowledge and tools to minimise negative effects. 249. The output does encompass action which is conditional for understanding of how the GIAHS functions. This makes it stand out as being of great importance to sound action planning at the local level. Unless the identification refers exclusively to new and emerging processes, the question can be raised whether at least this part should not have been a concrete output of PDF-B? In the event neither systematic identification nor monitoring has been stressed in sufficient measure during the FSP. This is because although the principle of identifying opportunities and threats 110 is correct, the how to do aspect is largely left unanswered by the project. 250. Based on the assumption that global and national GIAHS support is justified as a need, it is most plausible that identification and monitoring inputs are supported from these levels as well. If this were not the case it would suggest that the GIAHS does not require the external perspective and can carry out its own SWOT exercise and act upon it. This idea is clearly refuted by the fact that local analytical capacity is generally limited, for those processes influencing the GIAHS from “outside”. One clear indication is a dearth of evidence and therefore assumed limited progress on historical and current economic research related to GIAHS. Stakeholder interaction, involving joint public, civil and private sector involvement clearly has not been able to stand up to analytical

110 Strength, Weaknesses, Opportunities and Threats (SWOT)

Page 60: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

49

challenge posed. This is not to say that the collective capacity for analysis of the GIAHS does not exist111 . 251. It is more the limited capacity of the project to align these relevant forces and help them steer towards meaningful analytical outputs, capable of translating findings into GIAHS adaptation strategy. The evaluation considers this as an underestimation and under-resourcing of this project aspect, finally prohibiting it to become a key strategic oversight. The relative shallow nature of advancement of this output is to be traced to the limited scope of research and state institutional involvement, and the relative scarce presence of the global project to steer this output in a more favourable direction. 252. Good understanding of the contributing factors to GIAHS’s health and future simply requires a broader and more holistic approach, and the mobilization of more diverse qualified brains of different walks of life than is currently the case. Specifically the global level has fallen short in providing guidance and direction to this output, as it has often been too much distracted by operational trouble-shooting and seeking of expansion of the number of GIAHS at the global level. This is not a contradiction with the earlier stated necessity for the global tier to refrain from local project implementation as it signifies a different role, which is not implementation but monitoring oriented. 253. To expect the concrete result of community empowerment springing from (research) identification of political and socio-economic processes and (functional) measures is also rather a simplification of real world processes. Practically opportunities for change are often complex or exceedingly limited by vested economic interest, and competing or conflicting agendas for resource use. 254. Moreover many processes affecting GIAHS have strong national, international and global dimensions. These are not simply to be countered by community empowerment but will require concerted higher levels interventions to turn the tide. It is as much the wider environment which may allow community empowerment, as what the project framework considers here to be straightforward local level output. 255. National employment, taxation and revenue sharing policies, all of which may have a fundamental impact on GIAHS viability, are only but a few examples, which are all beyond the control of local level research, investigations, identification and community empowerment.

5.3.4 Achievements against Output 3.3

256. Output 3.3 was formulated as follows: Screening, testing and deployment of environmentally friendly technologies and practices that improve the management and productive capacity of agro-ecosystems and their traditional crops, as well as new co-evolved races. 257. The implicit assumption contained in the formulation of this output is that the project shall contribute “new” technologies meaningful and fitting in the GIAHS agro-eco-systems. Screening will act as a “sieve” to select between the appropriate and less or not appropriate technologies. To begin with this requires technological knowledge and innovation from both

111 At regional national and global level, across different societal sectors.

Page 61: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

50

inside and outside the system. It must be noted, that a fair part of “new and promising” technologies is expected to originate from outside the system. 258. What has been observed frequently is an extension of project supported community based interventions, which were already previously promoted and carried out with assistance of national and local GIAHS champions112. 259. Again, if fresh additional technological inputs were not the goal of this output, it would make the project facilitating role more or less redundant. This as one should expect farmers to be capable to adopt innovations stemming from the system itself, if these are technically, financially and economically viable to them, but lacking in resources to identify and import technologies unknown to them from outside. 260. The project (management) has left it an open question by whom and how such GIAHS appropriate technologies will be identified. This is another instance where the incremental benefit of the project has been left unclear. 261. Where the project has moved ahead with the introduction of technological innovation it has at the same time become clear that even when technologically viable, this is not a given for the socio-economic aspects. On the contrary not infrequently past stagnation of technological innovation, can be attributed to the low level of rejuvenation of GIAHS custodians. 262. This makes not the technology itself but those that have to manage it, or rather the absence of such individuals to become the real issue. The proposition made by this output than becomes difficult to match or to support by conclusive field based evaluation observations in favor of the need for technology. 263. Practically the evaluation has not come across convincing evidence of actual systematic screening and testing of “new” technologies, and in the majority of positive cases, these appeared neither technically innovative nor economically viable. 264. In other locations the evaluation came across technological “innovations” introduced through import from external sources113, which had never been properly analyzed in terms of their practical and economic viability. Had this been done the project supported intervention would have been halted already or would not have been adopted. 265. The evaluation also came across project supported technological interventions, which were mostly fundamental for their project funded impact, rather than technologically game changing nature or specific GIAHS relationship114.

5.3.5 Achievements against Output 3.4

266. Output 3.4 was formulated as follows: Design and implementation of programmes for alternative and/or supplementary livelihoods to assist people meet the challenges of reduced opportunities for working directly on the land.

112 Often academics associated with systems research. 113 Collective compost manufacturing in Tunisia is an outstanding example. 114 Manufacturing of sheep cheese in Puno, Peru in association with a revolving fund mechanism and pasture

improvement (the latter being the only real link with GIAHS)

Page 62: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

51

267. There is no solid evidence of the project making significant progress towards achieving this output. It is clearly understood that more considerable progress would be depending much on local private sector initiatives, supportive government policies and more in general markets for products and services. 268. The sheer complexity required for the enabling coordination and supportive arrangements while creating sustainable livelihood alternatives, simply excludes “out of a box design” Elevating and rolling out such measures at the programme level forms an even taller order115. 269. The closest the project has been able to approach any output results is through the creation of product processing initiatives and the facilitation / participation of agro-tourism related discussions. With some flexibility of interpretation efforts designed for (pre) cooperative support and revolving funds may also counted as achievements under this output. 270. None of these project initiatives can however be seen as more than “scratching the surface” in a programmatic sense. For this to become a system wide relevant programmatic effort, a lot more would be required. 271. Design wise, as the project is based on AM, there is little to be learned from experience and iteration, therefore the design process and its documentation are exceedingly narrow in scope. “Meeting the challenges of reduced opportunities to work on the land “does suppose a system wide scope of project impact. To address this as evidence based measurable evaluation output makes it a rather far-fetched “output” assertion. 272. Moreover the evaluation does not consider this output to be relevant to the current project or a future programme, as it invites an open ended livelihood approach, of which the funding responsibility is left undefined, and which in the end may be difficult to link to GIAHS interest116 273. Any form of alternative employment strategies will require adequate national and local policy interventions, including the implementation of individual or community based investments and extension of rural finance. Evidently this cannot be reached project wise but depends on joint national programmatic efforts. Preferably in the interest of the importance of GIAHS as a national interest.

5.3.6 Achievements against Output 3.5

274. Output 3.5 was formulated as follows: Documentation and publishing of information about the case histories of establishment and management of GIAHS. 275. Design wise the evaluation sees this output as a more or less, out of place under this outcome. Documentation and publishing of information is from a management perspective better housed under outcome 4 117. From a programmatic perspective this would transform outcome 4

115 By a project. 116 There is a classical analogy with buffer zone concepts here. By dealing with livelihood systems in buffer-zones,

conservation objectives in core areas are supposed to be served. However buffer zones often face the problem of being ill-defined and acting as a magnet for those that seek benefit in an opportunistic manner.

117 “Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion and up-scaling of the GIAHS in other areas/countries and creation of the

Page 63: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

52

into a pivotal national think tank role118. In other words the fourth step out of the AM process cycle should be applied in this context. 119. Unfortunately, the project seems to have found it hard to systematically abide by its own AM principles in this instance. 276. Case studies relevant to the 6 direct GEF project related GIAHS pilots have been conducted and multiple leaflets and posters describing individual GIAHS, their characteristics and evolution have been produced. These case studies differ in their quality and depth and so do the leaflets and posters. Having said so, the evaluation , within the project itself, has not come across quality examples of accessible internationally published case histories120. This despite repeated requests to show such evidence. To qualify this further the evaluation would expect a clear scientifically supported narrative of how the GIAHS initiative has concretely affected individual sites.

5.3.7 Overall rating of results under Outcome 3

277. At best, it is difficult to attribute much direct impact on local level GIAHS change / co-adaptation / transformation as a result of national or international co-funding. This does not mean that the project has not seriously attempted to undertake local level interventions. On the contrary it did so and these efforts have been central to the project, as reflected by the level of budgetary allocation. The evaluation considers this emphasis in tension with the principles of incremental benefit to be applied for GEF funding. Among others, the project actually made active use of the GEF funding to finance local physical interventions. The evaluation takes the standpoint that this undercuts the relevance of many outputs and their activities under this outcome. 278. Another flaw of this outcome is that local project interventions are often a literal continuation/ extension of prior activities. These were already underway by GIAHS Collaborators and Champions either from academia or civil society. This is a missed opportunity to introduce new insights and enriching the GIAHS debate121, again putting in question the relevance of project actions at the local level. 279. Positive exceptions were however recorded. These are mainly to be found there where local administrations have collaborated well with academic facilitators, made good use of national support and have sought active engagement with other stakeholders, such as those from civil society and the private sector. 280. Here local administrations have been able to assert a more pivotal role in relation to local level activities. This was simply done by consciously lending policy and facilitating support, while taking heed of the prevailing stakeholder dynamics and sharing of responsibilities and roles. Effectiveness and efficiency of local project and associated national efforts has considerably increased as a result.

GIAHS network (Global, National, Local) (Total cost: USD 5 126 650; GEF: USD 1 172 742; Co-financing: USD 3 953 909)”

118 As is desired in an up-scaling scenario, otherwise the GIAHS effort will fail to address the scale and magnitude of its human population component.

119 Analyse , Use and Adapt. 120 Despite the involvement of both national Chinese and Peruvian based national consultants the evaluation scope has

not allowed it to comprehensively access and assess the full body of scientifically published Chinese documentation., or for that matter all that was scientifically published in Spanish. Therefore these publications are left outside consideration.

121 At all levels local, national, global.

Page 64: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

53

281. Some of the concrete activities under the outputs of this outcome, specifically those related to improving GIAHS product and marketing exposure have also been more appreciated. They are seen as a value and project added by respondents at the local level122 a reflection on the effectiveness of this intervention that should be taken into consideration in a future global programme. 282. No substantial evidence can be found to demonstrate a tangible relationship between project pilot initiatives and the (desired) policy and legal framework outcomes. Therefore, altogether the sustainability enhancing effect of project pilot interventions must at best be considered questionable. Scale and scope of pilot project funding resources have played an overall constraining role, especially where funds supported interventions of limited and isolated nature, generally too small and spatially fragmented to act as system wide drivers of change. Overall, such interventions did little to reach a tipping point leading to effective and sustainable GIAHS changes. 283. National public and private efforts, financially and through employment and local economic integration, are called for. Chiloe is a good case, with substantial potential, this as long as the private sector is willing to invest into long term local capacity and employment generation, which currently is (not yet) a priority for the organized tourism sector there. 284. Mainly as a result of the design flaw of using a global project outfit with little decentralized capacity to implement directly at the local level, the GEF rating for this outcome is moderately satisfactory. There is an immediate association between the rating and achievement of the local component and that of the two preceding ones. With a more catalytic rather than direct implementation focused orientation the rating would have been more positive. At the level of individual countries ratings would vary from Satisfactory to Unsatisfactory so the rating as presented here can also be interpreted as an average of the cumulative local implementation results.

5.3.8 The way forward

285. Traditional problem solving type development interventions must not get the attention provided to them in the past. This type of interventions requires a complex micro-project delivery structure under a global project / programme umbrella, which is inherently inefficient. Rather than supplying plastic tunnels and trickle irrigation units and engage in the construction of public meeting halls on private land. The evaluation asserts that energy needs to be put into local level lobbying for fundamental changes in the relationship between GIAHS custodians, the private and public sector. The here mentioned example refers to the previously mentioned strategy to engage local people in the Chiloe tourism sector. 286. Strategically this is a sound approach, because the reliance of tourism is in large part on maintenance of man- made agricultural environment of which the local custodians are however out-migrating in increasing numbers to the main land in search for employment and more rewarding income opportunity. Keeping families at local level by providing local employment of an equally rewarding nature also ensures that there remain a sufficient number of people to engage in land management which is instrumental to attracting tourism on a continuing basis.

122 This based on recorded interviews with local stakeholders in all visited local sites.

Page 65: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

54

287. The project must strictly prioritize its outputs for the remainder of the project period. A modified Output 3.3, meaning to include full GIAHS economic valuation, must be urgently addressed during the remaining project period. Prioritizing will not be too difficult; despite low budget utilization for the local component. Still available funding is insufficient to match the scope of currently planned local outputs and activities both in time and space123. 288. A refocused outcome 3 must aim to bring a maximum amount of local implementation experience and lesson learned into the global discussion, and will have to closely associate with outcome 4. The evaluation recommends additional and independently commissioned case studies. These must concentrate on listing the financial and economic fundamentals underpinning local GIAHS continuity and sustainability. Case studies based on clear universal ToRs, must ensure global uniformity of approach and steering towards identification of appropriate local mechanisms and actionable content. 289. Among others, economic profiling of individual GIAHS must become the center-piece of the intellectual discourse regarding conservation and adaptive management124. It is recommended that case study findings are systematically categorized and launched in a global online forum discussion. The global discussion can be coordinated by the project, in close collaboration with those others FAO that are working towards a global programme structure for GIAHS. Facilitation and collection of local/ national contributions to this discussion is recommended to be carried out as a joint delegated task to national Counterparts supported / assisted by FAO Representations. 290. The forum discussion must answer a few basic questions among others. One of these is to try and reach consensus about the programmatic role in relation to local interventions. Are physical project / programme pilot interventions justifiable and logical in view of sustainable management and conservation of GIAHS? Or should the global programme centre on governance and regulatory issues, while leaving implementation at local level in the hands of others? 291. In light of up-scaling of previous project experience, what are fundable and non-fundable local level activities? Requirements to be met are sustainability125 and being of a catalytic nature. Based on the outcome of this discussion it is recommended that a future FAO global programme, again in consensus with participating member states, establishes clear criteria for allowable and non-allowable local interventions and funding mechanisms126 A first step in this process could be using the forum discussion to draw up a set of globally legitimate GIAHS interventions and pass this list for endorsement by the global project steering committee127. 292. It must be the role of a future programme and secretariat to inform / advice national and local authorities on how to become eligible for GIAHS designation and formal accession. What will be an important aspect is advice on how local GIAHS can accede to and benefit from existing international legal instruments as a result of the GIAHS status and the additional exposure this generates for the local system.

123 Volume. 124 Case studies perused by the evaluation have tended to focus more on the historical and cultural aspects and have

been less able to come up with feasible and actionable content as a result. 125 Within a programme concept, meaning non-dependency creating, provided through national regional and local

means and without the use of structural subsidy streams. 126 Where possible avoiding the undue use of open ended subsidies 127 Not only is this the mandate of the current project steering committee, this must also ensure coherence between the

project phase and the steering mechanism for an eventual FAO GIAHS programme.

Page 66: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

55

5.4 Achievements under Outcome 4

293. Outcome 4 was formulated as follows: Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion and up-scaling of the GIAHS in other areas/countries and creation of the GIAHS network (Global, National, Local). Total cost: USD 4,705,936; GEF: USD 1,238,593; Co-financing: GIAHS and Adaptive Management.

5.4.1 Achievements against Output 4.1

294. Output 4.1 was formulated as follows: Implementation of the project’s M&E plan at global and pilot-country levels and adapting project implementation according to the outcomes. 295. The project has created a basic tracking mechanism whereby it follows up on the progress of global and national work plan interventions and reports against these. The evaluation has already argued that what has been currently put into place is not a full-fledged M&E system. This as it is too much project reporting directed and not based solid objectively verifiable indicators and weak means of verification. 296. The reason why the GEF (FSP) project document refers to the system as a plan is moreover unclear. What it does suggest however is an evaluation approach which is focused more on M&E planning and design, than the actual use of an operational M&E system. Clearly a future programmatic M&E system must be based on a robust but flexible128 data base structure, containing detailed system information, local and national tiers of information129 297. What is crucially lacking in the current project M&E system is qualitative evaluative capacity. One of the problems observed by the evaluation is that the project in its mandatory progress reporting to GEF, has a tendency of embellishing results rather than giving forewarning of conceptual problems as would be expected from a more profound evaluative approach Of course the project has reflected delays in implementation progress more or less according to the actual situation, as such delays had to be recognized.

5.4.2 Achievements against Output 4.2

298. Output 4.2 was formulated as follows: Preparation of a global publication on lessons learned and best practices. 299. The MTE has not come across a specific and conclusive global publication outlining GIAHS lessons learned and best practices from pilot countries / sites. In certain countries such as China there exists an ample amount of GIAHS research documentation, which could be used as a (national) basis for such a publication. This is less so the case for other pilots. The visits of the evaluation to Peru and Chile coincided with the mission of an environmental journalist hired by the project secretariat. This journalist had been commissioned with the preparation of a picture based publication for popular use among the global public.

128 Modular 129 And evidently part of a further upgraded GIAHS website.

Page 67: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

56

5.4.3 Achievements against Output 4.3

300. Output 4.3 was formulated as follows: Preparation of scientific reports and publications arising from project investigations and implementation. 301. The Evaluation has come across a series of scientific reports130, which were either drafted by FAO consultants or independently internationally or nationally produced. The scope of the evaluation as well as the occasional language barriers has prevented a systematic and comprehensive qualitative analysis of all produced publications. It rests to say that what has been perused during the evaluation is uneven in scientific scope, reporting style and usefulness in terms of translation into practical use at local level and under eventual programme implementation. Often reports are rather focussing on (historical) system description than on a thorough analysis of the full complex of system dynamics131. Clearly the social, institutional and public administration angles of research seem to be under-emphasised in scientific research.

5.4.4 Achievements against Output 4.4

302. Output 4.3 was formulated as follows: Creation and maintenance of a web-based information management system that will include a database on existing and potential GIAHS, and will also be designed to serve as an electronic forum for sharing information and experiences across the various pilots. Pilot system communities and pilot countries will provide information through their own web-sites and publications. 303. The project has effectively contributed to the establishment of a web site on GIAHS. This web site is well maintained and contains most of the up to date database information on GIAHS and other materials produced in the course of the PDF-A, PDF-B and FSP. 304. The website could be effectively turned in the active online forum that the output intended, but which it did not reach under the project. This will however require more specific directed format and active web-hosting of the forum function by FAO. There seem to be good opportunities for this in relation to the real needs associated with FAO programme establishment 305. Although this website is accessible to all those with and internet connection and national GIAHS websites have also sprung up132 (China) it is not clear if pilot system communities / individuals have had direct access to this web based medium. The tendency of the evaluation is to assess that such an access has been limited.

5.4.5 Overall rating of results under Outcome 4

306. For more effective (global) impact from lessons learned under the current GIAHS pilots and future programme engagement, it is important to establish a more dedicated and strategic M&E, Global research and communication approach.

130 Annex 12 As provided to the evaluation by the project secretariat. 131 GIAHS 132 China is a good example.

Page 68: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

57

307. Under a future global programme, commissioning of targeted (scientific) case studies as listed under output 3.4 can be a contributing service carried out by a global programme structure and make up part of GIAHS global / national think tank efforts / investments. 308. The evaluation sees sufficient arguments and ground to declare GIAHS, as a result of the GEF project investments as a tested functional concept. This is conditional to continuing further increase of systematic and efficient critical analysis of accrued GIAHS experiences and lessons from the past 10 years. 309. Originally the designers of the GEF project may have not been intended this as an effective output. However critical analysis and providing a global experience based sound board to national and local GIAHS initiatives will be a future core programme function. 310. The GEF rating for this component can be judged Satisfactory, contrarily to the expectation according to which the project might have performed exactly according to the LF. The motivation behind this rating lies in the fact that with relative little means, and mainly through perseverance and creative networking, there has been a substantial spread of the GIAHS concept in recent years. Today this provides a greatly enhanced understanding by member states of the opportunity offered by the concept. Concrete evidence lies in the broadening of the extended project baseline.

5.4.6 The way forward

311. The project internal lessons learned exercise must concretely address its institutional experiences and place these on the agenda of the global steering committee133. 312. Analysis of the previous GEF project design and outcomes of the current evaluation should preferably play a central role and starting point for this discourse on: a) GIAHS programme formalization, b) GIAHS designation procedures, with generation of specific GIAHS benefits as a result. 313. In any future programme structure it is recommended to foresee an independent scientific monitoring and evaluation function. This function must be handled on a regular recurrent basis for the overall global programme facility. It also should, in a less frequent manner; jointly with national GIAHS representatives134 (NIAHS) provide international M&E capacity for designated GIAHS sites. Although the concept of NIAHS emerged in practice in China and makes up an important intermediary level for the global GIAHS governance structure, no explicit reference is found to it in the original project design. 314. Rather than focusing on publications of a scientific or more popular orientation, it is recommended that the project concentrates on further constructing its capacity for systematic analytical process. This process must systematically solicit the views of national stakeholders, actual and prospective project counterparts as well as FAO inter-departmental staff. This must be done to shape, maintain and guard the scope, role and function of a permanent GIAHS programme and its secretariat.

133 Currently that of the project and carrying this over to the programme steering committee and task-force structures. 134 Preferably NIAHS committees.

Page 69: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

58

315. It is recommended that it is the role of a future secretariat to highlight the importance of further scientific research into designated GIAHS systems. In order to effectively do so it may set research guidelines, support international university twinning arrangements, and seek opportunity to create national and local revenue from the facilitation of research projects. 316. The evaluation recommends the continuation, expansion and further operational activation of the GIAHS web site. This must be turned into an effective and well promoted national / international communication spill on GIAHS issues. The site must also house a fully accessible international database on designated and prospective GIAHS sites. 317. In the process of information upgrading it is also important to provide considerable attention to; a) further criteria detailing and subsequent negotiation process for formal NIAHS - GIAHS designation, b) elaboration of national programme content, and c) identification of input resources as well as135 d) detailing of advisory economic and technical aspects of GIAHS global. 318. Although it will be FAO that continues to host the web-site care must be taken that the future programme site is not primarily dominated by global level products, e.g. accounts of WS and international site events. 319. Further, it may be the evaluators own personal bias, but other items, suggesting project / programme promotion rather than GIAHS concept and operational promotion, must be limited e.g. : GIAHS recipes, GIAHS tourism labelling136, GIAHS project certificates without an accepted global status, GIAHS merchandise etc. 320. Crucial is that the GIAHS web-site develops itself in: a) an effective tool for global communication over GIAHS system functioning137, b) an information source on concrete options of GIAHS benefit generation, c) a database repository, d) a channel for international engagement and networking, f) service directory, and g) quality control facility. 321. The current GIAHS website is hosted under the www.giahs.org domain name and can also be found under the www.fao.org main domain. To increase further exposure of the concept, further active linkage and promotion of the web-locations through other relevant global convention and treaty initiatives could be considered useful. 322. In parallel the future programme must of course work on a more robust advisory and support role of how GIAHS that will allow present and future GIAHS to effectively relate to these international instruments. Linkage in cyberspace is only a tool but more actual functional content is also required.

5.5 Gender equality mainstreaming

323. The project does not contain any specific outcomes and outputs related to Gender and Social Inclusion. Varying cultural environments in which GIAHS are located as well as country specific policy and legislation related to Gender and Social Inclusion all play a significant role in determining the specific local status regarding these matters.

135 No longer to be channelled through a global programme but through national efforts. 136 Rather than calling it agro-eco-tourism, which is an already widely recognized name. 137 Local, national, global

Page 70: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

59

324. Historical roots and the traditional nature of GIAHS do often include deeply embedded Gender and Social specific role divisions within custodian communities. Exertion of external pressures such as globalization and specifically the associated market forces have frequently resulted in a change of traditional roles. Where systems have moved away from their subsistence character towards increased market integration this has often led to a proportional loss of female roles and influence. The aforementioned is not specifically documented or analysed for project pilots. However, such trends must be assumed to hold true on the basis of general evidence where traditional systems are subjected to increased commercial integration and external technology drivers. In such instances it is mostly men whom are better placed to interact with the outside world, which provides them an advantage in information capture and decision making based thereon. 325. The overriding project focus on the local level would suggest that Gender and Social Inclusion issues might have played a more prominent and visible role, at least at this level. By extension it might have been anticipated to also make overt mention of Gender and Social matters at the national and global project levels. That this is not the case can be attributed to an already overloaded project agenda in face of limited practical execution means. For the project to have a discernible national impact on Gender and Social Inclusion policies is as far-fetched as it having any impact on policy and legal matters in any other field of its concern (e.g. land tenure, economic investment, tourism, commerce). Realistically this would have rendered Gender and Social Inclusion issues to a largely paper subject, which might have featured more prominently in the international discourse at the global project tier. 326. Regardless of the fact that Gender and Social Inclusion is by and large omitted from the project the MTE assumes that the GIAHS can have a potential positive role on Gender and Social Relationships at the local level, based on a comparative analysis with the immediate initiatives existing to conservation and adaptive management of GIAHS. The increased rural – urban migration which would come with the gradual disappearance of GIAHS would at least initially lead to an almost certain and more disruptive change in the existing Gender and Social fabric of communities. 327. Although not specific to GIAHS the small scale rural finance initiatives employed in some project pilots in potential are advantageous to local women. The insignificant scale of such initiatives employed under the project has however not led to real and measurable impacts in relation to Gender roles. 328. In terms of project related employment and association with Government Agencies the MTE has noted that in several countries female staff has played significant roles at the national and local level. This applies both to official roles within national counterpart agencies as well as among associated staff within FAO representations, research institutions and among recruited project consultant staff. Numerical representation of female staff associated to the project is however by no means an indicator for more or less Gender balanced roles at any of the three project implementation tiers. A typical illustration of this is the fact that the principal technical officer for the project, responsible for day to day management is female, but because of her professional scale within FAO was never entrusted with more elevated policy or budgetary matters. 329. From a forward looking perspective the MTE stresses the importance of a more explicit focus of a future programme on monitoring Gender and Social Inclusion Impacts. Instead of aiming for separate programme criteria for Gender and Social Inclusion it is however proposed

Page 71: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

60

that the programme analyses, adopts and follows sovereign policies, legislation and regulations of countries associated with GIAHS. The opinion may be held by some that this proposed approach is too modest. However the MTE advocates maximized impact within the margins of the realistic rather than ambitious programme statements on Gender and Social inclusion, which cannot be implemented as a consequence of absence of an embedded policy and legal environment.

5.6 Capacity Development

330. Capacity Building is an implicit element of the overall GIAHS initiative, its strategy and implementation approach. Therefore capacity building in the broadest sense of its meaning is also part of the project. Most clearly this translates in project outcome 4 dealing with lessons learned and feedback of information into global, national and local implementation structures. All required and envisioned by the initiative to enhance GIAHS enabling conditions for optimal system survival and co-evolution one or another way depends on availability of appropriate skills sets. The initiative consists of complementing hierarchical tiers at global, national and local level. At all these levels capacity limitations and corresponding capacity development needs play a role of significance. This is of course aside from the political and economic factors determining the actual GIAHS status. 331. Although the project is clear about the mutual dependency of these different tiers this is not translated in the form of a comprehensive project wide capacity building plan, clearly outlining the typical capacity building requirements at each tier and their interrelationship. The project has made a point of organizing its capacity building horizontal and country specific basis. In principle this is not a bad choice as it allows for and adequate measure of national and local specific capacity building needs assessments. 332. The limitation is however that the project has not in parallel developed a cross-cutting and vertically oriented capacity development vision. Such a vision is required to bring a measure of global harmony in programme capacity building mechanisms, choice of tools, proportional resource allocation and capacity building impact monitoring. It would however lead to far to say that the project has followed a complete ad hoc approach when it comes to capacity building. 333. It is clear though that the absence of a strong unifying vision of specific skill requirements at different project tiers has led to a cover all approach of capacity development. This approach has further been defined by the use of a narrow set of capacity building instruments. Too often capacity building has been equated with training; exchange visits, on organization of general and topic specific workshops, as well as project provided technical assistance. Moreover the majority of energy in capacity building has been directed to the local levels. There is meanwhile lack of evidence that the project has interpreted capacity building in a more holistic form. This would require to consider capacity building in the context of institutional development, and have to include the role of individual capacity building beneficiaries and their prospects to influence (Government) systems, through career development. Clearly this is also a much more long term capacity building vision difficult to accommodate in a project. 334. Certainly the MTE wants to avoid an impression as if it holds the opinion that the project could have done a substantially superior practical capacity building job. This is out of the question with the resources, practical means and management structure available to it. Such capacity building limitations are also in line with an identified shortfall in political and institutional reach preventing the project from considerably and systematically advancing on multiple cross-sectorial policies, legal and regulatory aims.

Page 72: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

61

335. As programmatic outreach to actively engage in capacity building will most likely always be in short supply the MTE advocates a far more strategic approach towards capacity building, essentially focussing on setting of basic institutional and technical parameters for national and local institutional GIAHS support. This would leave from the premise of defining institutional and organizational profiles with matching skill requirements. These skill sets could be rather variable. They could vary from the creation of an improved interface between research and policy based capacities. More specifically it could provide an outline for the development of national GIAHS assessment skills, including sound steps for system economic valuation. The frequently mentioned issues of certification and labelling, GIAHS specific or not, should be in the forefront of a global GIAHS capacity development strategy. At the local implementation level (community) the programme may endeavour to translate the specific role of adaptive management. This should be done in local system relevant concepts and terminology and be based on practical examples. Inter alia these would include the preparation and dissemination of well analysed GIAHS case studies. 336. Of course the project in its current form has a similar aim. In practice the MTE has found however that actual relevant experience exchange between different GIAHS (national and international) has to be sought far and in between. Not least this is because case studies have taken on a GIAHS promotional rather than capacity building oriented function and approach. This is an obvious risk when mixing the objective of case studies with that of the presentation of success stories. 337. In conclusion by concentrating on setting of capacity development parameters and developing key building blocks, a future global GIAHS programme operates in the limits of what is feasible. It thereby actually provides a value added to the other programme tiers. The task of capacity building delivery should be entrusted to available national public, private and civil society supported institutions / initiatives and may further expand on these. In other words they should not form an integral global programme role. 338. A clear programme role however remains analysis of national provided arguments and evidence for GIAHS selection, accession and support delivery. This entails in practice a recurrent matter based on scheduled and joint national / global monitoring of set individual GIAHS goals. Practically a global programme unit is extremely unlikely at any time to be in position of individually monitoring all individual capacity building efforts at national and local level. The global programme is thus much better off monitoring the proxy indicators for national and local GIAHS advancement.

5.7 Sustainability

339. Considering the very nature of the GIAHS Initiative, sustainability plays a most central role in the GIAHS conceptual framework, at different levels. 340. At the local level, this framework, promoting sustainable agricultural production principles, leaves little room to dispute the relevance of the GIAHS Initiative. The project message of advocacy to preserve and adaptively manage GIAH Systems is clear and convincing. Several other sections in this MTE report are equally supportive of the relevance of the GIAHS initiative based on its sustainability enhancing potential. This encompasses (bio)-diversity, cultural diversity, and (socio)- economic aspects. The latter socio-economic importance depends obviously on the specific (local) conditions offering potential expansion room of specific GIAHS traits. If

Page 73: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

62

physical sustainability characteristics for individual GIAHS not always remain evident today they should have at least been demonstrably present in the recent past. 341. GIAH Systems are understood to be systems which embody co-evolution, adaptation, self-organization technical balance and resilience. So eligible systems must be in a state of, or close to equilibrium maintained by a balanced and sustainable flow of inputs and outputs . 342. Effective and efficient allocation of political and institutional attention, as well as regulatory, transportation, advertisement and cash and resources, to GIAHS are directly supportive to sustainability of the global GIAHS Initiative. For individual GIAH Systems, efficient capturing of available opportunities (originating internally and externally to the system) determine the level of positive impact on system improvement (i.e. synergy, integration and health) making the coherence and sound timing of external GIAHS support interventions very influential. 343. At the global and national levels, the key issue is the sustainability of the GIAHS Initiative itself, particularly the institutional and financial sustainability required to provide continued advocacy and support to individual GIAH Systems. Project steps in support of this aspect rely on the project’s effective dealing with policy, legal and institutional aspects, as well as lessons learned and information management. Among others this entails dealing with the paradigms and sustainability questions surrounding mainstream agriculture, and advancing the alternatives from within GIAHS. This requires accurate capturing of system information and analysis, resulting in knowledge that can be applied to similar systems and the wider agricultural domain. 344. Evidently none of this is a short-term undertaking. Global information management, advocacy and concrete steps for global bio-diversity conservation in GIAHS sites require presence of a robust programmatic backbone structure. Financial sustainability is a functional requirement in support of the institutional aspect. 345. Sustainability also stands central to services which GIAHS can collectively and individually contribute to the world. To begin this applies to ensuring sustained availability of knowledge and goods contained in GIAHS . That this is not a purpose by itself is evident. Services and goods must be measured for their usefulness to the extent they are mainstreamed and have a continuing impact on wider agricultural production systems. 346. Thus GIAHS sustainability appears to be the single most important defining factor at either individual or support system level. Since sustainability is so central to all conceptual, managerial and operational project aspects, the project’s achievements are best rated on the basis of what they contributed in the form of sustainability enhancing measures / interventions;

a. Enhanced physical, managerial of cultural sustainability of individual GIAH Systems covered directly by the project generally cannot be attributed to specific project supported interventions only. Simply project interventions are of a too limited coverage and too scattered within the individual systems to be able to assert a claim of documented, measurable and direct impact. From the sustainability point of view, it can also be noted that impact at the local GIAHS level is constrained by the lack of a coherent intervention strategy and technical choices springing from system analysis.

b. At national level, in none of the countries included in the MTE it can be claimed that the project to date has comprehensively reached its aims of policy, strategic and institutional anchoring. Much additional efforts through and outside the project are required to enhance national ownership and continuity (sustainability) of GIAHS concept at national level. Currently national sustainability achievements are rather

Page 74: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

63

uneven in character. There are promising rates of institutional success in China and Chili, with somewhat lesser rates in Peru and relatively low rates in case of the Philippines, Tunisia and Algeria

c. From the perspective of project gains in global GIAHS recognition there has been considerably more success. This is mainly thanks to the efforts of information management and promotion surrounding the Initiative. Indirectly the resulting international exposure of the GIAHS concept has also positively contributed to national conceptual acceptance and recognition. These form an important foundation on which further technical and institutional interventions, can be sustainably grafted.

5.8 Impact

347. When delivering on its intended outcome to provide a measure of permanency for the GIAHS initiative138, the project shall contribute to achievement of FAO’s Strategic Objective (SO) F. (“Sustainable management of land, water and genetic resources and improved responses to

global environmental challenges affecting food and agriculture”). Evidence for this impact is meant to materialize by elevating GIAHS to the status of programme as part of the Organizations regular programme of work. In particular the project impact will be noted under the related Organizational Results F1, F3, F4, F5 and F6. 348. The project has also had an impact on achievement of FAO’s Functional Objective (FO) X (“Effective collaboration with members states and stakeholders”). As demonstrated by ample discussion on the subject of project interaction with national counterparts and other national and local stakeholders, the term effective collaboration requires some further qualification. Clearly the measure of achievement of project outcomes is at least in part dictated by the projects contribution and impact on FO X. It must be noted that apart from outcome 4, none of the other three outcomes has truly materialized at the time of the MTE. There exists documented evidence of growing interest and desire to for GIAHS programmatic permanency139. 349. However national government proponents of GIAHS (representing the category member states as well as being institutional stakeholders) must as yet effectively witness the creation of a FAO GIAHS programme. The policy environment at national level has meanwhile not be irreversibly affected by project outcome 2. There is evidence from the preceding analysis presented by this MTE, that the project design has insufficiently taken into account the practical leverage a project may have on sovereign policy formulation and legislative process. 350. Thus it can be argued that the effectiveness of collaboration on this outcome has from the start been tainted by a lack of realism. This has negatively influenced the effectiveness of the collaboration. Finally Outcome 3 dealing with local interventions has resulted in certain tangible direct (project executed) and indirect (national executed project spin off impact140) impact in pilot locations. Such collaboration may be considered effective precisely and just because it has translated in tangible “on the ground” local results. For good measure the MTE wants to note however that effectives is more than a quantitative measure of success. When considering the qualitative side it is rather the sustainability and replicability of outcome 3 results that count. Here effectiveness of collaboration scores considerably lower.

138 Outcome 1 139 COAG and FAO Council 140 E.g. the selection and funding of additional sites in China, and selection of, and implementation in additional new

pilot GIAHS by countries not originally included in the original project list.

Page 75: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

64

351. Arguably the outcome 3 as a key aspect of the project has received a disproportional percentage of funding to result in remotely managed local project assistance. The technical quality, institutional and managerial continuity of local project interventions rather more creates than answers questions about local GIAHS support criteria, preferred implementation mechanisms and the use of external funding141. Project limitations in delivering on the need of a clear and conclusive implementation framework for local level interventions, effectively dampens the impact of member state and stakeholder collaboration with FAO in this respect. 352. With respect to the project’s impact on FAO Core Functions (CF) the MTE observations are:

Relevant Core Functions Project’s contribution

Assembly and provision

of information,

knowledge and statistics

The project has made inroads in the assembly and provision of information regarding existing pilot GIAHS. The structure in which such information has to be anchored and made available for systematic use and replication is, as yet, in-complete and subject to further refinement.

Development of

international

instruments and norms

Further FAO corporate action is required to approach, what is agreed to what is the feasible alternative for creation of a GIAHS international instrument; The negotiation of a neutral programme forum to govern the GIAHS initiative beyond the tenure of the current project.

Policy and strategy

options and advice

The project has been well intended about the delivery of GIAHS strategy options and advice to participating member states. To attain increased effectiveness in this respect the full weight of the Organization’s policy and normative capacity must however be brought to bear. The current project basis must be considered too limited in scope and capacity to match required inputs for this CF.

Technical support to

promote technology

transfer and build

capacity

The preceding capacity building section covers the project contribution to this aspect of FAO CF. In resume the project has mainly provided local level technical support, which in turn was based on local needs identification. By definition local needs identification does not cover technical support and capacity building falling outside the local remit. Technology identification, applicability screening and technology implementation transfer through technical support qualitatively and from a sustainability point of view provides for a mixed bag. Use of national and local recruited TA has led to variable results depending on planning realism of interventions and allocated resources. In instances there has been a continuation of pre-project interventions using the same involved national consultants. In that sense FAO as an institutional repository of technology has played a less significant role, while the project has continued and, at times, positively impacted on streamlining technology identification.

Advocacy and

communication

The project has performed above average when considering CF of International Advocacy and Communication of the GIAHS as an holistic entry-point for family agriculture, enhancement of food security, climate change mitigation and adaptation. This has resulted in broadened international interest among member states, research institutions and other stakeholders. Practically the result from advocacy and communication can measured by a growing number of national accessions to the GIAHS network.

141 Including use of existing and specifically dedicated GIAHS (national) subsidies

Page 76: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

65

Inter-disciplinarity and

innovation

GIAHS itself is and inter-disciplinary concept and as such leads to a similar approach for the project. This adds a considerable conceptual strength of GIAHS and the project. The project has however suffered from conceptual simplification when it comes to the appreciation of how inter-disciplinarity relates to institutions and governance. This has diminished impact which might otherwise have been expected from the inter-disciplinary and holistic approach the project stands for.

GIAHS are traditional systems with a strong inbuilt objective of risk management142. Suggested dominance of conservatism and lack of innovation are often stated short-comings when outsiders witness such systems. The reality of adaptive management (AM) does however learn that innovation is not foreign to GIAHS. GIAHS are the result of a combination of natural bio-diversity richness, cumulated results of centuries of human selection and innovation143. It is also true that GIAHS are mostly threatened by stagnation of innovative process. Mostly this is due to the marginalization144 and sometimes technological regression of the GIAHS custodians. The core function of the GIAHS initiative is to catalyze fresh innovative development to guarantee continued evolution, existence and benefits from individual GIAHS. Available project means are by any account to limited in scope and time to result in measurable innovation. The main innovation therefore lies in the presentation of GIAHS as an overarching umbrella concept, providing focus and direction to future innovations. The project has been reasonably successful in bring across this idea.

Partnerships and

alliances

The project has been successful in creating alliances with globally significant institutions and organizations dealing with bio-diversity, agricultural development and production, and cultural heritage. These alliances have been key in the gradual global expansion of the GIAHS concept among member states. Partnership building contribution by the project has progressed to a lesser extent. The reason for this is that partnership structures are supposed to carry a more goal oriented and operational character. By definition it is difficult for a project to engage in such partnerships as it requires broader organizational buy-in. The potential emergence of a GIAHS programme is required to lift the initiative to the level where true partnership development and engagement can come to fruition.

142 Spread of risk by relying on a broad range of agro-biodiversity, tested and proven traditional management

systems, often with more than just a technical, but also a cultural significance. 143 Technical and Cultural 144 Cultural, political, economic.

Page 77: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

66

6 Global Project Performance Rating

Monitoring and Evaluation Rating Comments

Overall quality of M&E MU M&E functions were not developed as independent and dedicated project function. The MTE was launched late in the project with the consequence that insufficient time is available to address M&E issues in the project life time.

M&E design at project start

up

MU The M&E design relied on a flawed and overambitious project LF. In an attempt to introduce results based monitoring physical and geographical indicators were introduce for which neither clear baselines could be established and for which the project lacked strategy (planning) and resources for adequate follow-up

M&E Plan Implementation MS M&E reporting against the Logical framework took place on a regular and timely basis. Attempts were made by the FAO GEF unit (TCID) to enhance the quality of indicators and means of verification. However much of the M&E reporting narrative does not directly address the information asked by the LF. There also is a tendency of avoiding the obvious LF flaws against which monitoring and evaluation must take place. Frequently the reporting shows a tendency of embellishing achievements of the GIAHS initiative which are not pertinent or relevant to the project itself.

IA & EA Execution

Overall Quality of Project

Implementation/Execution

MS Despite flaws in project design and the subsequent applied implementation approach and project management, the project has contributed significantly to raising the profile and international buy in among FAO member states for the GIAHS initiative. The evaluation considers this an important priority in line with the project’s global outcome 1. Project Implementation has also resulted in experience serving as useful lessons learned for continuation of the initiative. That National and Local project outcomes according to the LF have in most instances not materialized is of course a negative. However the evaluation considers this mainly the result of poor design and centralized global project management. Local on the ground project implementation (outcome 3) has served the purpose of feeding information and knowledge generation under outcome 4 and may continue to do so.

Implementing Agency

Execution

N/A

Executing Agency Execution MS As stated unrealistic project design as weel as weak assumptions regarding the interrelationship between project outcomes have created significant hurdles once execution of the project started. The executing agency, the former implementing agency (UNDP) as well as GEFSEC share the responsibility for the design as it was adopted. The executing agency carries the sole responsibility for failing to critically analyse the design problems during execution and not undertaking timely action to address identified issues. The failure to do so, goes back to a highly centralized global project management, insufficient empowerment of day to day ptoject staff to address project strategic matters, and lack of significant corporate buy-in and platform building for the initiative within FAO.

Outcomes Overall Quality of Project

Outcomes

MS As part of the general project design, individual project outcomes at global , national and local level all have a legitimate right of existence. However the manner in which these outcomes have detailed, prioritized and internally linked leaves much to desire. Aside from the overambitious and unrealistic targets, the most serious short coming is that the project has tried to leverage global and

Page 78: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

67

national results through disproportionate focus on local level implementation. The linear logic that fragmented local demonstrations will help trigger national policy changes and building global mechanisms does not hold however. The project in most pilot sites lacks the resources (project and co-funding) to create sufficiently significant local impact. Pre-occupation with delivery of local impact, combined with insufficient dedicated HR dealing with global and national aspects led to unbalanced results among project outcomes. On the ground project presence was used to raise the global profile of GIAHS through outcome 4. In most instances this happened without leveraging national policy development and no discernible impact on the creation of a global mechanism can be recorded. There are however positive exceptions for the national project outcome. China is the most positive illustration of the creation of national buy in. Among others this has resulted in recognition a National pre-cursors for GIAHS (NIAHS) as well as higher than originally committed national co-funding. In Chile and Peru there are also positive national developments worth mentioning. Chile is essentially adopting the GIAHS concept and will be able to mobilize national support resources under coordination of the MoA. Private sector initiatives have potential there, but need to further emerge, while in case of Peru private sector involvement with GIAHS is steadily growing, which is however not a project achievement but rather a parallel development driven by increasing popularity of Andean Gastronomy.

Relevance S The overall relevance of the GIAHS concept as contained in the project is largely undisputed and therefore satisfactory. Co-evolution of bio-diversity and culturally rich agricultural systems through delivery of products and services into the national and global market environment has the potential to generate positive socio-economic impact. Such impact is the ultimate driver which determines the future existence of individual GIAHS. Individual local GIAHS will benefit from being anchored in national and international mechanisms providing better access to resources and markets, so interventions at these levels are equally relevant.

Effectiveness MS Although the individual building blocks for local, national and global linkage as well as information management and lessons learned (outcome 4) are all contained in the project, these have for several reasons not been effectively cemented together. As stated the failure to do so is rooted in design as well as subsequent project management. Altogether this lowers the effectiveness of the project outcomes. Compared to the more positive scoring for relevance, effectiveness receives a lower rating. The rate reduction is only limited. This is since as a result of the project and associated other interventions under the GIAHS initiative there is remaining potential that FAO as an Organization will forge ahead with the ground work to elevate GIAHS to the programme status. .

Efficiency MU A lowered rating for efficiency is attributed to the effect of both design and management on implementation. The lowered rating is especially caused by the low delivery efficiency of local interventions through a global structure. Had the project stuck to its originally intended catalytic role and left local implementation to national and local dynamics, at most facilitated through a national project component rating would have been more favorable.

Catalytic Role

Production of a public good Y On a limited scale project investments at local level have created public goods. These are relatively insignificant were they have relied on project delivery alone. The national attention for delivery and value of a variety of public goods from GIAHS is more significant than what the project itself has achieved. The aspect of exposure to public good generation and valuation clearly fits better in a catalytic strategy than the direct project delivery of such goods. It must however be accepted that exposure creation is a less tangible feat than creation of physical outputs.

Demonstration Y Similar there is evidence that in countries like China, Chile and Peru there has been an increased sharing of similar support approaches and market integration efforts for individual GIAHS. The effect of this can only be partly attributed to the project as parallel initiatives related to organic production and certification as well as tourism investment have also played a role.

Replication Y Clearly the GIAHS initiative has managed through tireless advocacy of the concept to replicate itself into new countries.

Page 79: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

68

Developments in India and Japan are notable. Further replication is however much dependent on the formal allocation of international credibility to the concept. This depends now on the creation of a FAO GIAHS programme in the short to medium term. Replication momentum may be easily lost in case such a formal status is not provided to GIAHS.

Scaling up N The projects itself as well as the evaluation see it as too early to speak of true up-scaling of the GIAHS concept. As for replication the element of international recognition as well as provision of an improved set of governance rules, setting selection, accession and benefit delivery criteria is required to make the step to up-scaling.

Sustainability

Overall likelihood of risks to

Sustainability: MS The GIAHS initiative is likely to survive into the future in one form or another. In case the international governance aspect would be

eliminated as the result of non-adoption of GIAHS as a FAO programme the impact of the initiative would certainly reduce. This is moreover the case because the likelihood of a separate GIAHS category under UNESCO WHC is negligible. On the contrary if FAO adopts a GIAHS programme there are good opportunities for GIAHS to become institutionally sustainable and develop meaningful cooperation with UNESCO on cultural aspects as well as ILO in relation to labor related issues.

Financial resources S The GIAHS initiative continues to draw in financial resources from a variety of partners. Most recently funding proposals for the initiative were launched independently by China and Japan. Funding of global and national support initiatives therefore does not appear to be a major issue. Still it is important to consider whether funding would continue in a substantial manner if it turned out to be impossible to guarantee a long term international GIAHS programme in FAO.

Socio-economic MS The promise of positive socio-economic impact is welcomed by most custodians of GIAHS. The project has however fallen short in carrying out thorough socio-economic analysis of systems and providing a workable forward looking strategy for individual systems. This does not mean that among GIAHS there will be several that will thrive as the result of increased focus and exposure on their characteristics and the successful valuation of these. As the project has not truly fulfilled a catalytic role in this respect and such positive exceptions rely mainly on existing market forces the rating fails to surpass MS.

Institutional framework and

governance MS Depending on member state support for a GIAHS programme there is about a 50:50 chance that such a programme will emerge in the

foreseeable future. Once a programme is secured the governance framework may be elaborated on the basis of wider Organizational, International and National involvement. This would ultimately lead to an increased rating of either S or HS.

Environmental U/A The scale of project interventions has been too modest and is insufficiently researched against already incomplete baselines to make a relevant assessment of environmental sustainability of interventions. Moreover if project proposed interventions hold the potential of environmental sustainability, but fail to match the requirements for socio-economic sustainability the former may still be jeopardized.

Overall Project Results MS Because of the design problems, less than optimal management and relative institutional isolation, the project as a whole receives a neutral rating. Provided coordinated action to adjust, re-direct and consolidate the GIAHS initiative takes place this rating has the potential to increase.

Page 80: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

69

7 Conclusions

7.1 GIAHS and FAO corporate decision making

353. FAO launched the GIAHS initiative in 2002. Almost from the start, GIAHS has been supported through GEF funding. Support translated in project preparatory (PDF-A and B) assistance as well as the FSP, which is subject of this MTE. Over 10 years the GIAHS initiative was kept alive, and at times against the odds expanded its international recognition. This is despite considerable delays in FSP preparation, halving of anticipated GEF funding levels and meeting with the inherent restrictions of trying to operate what essentially should be a FAO programme effort, through a project modality. Using a project vehicle may have been an unavoidable process step in reaching full maturity of the GIAHS concept. It has also allowed FAO as an Organization to side-step formal corporate decision making regarding GIAHS elevation to programme status. Furthermore adhering to project implementation modalities has led to a disproportional focus on local level interventions at the cost of dealing with fundamental national and global governance issues, including broadening of inter-departmental buy-in for the GIAHS concept within FAO.

7.2 GIAHS and GEF Project Implementation

354. The GEF funded FSP, started in 2008, has been at the centre of a series of associated FAO project and national government initiatives to create an interlinked global, national and local governance and support mechanism for selected individual GIAHS pilot sites. The project and its associated initiatives, part of which can be reckoned as the GEF project baseline, has thus far partly delivered on its outcomes. The Global Outcome 1 has as yet not resulted in a permanent (FAO) programme and statutory secretariat. However at the time of the MTE positive indications in support of these intended project outcomes have been recorded. The National Outcome 2 has been limited in its success to shape and direct National country policies in support of GIAHS. For various reasons, including the project management structure the project has effectively lacked adequate resources leverage potential and instruments to meet its policy and associated aims. Partly by default the project has concentrated on local level interventions (Outcome 3), distillation of lessons learned and communication of the GIAHS concept (Outcome 4). Local level project interventions have been variable in their technical quality, innovation, sustainability and impact. Collective local GIAHS pilots have served the purpose creating a globally visible GIAHS brand, by feeding documentation and promotion activities under project outcome 4.

7.3 GIAHS at National Level

355. There is evidence that, among others through the project, FAO achieved in raising Member State interest and a perception of the usefulness of GIAHS as a national conservation and resource management category. The fact that Member States are asking for institutional formalization of the GIAHS initiative illustrates this. Apart from legitimate concerns over global decline in agro-biodiversity and cultural heritage, there are several more arguments for member states to follow this path; 1) GIAHS allows nations to pointedly draw attention to the diversity and uniqueness of their national agriculture. This is beneficial to the overall perception of their agricultural sector; 2) GIAHS is considered as a potential

Page 81: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

70

internationally recognized framework, which could act as an intermediate step towards classification of systems under existing conventions related to bio-diversity and culture; 3) International GIAHS recognition is seen as an entry point and driver for action on issues of conservation, development support and protection of (indigenous) intellectual property.

7.4 International instruments

356. In the course of the project life to date, a diverse range of possible linkages of GIAHS and existing International Instruments has been pondered. A quality analysis of opportunities and limitations in this regard was produced during the PDF-B stage (2005). It has been known and was admitted as such in the FSP Document that creation of a GIAHS specific international instrument is an unlikely option. This makes partnering with existing convention and treaty secretariats all the more important. As adaptation of conventions and treaties specifically for the purpose of GIAHS is unlikely, terms of association are likely to be dictated by the contents of these existing international instruments

7.5 Systems analysis

357. The majority of GIAHS sites currently selected as pilots under the project, added elsewhere or under consideration do qualify to be part of the initiative. However, incorporation of systems has mainly taken place on grounds of current, if not their past, agro-biodiversity status and cultural history. From project provided system selection criteria, it remains unclear to the MTE how GIAHS have been selected on score weighed basis. The latter weighing of selection criteria must be considered essential to reflect system management and support priorities, but is absent from the current selection approach. Thus, it is observed by the MTE that comparative analysis between representative system exponents as well as system recovery, management and long term sustainability potential does only play a limited, if not negligible role in GIAHS selection and recognition. The global GIAHS governance tier has an explicit responsibility in the introduction, and adherence supervision to quality system analysis. Economic viability and sustainability assessment in view of the rate of established system decline / “erosion” is to be made a key selection consideration. To balance the need for sufficiently rapid GIAHS accession, with the aforementioned need for technical and economic viability it is essential that the burden of proof for individual system candidates is placed with National Governments. Subsequent monitoring of individual GIAHS evolvement should feed into a time bound and step wise designation and system certification process.

7.6 FAO Advisory Role to Governments

358. Since formulation of the original GIAHS concept during the PDF-A phase, design wise the project has moved to a less catalytic / programmatic role and more to a traditional project implementation mode. A deficit in programmatic thinking is partly inherent to the project’s inability to draw on wider FAO programmatic resources. As important is however the inbuilt design balance which the project obtained as the result of both Divisional (NRL) and GEF priorities. While the original global and national level objective / outcomes were maintained throughout the PDF A, PDF B and FSP phases, the emphasis on local project action in pilot GIAHS has proportionally grown. Because of obstacles experienced in

Page 82: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

71

implementation national level outcomes relative domination of local level interventions thus ensued and project operations became increasingly anchored at this level.

8 Recommendations

Recommendations to FAO Corporate Management

Recommendation 1: GIAHS and FAO Corporate Decision Making:

If FAO is to meet its commitment to a permanent status for GIAHS, it is recommended that FAO corporate management takes specific note of growing demand among member states regarding the need to take a clear position on the future status of GIAHS as an initiative within FAO. With the shortest possible delay this must lead to an unequivocal corporate management decision to support and take formal governance steps, which could lead to GIAHS programme elevation. The Evaluation recommends strengthening the M&E capacity under such a follow-up program initiative.

Recommendation 2: GIAHS at National Level:

It is recommended to FAO corporate management through the appropriate committees and governing bodies, to undertake formal process steps which: 1) will squarely put GIAHS programme formulation on the Organizational governance agenda; 2) carry out up-stream programme preparation work under guidance of the NR departmental head; and 3) commit to put GIAHS programme establishment to a vote during the 2013 FAO conference. The combined steps requested from FAO must insist on specific commitment of departmental resources, allocation of consultant support as required, close coordination with the GEF GIAHS project to ensure optimal benefit from lessons learned and capturing its remaining financial and human resources. Programme preparation must however go well beyond issues covered by the GEF project to date, thereby actively distancing itself from the local project emphasis and pulling in corporate technical/ normative and diplomatic resources in the broadest sense.

Recommendations to FAO/NR Management

Recommendation 3: International Instruments

As part of a concerted drive towards GIAHS programme establishment, it is recommended that the NR Department define and make public what is its strategy to facilitate / broker member state interest to accede to existing international instruments, as part of adding content and value of GIAHS recognition. In order to achieve this important goal of turning GIAHS into an umbrella facility, it is essential that NR management prioritizes information exchange and negotiation with partner organizations like UNESCO (cultural heritage), ILO (labour related issues), CBD (bio-diversity aspects) and UNFCC (climate change mitigation and adaptation)

Recommendation 4: System analysis:

To match global, national and local capacities and resources for future GIAHS support it is recommended that GIAHS selection takes active recognition of system (national and

Page 83: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

GIAHS mid-term evaluation report

72

regional) representativeness. Moreover thorough and systematic technical as well as economic system analysis must be introduced in order to prioritize support needs on basis of system recovery / development and economic integration potential. 145

Recommendation 5: FAO Advisory Role to Governments:

It is recommended that, for the remaining project duration, the focus of attention be put on mapping and qualitative analysis of existing national policies in relation to GIAHS, as well as the influence / impact of legal and regulatory instruments. It is essential that the project as well as an emerging FAO programme structure for GIAHS acts in an advisory role to National Governments. The principle of introducing systematic gap analysis prior to the formal start of GIAHS selection designation and eventually certification must be adhered to if FAO wants to live up to its Policy and Normative mandate towards National Governments.

Recommendations to GIAHS Project Management team

Recommendation 6: GIAHS and GEF Project Implementation

During the limited remaining project time146 GIAHS project management is recommended to focus remaining project resources on optimal documentation and transfer of accumulated insight regarding global and national GIAHS organizational mandates. To the maximum extent the project must solicit and structure information inputs from participating national governments as well as FAO Representations in participating countries. Such information as made available by the project, must feed into GIAHS programme establishment, when initiated by corporate management and as agreed by the FAO governing bodies. Local level project initiatives should be continued, but realistically re-scaled on basis of what is considered feasible in terms of available financial and human resources as well as remaining project implementation time. This will require active participation of work plan revisions by both national counterpart agencies and FAO Representations. Limited budget neutral extension for the project may be considered in agreement with the GEF and NR departmental management. Any steps directed towards, project based continuation of the GIAHS initiative must meanwhile be discouraged, as there is no evidence that additional project initiatives will bring programme prospects closer within reach.

145 Integration in terms of opportunity to generate sustainable self-financing in support of system continuity. 146 The project is due to end by mid 2013

Page 84: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

April 2013

Evaluation report, Annexes

Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS) Project- GCP /GLO/212/GFF

Office of Evaluation

Food and Agriculture Organization of the United

Nations

Page 85: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

Food and Agriculture Organization of the United Nations

Office of Evaluation (OED)

This report is available in electronic format at: http://www.fao.org/evaluation

The designations employed and the presentation of material in this information product do not imply the

expression of any opinion whatsoever on the part of the Food and Agriculture Organization of the United

Nations (FAO) concerning the legal or development status of any country, territory, city or area or of its

authorities, or concerning the delimitation of its frontiers or boundaries. The mention of specific companies or

products of manufacturers, whether or not these have been patented, does not imply that these have been

endorsed or recommended by FAO in preference to others of a similar nature that are not mentioned.

The views expressed in this information product are those of the author(s) and do not necessarily reflect the

views or policies of FAO.

© FAO 2013

FAO encourages the use, reproduction and dissemination of material in this information product. Except where

otherwise indicated, material may be copied, downloaded and printed for private study, research and teaching

purposes, or for use in non-commercial products or services, provided that appropriate acknowledgement of

FAO as the source and copyright holder is given and that FAO’s endorsement of users’ views, products or

services is not implied in any way.

All requests for translation and adaptation rights, and for resale and other commercial use rights should be

made via www.fao.org/contact-us/licence-request or addressed to [email protected].

For further information on this report, please contact:

Director, OED

Viale delle Terme di Caracalla 1, 00153

Rome, Italy

Email: [email protected]

Page 86: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

5

Table of contents Annex 1: GIAHS MTE ToR ................................................................................................ 6

Annex 2: Generic ToR for National Evaluation Consultants .......................................... 22

Annex 3: MTE Itinerary for Bart Dominicus................................................................... 25

Annex 4: List of people met ............................................................................................... 27

Annex 5: Evaluation Matrix .............................................................................................. 34

Annex 6: Co-financing details, as provided by the project .............................................. 43

Annex 7: GIAHS Budget Overview June 2012 (Provided by Project) ............................ 45

Annex 8: Objectives / Outcomes by Project Phase ........................................................... 48

Annex 9: Logical Framework Analysis ............................................................................. 51

Annex 10: Extended MTE Findings ................................................................................. 66

Annex 11: Background to Project Consultancies ............................................................. 82

Annex 12: List of Project Publications (Selection) ........................................................... 84

Page 87: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

6

Annex 1: GIAHS MTE ToR

1 Background of the GIAHS Initiative

1.1 Rationale

What are GIAHS?

1. Globally Important Agricultural Heritage Systems are defined as "remarkable land use systems and landscapes which are rich in globally significant biological diversity, evolving from the co-adaptation of a community with its environment and its needs and aspirations for sustainable development".

A Heritage for the Future

2. Worldwide, specific agricultural systems and landscapes have been created, shaped and maintained by generations of farmers and herders based on diverse natural resources, using locally adapted management practices. Building on local knowledge and experience, these ingenious agri-cultural systems reflect the evolution of humankind, the diversity of its knowledge and its profound relationship with nature. These systems have resulted in the maintenance and adaptation of globally significant agricultural biodiversity, outstanding landscapes, indigenous knowledge systems and resilient ecosystems, but above all, in the sustained provision of multiple goods and services, food and livelihood security for millions of poor and small farmers.

GIAHS Partnership Initiative

3. In response to the global trends that undermine family agriculture and traditional agricultural systems, in 2002, during the World Summit on Sustainable Development (WSSD, Johannesburg, South Africa), FAO launched a Global Partnership Initiative on conservation and adaptive management of Globally Important Agricultural Heritage Systems. The overall goal of the partnership is to identify, safeguard and promote international recognition of the Globally Important Agricultural Heritage Systems and their associated landscapes, agricultural biodiversity, knowledge systems and cultures through catalyzing and establishing a long term programme to support such systems and enhance global, national and local benefits derived through their dynamic conservation, sustainable management and enhanced viability. To achieve this goal, GIAHS Initiative implements activities according to three major objectives: (1) leverage global and national recognition of the importance of agricultural heritage systems and institutional support for their safeguard; (2) Capacity building of local farming communities and national institutions to conserve and manage GIAHS, enhance income generation and add economic value to goods and services of such systems in a sustainable fashion; (3) promote enabling policy, regulatory and incentive environments to support the conservation, evolutionary adaptation and viability of GIAHS.

Operations and Funding Support 4. The operations and implementations of the GIAHS Initiative is funded through various projects. The Main donors are the Global Environmental Facility (GEF), BMELV, with contributions from IFAD, Food and Agriculture Organizations (FAO), FAO Turkey Partnership Programme, ITPGRFA, ISESCOand partner governments. The GIAHS Initiative

Page 88: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

7

also received technical contributions from the Chinese Academy of Sciences, United Nations University, MS Swaminathan Research Foundation, Asia Pacific University and other research institutions. The Project finance amounts to an estimated USD 20,342,761 million (see the contributions below). FAO is both the GEF Agency and the executing agency of the project. Funding sources and amounts

Funding Source Amount (USD) Remarks

GEF Allocation 3,500,000

IFAD (thru Oxfam International) 200,000

IFAD Small Grants 200,000

Germany (BMELV) 2,200,000

FAO (cash) 600,000

FAO (in kind) 1,832,761

HEADS 50,000

Roman Forum 1,000,000

Governments

Algeria 100,000

Chile 990,000

China 1,200,000

Peru 1,600,000

Philippines 1,000,000

Tunisia 100,000

FTPP (for Turkey and Azerbaijan) 250,000

Confirmed after GEF FSP endorsement, GIAHS

Initiative budget as of Dec 2011

Japan 5,000,000

UNU (in kind) 50,000

MSRRF (India) 50,000

ISESCO (initial funding support) 220,000

ITPGRFA (for Tunisia) 200,000

Total Project Budget 20,342,761

Intervention Strategy

5. In order to provide systematic support for the conservation and adaptive management of GIAHS, the strategy is to make interventions at three distinct levels. First, at the global level, it will facilitate international recognition of the concept of GIAHS wherein globally significant agricultural biodiversity is harboured, and it will consolidate and disseminate lessons learned and best practices from project activities at the pilot country level. Second, at the national level in pilot/participating countries, the project will ensure mainstreaming of the GIAHS concept in national sectoral and inter-sectoral plans and policies. Third, at the site-level in pilot/participating countries, the project will address conservation and adaptive management of agro-ecosystems at the community level. It is expected that the project will also contribute to sustainable development through: (i) enhancing the benefits derived by local populations and indigenous peoples from the management, conservation and sustainable

Page 89: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

8

use of agricultural biodiversity and natural resources; (ii) adding economic value and sharing derived benefits from these systems; (iii) enhancing food security and alleviating poverty. The project has been implemented in several sites in the following countries:

• GEF pilot countries: Chile, China, Tunisia, Algeria, Peru, and the Philippines • BMELV countries: Kenya and Tanzania • FTPP countries: Turkey and Azerbaijan • IFAD thru Oxfam international: Morocco • IFAD small grants: India and Sri Lanka • ITPGRFA support to GIAHS for Tunisia • Self funded countries: Japan, Iran, additional sites in China and India

1.2 Project objectives

Development Objective: 6. To “protect and encourage customary use of biological resources in accordance with traditional cultural practices that are compatible with conservation or sustainable use requirements” [cf. CBD: Article10(c)], specifically within agricultural systems. 7. Project Purpose/ Immediate Objective (GEF-related): The project objective is to promote conservation and adaptive management of globally significant agricultural biodiversity harbored in globally important agricultural heritage systems or GIAHS. GIAHS are defined as agricultural systems that exemplify customary use, knowledge, innovation and indigenous land management practices essential for the conservation and sustainable use of this agricultural biodiversity. This objective will be realized through the four outcomes described below. 1.2.1 The project has four components (or Outcomes)1 Outcome 1: An internationally accepted system for recognition of GIAHS is in place (Global) 8. Through this outcome the project will aim to raise awareness at the international and national levels of the intrinsic value of GIAHS and the need to promote their long-term sustainability. The underlying strategy for identifying and managing GIAHS will be to avoid or reverse the loss or degradation of essential features and attributes of these systems especially their biodiversity while allowing their necessary evolution and enhancing the socio-economic development of resource users and national benefits. This will require careful consideration of the critical issue of how to meet often-conflicting goals of conservation and development, for instance avoiding creating “ethno-museums” where preserving the key characteristics of the systems might extinguish their human vitality. This is a challenge that requires innovative and adaptive approaches, which the project will devise, develop and demonstrate in the pilot sites.

• Public endorsement of the GIAHS concept, definition and criteria by key international institutions and pilot country governments.

1 The wording below corresponds to the wording in the Logical Framework. The Evaluation team will receive

all original documents in due time for further analysis.

Page 90: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

9

• Establishment of interim GIAHS Secretariat with a statutory mandate and Scientific Advisory Committee, as well as articulation of a process for designating agricultural systems as GIAHS. The institutional arrangements (e.g., structure, composition, TORs, reporting lines) will be developed and agreed upon through an intergovernmental process to be completed by the end of the project. As part of this process feasibility studies and needs assessments will be undertaken.

• Establishment of a sustainable financing mechanism and institutional support for consolidating and expanding the GIAHS approach as a long-term open-ended program.

Outcome 2: The conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National) 9. The focus of this outcome will be on ensuring that key sectoral and inter-sectoral policies and plans (such as policies on protected areas, cultural heritage, in situ conservation of genetic resources for food and agriculture, agricultural extension, public participation, indigenous peoples, land-tenure and access to natural resources) take explicit account of the significance of GIAHS. The following outputs are based on the “Pilot Frameworks” developed under the PDF-B 10. Drawing on PDF-B assessments, identification and implementation of specific measures through which sectoral and inter-sectoral policies and regulations can be improved to support conservation and adaptive management of GIAHS, for instance through official recognition of GIAHS in national policy documents. Concrete activities will include workshops to develop GIAHS designation in national protected area and cultural heritage systems (all countries); development of guidelines to ensure sound environmental management, community participation (PIC) in designated areas; mainstream GIAHS considerations in NBSAPs and GRFA strategies; field visits of policy makers to GIAHS pilots systems to discuss policy bottlenecks and opportunities with farming communities (all countries); development of policy proposals for adjustments of land-tenure and access to natural resource regimes (Algeria, Tunisia, Peru and China); workshops and development of policy proposals to include GIAHS considerations into national legislation on indigenous peoples and minorities (Peru, Chile, the Philippines); proposals for adjusting national, provincial and local policies and programs on sustainable tourism, including guidelines to safeguard community interests and sound management of the agricultural biodiversity and heritage (all countries); lobby and awareness raising activities, including through the identification of GIAHS “champions” in national governments and partnerships with civil society partners. The PDF-B identified the following as key sectors:

• Environment: biodiversity conservation, land and water management, ecological services, protected areas

• Family agriculture: genetic resources conservation and management (including crop wild relatives and wild species, and neglected and underutilised crops), rural development, good agricultural practices, trade and marketing, customary access to natural resources and land tenure system

• Rural development and link with the global economy: marketing of GIAHS products, development of niche markets and agro-tourism, relevant participation and implementation mechanisms for capacity building and decision-making

• Culture and heritage: valorisation of indigenous and indigenous/traditional agricultural patrimony

Page 91: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

10

• Rural education: inclusion of traditional knowledge and agricultural patrimony in primary education at local level

11. Development of capacities of national-level institutions to mainstream GIAHS in sectoral and inter-sectoral plans and policies. The PDF-B identified training on the concept of GIAHS, its importance and ways of mainstreaming it in national policies as the main area where capacity needs to be developed at the national-level. Concrete activities will include workshops and policy-briefs on the concept and importance of GIAHS, including their multiple environmental and livelihood benefits; training sessions on the legal and policy requirements for the conservation of GIAHS and its globally important biodiversity (all countries). Additional (sectoral and inter-sectoral) capacity building needs emerging from the activities above will be responded to as well. Outcome 2: The conservation and adaptive management of globally significant agricultural biodiversity harbored in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National)

• Identification and implementation of specific measures through which sectoral and inter-sectoral policies and regulations can be improved to support conservation and adaptive management of GIAHS, for instance through official recognition of GIAHS in national policy documents.

• Capacities of national-level institutions to mainstream GIAHS in sectoral and inter-sectoral plans and policies will be developed.

Outcome 3: Globally significant agrobiodiversity in pilot GIAHS is being managed and sustainably used by empowering local communities and harnessing evolving economic, social, and policy processes and by adaptation of appropriate new technologies that allow interaction between ecological and cultural processes (Local) 12. The strategy for this outcome explicitly recognizes that change in "traditional" political, social and economic processes is inevitable; they cannot be frozen or re-created. Consequently, it adopts the “adaptive management” approach to explore and develop novel political, social and economic processes that strengthen the existing management systems, and which generate the same biodiversity outcomes as much as possible– that is, maintain the same races, species and agroecosystems. Thus, the processes may be different and contain new and modern elements, but the way they interact with the biophysical world will maintain the values of these agroecosystems. The project has identified a range of different systems to test such new approaches on a case by case basis in a wide variety of settings. These pilot sites are: Chiloe Islands (Chile); Rice-fish system in Longxiang village of Zheijang Province (China); El Oued, Gafsa Oases in (Algeria, Tunisia respectively); Micro del Carmen in the Vilcanota valley and Cuenca de Lares, both in Cusco Department, and Micro Cuenca de San José and Comunidad de Caritamaya, Provincia Acora (bordering on the southern side of lake Titicaca) in Puno Department (Peru); and Ifugao Rice Terraces (Philippines). The outcome will address the obstacles for long-term sustainable management of GIAHS and will help the people living in and around GIAHS to establish strengthened socio-political (governance) and economic processes (markets and alternative livelihood opportunities) that help them address the challenges of today’s world (with all its modern pressures) and let them to take advantage of the opportunities of modern living, while at the same time maintaining the remarkable values (and co-evolving processes) of their agroecosystems. The following site-specific outputs are based on the pilot frameworks developed under the PDF-B.

Page 92: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

11

• Establishment of appropriate stakeholder set-ups at the site level that brings together customary, state and non-government institutions (including private sector actors) that will support local farmers to engage in collaborative management and promotion of GIAHS.

• Identification and monitoring of political and socio-economic processes that impact biodiversity and cultural values in GIAHS in order to enhance positive effects and empower local communities with knowledge and tools to minimise negative effects

• Screening, testing and deployment of environmentally friendly technologies and practices that improve the management and productive capacity of agroecosystems and their traditional crops, as well as new co-evolved races.

• Design and implementation of programmes for alternative and/or supplementary livelihoods to assist people meet the challenges of reduced opportunities for working directly on the land

• Documentation and publishing of information about the case histories of establishment and management of GIAHS.

Outcome 4: Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion and upscaling of the GIAHS in other areas/countries and creation of the GIAHS network (Global, National, Local)

• In order to facilitate further replication and expansion of the GIAHS concept, this outcome will focus on documenting lessons learned and best practices, and enabling exchange of experience.

• Implementation of the project’s M&E plan at global and pilot-country levels and adapting project

• Preparation of a global publication on lessons learned and best practices emerging from the pilot countries on the identification, designation and participatory management of GIAHS.

• Preparation of scientific reports and publications arising from project investigations and implementation.

• Creation and maintenance of a web-based information management system that will include a database on existing and potential GIAHS, and will also be designed to serve as an electronic forum for sharing information and experiences across the various pilots. Pilot system communities and pilot countries will provide information through their own web-sites and publications.

Expected Outcomes (GEF-related):

• An internationally accepted system for recognition of GIAHS is in place (Global) • The conservation and adaptive management of globally significant agricultural

biodiversity harbored in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National)

• Globally significant agricultural biodiversity in pilot GIAHS is being managed effectively by indigenous and other traditional communities (Local)

• Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion of the GIAHS network (Global)

Page 93: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

12

1.3 GIAHS execution and management structure

13. At FAO HQ: The Global Project Implementation Unit, GPIU (also called GIAHS Secretariat) is responsible for the day-to-day management of the Project. The GPIU is composed of the Director of Land and Water Division, as the Global Coordinator, supported by a Technical Officers (one is based in Tanzania), one full-time administrative staff, two part-time administrative staff, and seasonal consultants. The Information and Communications officer will be responsible for development and implementation of the communication strategy, data collection and management, web-site maintenance and the overall outreach to all the stakeholders and target groups. 14. At international level: the project has established an International Steering Committee (ISC) as the umbrella policy body for the project. The ISC is composed of FAO (Executing Agency), National Focal Point Institutions (NFPIs) from the participating countries, the national GEF Operational Focal Points, and representatives from co-financing bodies. Representatives of potential GIAHS farming communities and their organisations and networks are also invited to assist and inform Outcomes of the Project. The ISC main tasks are: (i) advising on the legal and institutional frameworks that will be proposed and recommending steps to be taken for their adoption; (ii) providing strategic advice and assisting in the formal international recognition of GIAHS, including the mandate and legal framework of the institutional mechanism for supporting them prior or during the GIAHS Forum. The GIAHS Forum is being held every other year. In-Country Arrangements 15. In each country, the project is guided by a Project Steering Committee (PSC). Their role is analogous to that of the ISC, but at the national level. Members of the PSC would be nominated by participating Ministries but will also include representatives from non-governmental, universities and academes, civil society and private sector organizations. The PSC will consider and endorse the Annual National Work Plan (ANWP) prior to submission to the GPIU including specifications for work within the country over the next year, and support the timely undertaking of the work plan. 16. The project is implemented in five pilot systems represented by 12 pilot sites in six countries: Chile, China, Tunisia, Algeria, Peru, and the Philippines. National governments and ministries will play a leading role in the project activities, by providing technical support and other services through their administrative system. Financial arrangements will be made through letters of agreement with the leading institutions of each pilot system for the implementation of stakeholder participation processes. 17. Each Pilot System is coordinated locally by a national focal point institution (NFPI) which designated a national and/or local coordinator (or recruit, if need be). The designated national/local coordinators is responsible for the technical, financial and administrative follow-up of the selected site(s). The designated (or recruited) coordinator ensures the implementation of the work plan, both at the local and national levels.

2 Purpose of the Evaluation

Page 94: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

13

18. In accordance with the Project Document, an independent Mid-term Evaluation is scheduled for the project. However, due to unexpected delay in Chile, Peru and the Philippines, this mid-term evaluation was requested to be delayed by one year. It will be thus held this year (2012). 19. As stated in the Project Document, the Evaluation will determine progress being made towards achievement of outcomes and will identify corrective actions if necessary. It will, inter alia:

• review the effectiveness, efficiency and timeliness of project implementation; • analyse effectiveness of implementation and partnership arrangements; • identify issues requiring decisions and remedial actions; • identify lessons learned about project design, implementation and management; • highlight technical achievements and lessons learned; • analyse whether the project has achieved any of the benchmarks for moving towards

a second phase (upscaling and outscaling) of the GIAHS; and • propose any mid-course corrections and/or adjustments to the Work Plan as

necessary. 20. The Terms of Reference for this Mid-term Evaluation were prepared in close consultation with FAO Office of Evaluation (OED) and the FAO GEF Coordination unit within FAO Investment Centre (TCID) in accordance with the evaluation policies and procedures of FAO and the GEF. The TORs were discussed with and endorsed by the participating GEF funded GIAHS countries and GIAHS participating partner countries. 21. The MTE is planned to take place in the period mid April to mid May 2012; and the presentation of the findings not later than June 2012.

3 Scope of the Evaluation

22. The evaluation will critically assess the GEF funded project through internationally accepted evaluation criteria, i.e. relevance, efficiency, effectiveness, impact and sustainability, taking into account the specific dependence relationship of the project with baseline activities 23. The evaluation will assess the project looking in particular at:

a. Relevance of the project to: CBD Article 10c and 8j; GEF Strategic Programme (SO-2) under which the project was approved; GIAHS countries development priorities and needs for adaptive and sustainable management of agricultural heritage and the safeguard of its functionalities; FAO Global Goals, Strategic Objective F Sustainable management of land, water and genetic resources and improved response to global environmental challenges affecting food and agriculture; Organizational Result/s FO1G130, FO1G202, and Core Functions2/;

b. Robustness and realism of the theory of change underpinning the project, including

logic of causal relationship between inputs, activities, expected outputs, outcomes and impact (against specific and development objectives) and validity of indicators. Particular attention will be paid to the validity of assumptions and risks as initially

2 See Annex 2 of this ToR

Page 95: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

14

identified in the project document and whether unforeseen issues are affecting negatively project implementation and progress towards objectives.

c. Quality and realism of the project’s design, including:

• Duration; • Stakeholder and beneficiary identification. • Institutional set-up and management arrangements; • Approach and methodology; • Quality of the logframe

d. Financial resources management, including:

• Adequacy of budget allocations to achieve outputs and promote outcomes; • Coherence and soundness of Budget Revisions in matching budget adjustments to

implementation needs and project objectives; • Rate of delivery and budget balance at the time of the evaluation.

e. Management and implementation:

• Effectiveness of management, including quality and realism of work plans; • Efficiency and effectiveness of operations management; • Gaps and delays if any between planned and achieved outputs, the causes and

consequences of delays and assessment of any remedial measures taken; • Efficiency in producing outputs; • Effectiveness of internal monitoring and review processes; this will also include

information provided by the project through GEF Tracking Tools; • Efficiency and effectiveness of coordination and steering bodies, e.g. the ISC, PSC; • Quality and quantity of administrative and technical support by FAO to the project,

including the Lead Technical Unit and the Budget Holder; • Quality and quantity of administrative support by the Country Offices to the National

Counterpart Institutions.

f. Timeliness and adequacy of resources and inputs made available through co-financing from participating countries and resource partners.

g. Extent to which the expected outputs have been produced, their quality and

timeliness, against project planning at the time of the evaluation, i.e. at completion of year 3 of implementation (key outputs and outcomes from the log-frame for the evaluation to assess are listed in Annex 3).

h. Use made by the project of FAO’s normative products and actual and potential

contribution of the project to the normative work of the Organization. Equally, the team will identify potential products by the Initiative for wider diffusion of lessons learned, and assess the identification and implementation of specific measures (global, national levels) to promote and enhance understanding of GIAHS leading to mainstreaming of GIAHS in each country.

i. Assessment of gender mainstreaming in the initiative. This will cover:

• Analysis of how gender issues were reflected in project objectives, design, identification of beneficiaries and implementation;

• Analysis of how gender relations and equality are likely to be affected by the initiative;

Page 96: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

15

• Extent to which gender issues were taken into account in project management.

j. The prospects for sustaining and up-scaling the initiative's results by the participating countries after the termination of the currently on-going project/s. The assessment of sustainability will include, as appropriate:

• Institutional, technical, economic and social sustainability of proposed innovations, coordination mechanisms and processes;

• Perspectives for uptake and mainstreaming of the newly established mechanisms capacities at local, national and global levels;

k. Overall performance (cost-effectiveness) of the project/programme: extent to which

the initiative has attained, or is expected to attain, its intermediate/specific objectives and FAO Organizational Result/s (impact), and hence, contribute to the relevant Strategic Objectives and carry out its Core Functions; this will also include the identification of actual and potential positive and negative impacts produced by the initiative, directly or indirectly, intended or unintended.

l. The catalytic role of the project and its impact in supporting the creation of an

enabling environment with a view to achieve sustainable global environmental benefits.

24. Based on the above analysis, the evaluation will draw specific conclusions and formulate recommendations for any necessary further action by Government, FAO and/or other parties to ensure sustainable development, including any need for follow-up action. The evaluation will draw attention to specific good practices and lessons of interest to other similar activities. Any proposal for further assistance should include specification of major objectives and outputs and indicative inputs required.

4 Evaluation methodology

The evaluation will adhere to the UNEG Norms & Standards3/. 25. The evaluation will adopt a consultative and transparent approach with internal and external stakeholders throughout the evaluation process. Triangulation of evidence and information gathered will underpin the validation of evidence collected and its analysis and will support conclusions and recommendations. 26. The evaluation will make use of the following tools:

• review of existing documentation: project document, outputs, monitoring reports (e.g. Project inception Report, Meeting Reports and reports from other relevant meetings; Project implementation Reports; quarterly and six-monthly progress reports), and other internal documents including consultant and financial reports, Project website, annual work plans, publications and other materials and reports;

• semi-structured interviews (direct or by phone) with key informants, stakeholders and participants, including the Global Coordinator, Technical Officers, other members of the GPIU as relevant; the Lead Technical Unit and Budget Holder,

3 http://www.uneval.org/normsandstandards; both GEF and FAO evaluation units are members of UNEG and

subscribe to its Norms and Standards

Page 97: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

16

National coordinators and related national counterparts, staff and national institutions involved in project implementation (including in the countries that will not be visited, to canvass their views on achievements, issues and ways forward);

• direct observation during field visits. 27. To the maximum possible extent, an evaluation matrix, standardised interview protocols and check-lists etc will be used throughout the evaluation, so as to ensure comparability of findings across countries.

i. In the visited countries, the team will also meet other actors who are actively working on GIAHS, to assess actual and potential areas of collaboration and partnership;

ii. If considered appropriate/timely, interaction with ISC members and GIAHS participating countries shall be done.

28. The evaluation team will visit selected pilot countries in Latin America, in Asia and in North Africa, in order to capture a varied perspective of the different context in which the programme operates and the specific challenges and progress. The selection criteria included the state of progress, allowing a mix of more and less advanced, location of sites and FAO offices, time and cost considerations, and were agreed by OED, the GPIU, LTU and the FAO GEF Unit in the Investment Centre Division. 29. The Evaluation Team will independently decide which outputs and outcomes to assess in detail, within resources available, after consultation with OED and programme management.

5 Consultation process

30. The evaluation team will maintain close liaison with the FAO Office of Evaluation, FAO Representation at regional/country level, and all key stakeholders. Although the mission is free to discuss with the authorities concerned anything relevant to its assignment, it is not authorized to make any commitment on behalf of the Government, the GEF or FAO. 31. At the end of the mission, the Evaluation Team will present its preliminary conclusions and recommendation to the LTU, FAO-GEF Unit and OED either in HQ or in one of the pilot country, as appropriate. 32. The draft Evaluation Report will be circulated among key stakeholders before finalisation; comments and suggestions will be incorporated as deemed appropriate by the Evaluation Team.

6 Composition of the Evaluation Team

33. Mission members will have had no previous direct involvement in the formulation, implementation or backstopping of the initiative. All will sign the Declaration of Interest form of the FAO Office of Evaluation. 34. The Evaluation Team is responsible for conducting the evaluation and applying the methodology. The Team Leader, and other team members to the extent possible, will participate in the briefing and debriefing meetings, discussions and field visits. Under the

Page 98: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

17

guidance of the Team Leader, all members will contribute to the evaluation with written inputs. 35. The Evaluation Team is fully responsible for its independent report which may not necessarily reflect the views of the Government or of FAO. An evaluation report is not subject to technical clearance by FAO although OED is responsible for ensuring conformity of the evaluation report with standards for project/programme evaluation in FAO. 36. The Evaluation Team will comprise the following skill mix:

• Demonstrated experience in the evaluation of large/complex, regional/global technical assistance projects

• Familiarity with the objectives of the GEF OP-13/SO-2 Biodiversity programme • Familiarity with CBD, ITPGRFA, and other multi-lateral instruments • Understanding of governance, political, economic and institutional issues associated

with natural resources, traditional knowledge, culture, indigenous peoples. • Integrated Natural Resources Management • Sustainable Land Management

37. In so far as possible, the Team will be regionally and gender balanced. It will tentatively be composed as follows:

• one International Team Leader with extensive experience in evaluation of regional development programs, with good writing skills in English, and fluent in French and/or Spanish;

• one National Natural Resources Management Specialist in each country to be visited, with experience in sustainable agriculture and rural development, agroecology, biodiversity management/natural resource policy development, with extensive field experience in the countries participating in the programme.

38. All team members will have a University Degree and a minimum of 15 years of professional experience, or equivalent level of competence, in their respective areas of specialization. Individual Terms of reference will be developed referring to these global Evaluation ToR, upon recruitment of each team member.

7 The Evaluation Report

39. After the desk review and before the first field visit, the Team Leader will submit to OED an outline proposal for the report, together with the Evaluation Matrix. The annotated outline Report Structure included in Annex I to these Terms of Reference can be modified by the Evaluation Team, as long as the key contents are maintained in the report and the flow of information and analysis is coherent and clear. 40. The Evaluation Report will illustrate the evidence found that responds to the evaluation issues, questions and criteria listed in the ToR. It will include an executive summary. Supporting data and analysis should be annexed to the report when considered important to complement the main report. 41. The recommendations will be addressed to the different stakeholders and prioritized: they will be evidence-based, relevant, focused, clearly formulated and actionable.

Page 99: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

18

42. The Team Leader bears responsibility for submitting the final draft report to OED within 4 weeks from the conclusion of the mission, which will provide comments within one week. The revised report will be circulated to other FAO stakeholders, who within two additional weeks will submit to the team comments and suggestions that the team will include as appropriate in the final report within one week. 43. Annexes to the evaluation report will include, but are not limited to:

• Terms of reference for the evaluation; • Profile of team members; • List of documents reviewed • List of institutions and stakeholders met during the evaluation process;4/ • Itinerary of the evaluation team mission; • Data collection instruments (e.g. copies of questionnaires, surveys – if applicable) •

44. The report will be prepared in English, with numbered paragraphs. Translation of “Summary, Conclusions and Recommendations” in Chinese, French and Spanish will be FAO’s responsibility. Ratings 45. In order to facilitate comparison with routine reporting to GEF and contribute to the GEF programme leaning process (IWLearn), the Evaluation will rate the project elements on the GEF six-point scale system: Highly Satisfactory (HS), Satisfactory (S), Marginally Satisfactory (MS), Marginally Unsatisfactory (MU), Unsatisfactory (U), and Highly Unsatisfactory (HU). 46. Each of the elements listed below should be rated separately, with comments and then an overall rating given.

• Achievement of objectives • Attainment of outputs and activities • Progress towards meeting GEF-4 focal area priorities/objectives • Cost-effectiveness • Impact • Risk and Risk management • Sustainability5/ • Stakeholder participation • Country ownership • Implementation approach • Financial planning • Replicability

4 The team will decide whether to report the full name and/or the function of the people who were interviewed

in this list. 5 Sustainability will be assessed in terms of Likelihood: Likely (L): There are no risks affecting this dimension

of sustainability. Moderately Likely (ML). There are moderate risks that affect this dimension of sustainability. Moderately Unlikely (MU): There are significant risks that affect this dimension of sustainability Unlikely (U): There are severe risks that affect this dimension of sustainability.

Page 100: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

19

• Monitoring and evaluation.

8 Annexes to the ToR

Annex 1, Annotated Report Outline Annex 2, FAO Global Goals, Strategic Objectives and Organization results Annex 3, Key outputs and outcomes from the log-frame Other Annexes: Project documents, progress reports; Technical reports and BTORs; Budget revisions (if any)

Page 101: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

20

Annex 3 to the ToR - Key outputs and outcomes from the log-frame The key outputs and outcomes from the log-frame for the evaluation to assess are listed below; however the evaluation team should feel free to explore in detail other specific outputs and outcomes. Outcome 1: An internationally accepted system for recognition of GIAHS is in place (Global)

• International and national local partners with technical and financial support to GIAHS are mobilized (i.e. IFAD, FTPP, ITPGRFA, ISESCO, UNU, GOJ, India, MSRRF, etc)

• Case studies with potential for GIAHS upscaling and outcaling are carried out • Proposals from Japan, India, additional sites in China are received and included in

the existing GIAHS • Received 2 GIAHS proposals from the Government of Japan • Expressions of interest from various governments and organizations are received • The concept of GIAHS has received great appreciation by many Institutions,

Organizations and Government. References are made to GIAHS in official UN (CBD - Decision X/34. and Ramsar Convention - Resolution X.31) and non UN Documents and declarations; several book and publications refer to the concept and its international recognition.

• Promotion and awareness raising are conducted at various international and national events e.g. CBD meetings, National Fair (country levels), other relevant gatherings/meetings of interest to GIAHS.

• Print and media promotion (local tv series, radio broadcasting, etc) Outcome 2: The conservation and adaptive management of globally significant agricultural biodiversity harbored in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National)

• National Project Framework are developed for the pilot countries • National Guidelines for dynamic conservation are developed (Tunisia Charter for

GIAHS, National Guidelines in China) while other countries are currently developing their own charter

• In-country funding support are sought (internal and external) Outcome 3: Globally significant agricultural biodiversity in pilot GIAHS is being managed effectively by indigenous and other traditional communities (Local)

• Number of capacity buildings and trainings specific for: (i) agricultural biodiversity conservation and management; (ii) land and water management; (iii) livelihoods approaches and development; (iv) crop management; (v) training for women farmers; and (vi) agro-eco-tourism and marketing

• Seed exchange programme • Local and national publications

Outcome 4: Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion of the GIAHS network (Global)

Page 102: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

21

• Two international Fora: one in Buenos Aires during the World Forestry Congress

2009; and the recent was in Beijing, China n 2011. The forum is being organized every other year, as a platform for knowledge exchange, information dissemination, GIAHS products exhibits, and discussion of GIAHS dynamic conservation issues and strategies.

• GIAHS information/publication materials: (i) booklets, brochures, posters have been published in UN language (Arabic, Chinese, English, French, Spanish); (ii) Video documentary of individual sites are made available; (iii) experiences and lessons learned are published in scientific journals, science magazines and other form of media; (iv) at country levels, websites and social networks are created and link to global GIAHS website.

• GIAHS Regional Workshops/networking and cross visitations (i.e. China, Philippines, India, Japan in Asia; Algeria, Morocco and Tunisia in the Maghreb Region)

• GIAHS concept promotion through side events, university lectures, fora, etc

Detailed progress of the project is available i.e. bi-annual and annual progress reports.

Page 103: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

22

Annex 2: Generic ToR for National Evaluation Consultants

This Terms of Reference covers National Consultants (NC) inputs to the GIAHS Mid Term Evaluation. Country level preparatory information gathering and analytical work carried out by independent NC form an integral contribution to the global GIAHS mid-term evaluation under auspices of the International Evaluation Consultant (IEC). Because of standard administrative arrangements NC are hired by the FAO Representation in their respective countries of origin. As far as their day to day duties, described below, NC will however directly coordinate with and report to the IEC, responsible for the independent mid-term review. While keeping the IEC in the loop of all communication, for any logistical and cross national contracting related matters the NC shall directly communicate with the responsible Evaluation Manager (EM), who represents the Office of Independent Evaluation of FAO (OED) for this mid-term evaluation. The importance of locally and nationally specific contextual appreciation and clear understanding of GIAHS initiative implications are reflected by the following detailed list of duties. The synthesis of the overall global impact of GIAHS falls outside the scope of work of the NC, as this requires the comparative view among participating countries and system approaches. The immediate issues and impacts generated in function of national country initiatives including regional exchange and support as well as interaction with the global GIAHS management and governance structure are however elements to be covered under the NC contributions:

1) Collect relevant national policy and legal and regulatory materials with bearing on the

GIAHS initiative. Such information shall cover agro-ecological/ biodiversity conservation, cultural, tenure, labour organization / regulation, rural service delivery and product branding/marketing aspects. The collection and subsequent research of materials will be guided across countries through the structure and specific evaluation questions posed in the Evaluation Matrix (EMa)6

2) Analyse the collected information against the evaluation questions and indicators provided by the EMa. Reflect the analytical results in a standard tabulated format conforming to the structure of the EMa. The analysis will ensure the availability of a country specific constructive critical analysis of issues pertaining to adequacy/ shortcomings/lacking elements of the national enabling policy, legal and regulatory environment relating to GIAHS.7

3) Provide a detailed account of the country level institutional structure relating to GIAHS. Obtain / update / produce a detailed organogram indicating specific institutional participation, including those of the national steering mechanism, ministerial, local government and academic institutions.

4) Collect, analyse and annotate/comment on a full set of GIAHS national steering committee (NSC) meeting minutes. Comments should systematically cover the key decision making points / sequence of NSC meetings. Any available evidence of such decision making in the form of recorded correspondence shall be collected through the national GIAHS focal point in order to be made available for further evaluation analysis.

6 This EMa will be produced by the IEC in close coordination with the OED EM and will be issued in a

cleared format, prior to the start of NC inputs at country level. 7 The reporting language will be pre-agreed for each individual NC contract, when possible preference will be

given to English as this will be the medium for the overall Mid Term Evaluation Report, the importance of quality reporting will however prevail over language considerations.

Page 104: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

23

The NC will also produce a graphical timeline8, which will illustrate decision making interaction among national stakeholders .

5) Based on availability of the updated GIAHS organogram prepare an annotated list of organizational representatives, which includes their designations, contact information and a graphic overview of reporting and decision making lines9. These protocols will specifically include the discussion of how site selection criteria have been applied and which specific implementation arrangements have been instituted at national level. Any variation of implementation arrangements at the local (site ) level will be highlighted.

6) Document national action undertaken following the approval of pilot sites. This shall include but not necessarily be limited to information on national budget, human and other material resource allocation and interaction with the International (Global) GIAHS secretariat on issues of external support, classification and accreditation of selected pilot systems.

7) Request, obtain and categorize documentary evidence, such as correspondence, notes to file mission and back to office reports regarding the FAO Representation role in relation to liaison, advocacy / promotion and operational support to national GIAHS initiatives. Similarly provide documentary evidence of any communication interaction (including technical support missions) from the regional and global GIAHS level. Hereby the specific purpose of such visits / interaction should be categorized by its main functions10

8) Based on local interviews, document11 site specific issues of a systemic rather than a one off character e.g. dealing with legal, institutional, marketing, subsidy and other incentive systems. The review of specific GIAHS sites shall be guided by the use of specific Evaluation Questions from relevant sections of the EMa. This in order to ensure comparative analysis among individual sites.

9) Aside from the systemic analysis of each visited solicit and document queries / comments from key national and local GIAHS stakeholders. These will include but will not be necessarily limited to national and local line departments staff, researchers and last but not least community representatives of GIAHS sites12 . An important element of the ToR is to establish through interviews how GIAHS stakeholders perceive that the initiative has been able to add innovative elements, additional value to use and sustainability of concerned agricultural / cultural systems. Distinction will be made between the effect resulting from local, national and global interaction. Sustainability of introduced innovations action will be rated on a scale from 1-613 ( where 6 is highly sustainable and 1 is marginally sustainable)

10) Through consultation with the national GIAHS focal point, the FAO Representation, and while regularly keeping abreast the independent international GIAHS mid-term evaluation consultant, schedule a detail visit programme for the latter. This programme shall include interview meetings with key national GIAHS personnel, relevant FAO representation staff, other collaborating organizations / partners and allow for the visit to at least one pilot site.

8 A standard format will be provided prior to the mission. 9 This will be in the form of a second organizational diagram, substituting the institutional blue print for an

operational one. 10 Attendance of Seminars/ Workshops / Meetings, National Organizational Support, Local Organizational

Support, Local Level Technical Backstopping or a proportional combination of the aforementioned. 11 Documentation will result in a site specific Aide Memoire, for which a standard outline will be provided

prior to the NC mission. 12 A gender balanced approach will be adhered to. 13 Where 6 is highly sustainable and 1 is marginally sustainable

Page 105: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

24

11) Act in a facilitating capacity during the country visit of the independent international GIAHS mid-term evaluation consultant, which will entail introductions during national and regional interview meetings and accompanying of the consultant to the pilot site.

12) Reserve 1 day availability, post evaluation visit, to allow for follow-up communication with the independent international GIAHS mid-term evaluation consultant and to act on behalf of the consultant when and where national information requests must be pursued.

13) Produce a concise report, constituting of the elements as outlined in the various sections of this ToR and obtain clearance of the report content from the IEC prior to claiming final payments.14

14) Take on any additional duties, as may be expected in the ordinary line of work as described in this ToR and based on consultation with the IEC and the OED EM.

Qualifications: Experienced Agronomist or Rural Sociologist with affinity and background of national traditional agricultural systems Schedule:

14 Reporting time and delivery schedule is indicated in the schedule section below.

Page 106: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

25

Annex 3: MTE Itinerary for Bart Dominicus

Date Location Objective

April 2, 2012 FAO Rome Briefing Meetings April 3, 2012 FAO Rome Briefing Meetings April 17, 2012 Amsterdam Departure Philippines and China

country visits April 17, 2012 Manila Arrival Philippines April 18, 2012 Manila Desk Work April 19, 2012 Manila Evaluation Meeting with DENR April 19, 2012 Manila / Ifugao Night travel by car to Ifugao April 20,2012 Ifugao – Manila Site visits and talks with local

stakeholders, return by car to Manila. Meet with FAOR staff upon return

April 21, 2012 Manila Meeting with Ben Malayang former secretary DENR, Desk Work

April 22, 2012 Manila to Beijing Departure for China Country visit. Arrangement contacts for next day trip to Qingtian County

April 23, 2012 Beijing to Wenzhou by air and Qingtian by minibus

Onward travel with delegation to Qingtian

April 24, 2012 Qingtian and Longxian village Presentations about GIAHS is China, discussions and field visit during the afternoon

April 25, 2012 Qingtian, Wenzhou and Beijing Evaluation interviews, return to Beijing via Wenshou.

April 26, 2012 Beijing Continuation Evaluation interviews and discussions

April 27, 2012 Bejing Desk Work and Free Time April 28, 2012 Return to Amsterdam and onward

to home Retrurn travel

May 02, 2012 Travel to Rome from Brussels Arrival Rome during evening May 3, 2012 Rome to Tunis Meetings and discussions, prior to

departure for Tunis May 4, 2012 Tunis Meetings , FAOR, MoE Tunisia,

INRAT, Association Appui aux Initiatives de Developpement

May 4, 2012, Afternoon Tunis to Gafsa Travel by car May 5, 2012 Gafsa Evaluation Interviews ASM , INP,

Commissariat Regional au Developpement, Inspecteur Regional du Patrimoine a Gafsa, Association Iniatives a Gafsa

May 6, 2012 Gafsa Field visit Oase de Gafsa, meeting with farmers, ecotourism representatives, ASM, MoA in Gafsa town.

May 7, 2012 Gafsa to Tunis Discussions with national Evaluator Mr. Ferchichi

May 8, 2012 Tunis to Rome Travel by air. May 8 to 14, 2012 Rome Meetings and Desk Work at FAO

HQ May 14, 2012 Rome to Brussels Return to home base June 06, 2012 Brussels to Lima, via Madrid and

Bogota Start country visits Peru and Chile

June 07, 2012 Lima Meetings FAOR, MINAM June 08, 2012 Travel to Juliaca , Puno Meeting Regional Directorate of

Agriculture, Puno

Page 107: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

26

June 09, 2012 Chaupi Sahuacasi, Distict Ázangaro _ San Jos

Field Visit, Meetings with farmers

June 11, 2012 Lares Field Visit, Meetings with Beneficiaries Project

June 12 , 2012 Juliaca to Cusco Meeting Cusco Dept. natural resources, Field visit community of Huama District Lamay, Cusco

June 13, 2012 Lima to Santiago Start of Chile country visit, Meeting with FAOR

June 14, 2012 Santiago Meetings with MoA and MoE, Departure for Puert Montt

June 15 , 2012 Puerto Montt Joint meeting with Agriculture Office, tourism , Environment _ Export Promotion of Regional Govt., Meeting with Intendant ) Political Authority of the region, departure for Chiloe

June 16, 2012 Chiloe , Castro and across island Meetings with local authorities, including Governor of Chiloe tourism private sector, NGO´s farmers

June 17, 2012 Chiloe to Puerto Montt and onward travel to Santiago

June 18, 2012 Santiago Desk Work, wrap up June 19/20, 2012 Santiago Departure to Brussels via

Madrid

July 22 to 31, 2012 Brussels to Rome and return Brussels

FAO HQ, Desk Work, Intial report drafting, follow up meetings

Page 108: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

27

Annex 4: List of people met

Location and Date Name Function and Role Philippines Department of Environment and Natural Resources (DENR) Manila, Philippines Mrs. Annaliza Teh Undersecretary Mrs. Cristy Regunay

Mrs. Maritess Agayatin,

Protected Areas and Wildlife Bureau (PAWB)

Mr. Clarence Baguilat,

Regional Executive Director (DENR-Cordillera)

Mrs. Linda Mamaradlo

Bureau of Soils and Watershed Managment (BSWM) Mr. Lauro Hernandez, Former Consultant on Institutional

Issues Department of Agriculture Mr. Adam Borja, Representation of MoA Congress of the Phillipines Mr. Felix Baguilat

Representative of Congressman Teddy Baguilat

FAO (Representation) Mr. Aristeo Portugal,

Assistant FAOR for Programs

Mrs. Sarah Lacson Assistant FAOR for Finance and Administration

Mrs. Brenda Saquing,

FAO GIAHS project facilitator (local consultant)

Mrs. Janet Corpus Former FAO GIAHS project facilitator (local consultant) by phone

Local Representatives Project Site Hungduan Mauro Bandao Municipal Planning and Development

Officer Farmers of Hungduan China Ministry of Agriculture , International Cooperation Department Beijing, China Mr. Wang Ying Director-general Mr. Luo Ming Division Chief Ms.An Jing Division Chief Mr.Zhao Lijun Government Official Ministry of Agriculture International Exchange Service Center Mr.Sun Yonghua Division Chief Mr.Xiong Zhe Interpreter Institute of Geographic Science and Natural Resources Research (IGSNRR) , Chinese Academy of Sciences

Mr.Liu Yi Director Mr.Zhang Ming Chief Foreign Affairs Division Mr.Min Qingwen Professor Ms.He Lu PhD Candidate Ms.Shi Yuanyuan GIAHS Project Assistant Qingtian County Government Fangshan Township Mr.Ye Qunli Vice Governor Qingtian County Government, Foreign Affairs Office,

Page 109: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

28

Location and Date Name Function and Role Mr.Xu Xiaofang Director

Qingtian County Agricultural Bureau Mr.Xu Xiangchun Director Mr.Chen Junxiong Government Official Mr.Wu Minfang Senior Agronomist Qingtian County Forestry Bureau Mr.Zhang Liyong Director Qingtian County Scienic Area Toursim Bureau Mr.Yang Guang Director Qingtian County Publication Board for Culture, Radio, TV and News Mr.Chen Lin Deputy Director

Qingtian County Rural Area Offcie Mr.Wang Xuhai

Deputy Director

Fangshan Township Government, Qingtian County Mr.Wang Jun Secretary of Communist Party

Mr.Xu Wei Administrative Chief Longxian Village Village Committee, Fangshan Township, Qingtian County Mr.Wu Suiming Diretor Tunisia Bureau sous-Régional de FAO à Tunis Tunis, May 4th Mr Bénoit Horemans FAO Representative Mr. Ahmed Bouguecha Programme Officer Melle Damergi Programme Officer Ministère de l’Environnement Mr. Abdelhakim Issaoui GEF Focal Point Mr. Nabil Hamada Bio-diversity Focal Point INRAT Mr Mohamed Kharrat Association Appui aux Initiatives de Développement Mr Ayoub Ben Ali ASM Gafsa Mr. Lazhar Chérif Président UNESCO Mr Mustapha Khanoussi National Officer Centre Régional de Recherches en Agriculture Oasienne Mr Ali Zouba Directeur Général ASM Gafsa Gafsa, May 5th Mr Atef Dhahri Coordinateur du Projet Commissariat Régional au Développement agricole Mr Taieb Kahouli, Directeur Général Inspection de Patrimoine National Mr Mondher Ibrahim: Inspecteur Régional L'association Initiatives à Gafsa May, 6th Mr Mohamed Arfa: Président Farmers in Gafsa Oasis Ministère de l'Agriculture. Tunis, May 7th Mr Hedi Mekni: Legal Expert Peru FAO Representation Lima, 7/6/2012 Mrs. Jazmine Casafranca FAOR a.i. Programme Assistant Ministry of Environment (MINAM)

Page 110: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

29

Location and Date Name Function and Role Dra. María Luisa Del Río, Directorate of Biological Diversity Mrs. Kathia Soto Mr. Walter Huamaní Office of Int. Cooperation Mrs. Danilia Rojas Mrs.. Miriam Cerdán, Former Director General GIAHS Project Puno Province Juliaca, 8/6/2012 Mr. Alipio Canahua Puno GIAHS Project Coordinator,

University of Puno Puno Directorate Agricultural Planning (Planificación Agraria GORE) Ing. Claudio Ramos, Director Dra. Nemesio Limachi Gerente Community of Ccaritamaya: Caritamaya, 9/6/2012 12 Representatives

Mr. Evaristo Huarcaya Mr. Bellarmino Quispe Mr. Naim Cruz Mrs. Sonia Ticona Others

Local Farmers involved with GIAHS Project

Community of Chaupi Sacawasi Chaupi Sacawasi 6 Representatives

Mr. Julio Paco Mr. Martin Quispe Mr. Fulgencio Quispe Mrs Amanda Peralta Others

Local Farmers involved with GIAHS Project

Municipality of San José San Jose 26 participants Local Farmers involved with GIAHS

Project FAO Regional GIAHS Coordinator (Consultant) Cusco, 10/6/2012

Mr. Hernan Mormontoy

District Municipality Lares Lares, 11/6/2012 Mr. Urbano Alfa

Mr. Clímaco Villegas Mr. Nerio Palma Mr. Cristobal Pfocco Mr. Herminio Chavez Mr. Zenobio Rodriguez, Mr. Alberto Junco, Mr. Juan Quispe

Municipal GIAHS Responsible Municipal GIAHS Responsible Economic Affairs Regidor Regidor Community Representative Community Representative Community Representative

District Municipality Lamay Lamay, 12/6/2012 Mr. Juan Quispe

Mr. Raul Mamami Mrs. Noemi Callasi Mr. Rafael Benites Mr. Justino Pilco

Economic Affairs ONG Ricchay Ayllu Huama Comminity Representative Huama Comminity Representative

Former FAO GIAHS Consultant Lima, 12/6/2012 Dr. Mario Tapia Agro-Biodiversity Expert and Peru

GIAHS Co-designer

Chile FAO Chile Santiago, June 13th Alan Bojanic FAO Chile Representative Vicente Ossa Field Program National Monitoring

Officer

Page 111: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

30

Location and Date Name Function and Role Andrea Sáez Assistant FAO Chile Ministry of the Environment Santiago, June 14th Javier García Head of Office of International Affairs Ximena George-Nascimento Office of International Affairs,

Operational Focal Point GEF Ministry of Agriculture/ODEPA Fernando Astaburuaga General Coordinator of Foreign Affairs Alfredo Apey Alternate National GIAHS Coordinator

Replacing Mrs. Teresa Agüero. Ignacio García

Head of Department of Horizontal Issues

GIAHS Steering Committee Puerto Montt, June 15th

Rodrigo Mardones Regional Secretary, Ministry of Agriculture (Agriculture SEREMI)

Gonzalo Larraín Service Head of National Tourism Planning (SERNATUR)

Edgar Wilhem Regional Secretary, Ministry of the Environment (Environment SEREMI)

Roxana Monsalve Professional Assistant, Directorate of Export Promotion (ProChile)

Pablo Barrientos Former GIAHS Regional Consultant. Feb 2011 to Feb 2012

Marianela Rosas GIAHS regional consultant. Since April 2012

Regional Authority Juan Sebastián Montes Regional Intendant (Political) Steering Committee Chiloé, June 16th Carlos Venegas Director Centre of Education and

Technology (CET) Cecilia Guineo President of the Chiloé Association of

Organic Farmers Santiago Elmudesi Rilán Group (Hospitality, Tourism

Industry) Regional Authority César Zambrano Governor of Chiloé’s Province Other stakeholders Renato Arancibia Pehuén Tourism Cyril Christensen Chiloé Natural Rafael Maripán Sheep Cheese Producer Macarena Rivera Head of Productive Development.

Castro’s Municipality Mirta Miranda Vegetable Grower Curahue

Community Juan Cárcamo Technician of the Local Development

Program (PRODESAL) from Castro’s Municipality

Algeria FAO Representation Algiers Nabil ASSEF Bousb Ratiab INRAA Fouad Chehat Director Head of International Cooperation DSA

Page 112: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

31

Location and Date Name Function and Role M.Fallah Youcef Director of Agriculture Services National Office of Agriculture lands (ONTA) Wilaya d’El-Oued Nacer-Eddine Rekrouki Director Agriculture Council of Wilaya (CAW) Bachir Oubira Representative Association Sundarouss Ferjani President Messai Secretary Association WIFAK Ababa President Boudisa Secretary Association Assala Abdelkader Ouassa President Abdelwahab Hariz Secretary Local Farmer Representatives 15 Fellahs Farmers and owners of Ghouts Local Authorities: People’s Congress of Wilaya Oued (Assemblée Populaire de la Wilaya : APW) Haouameda President Rezgue President Agriculture Commission Directorate Youth and Sports Djafer Nadjar Director Toufik Mosbah Service Head Layache Haddou Service Head Department of Forests Hakim Arbouche Conservator Robbah Abdelhadi Responsible circumscription Directorate Culture Mohammed Kachou Director Ammar Chemsa Documentation Kamal Kachou Heritage Directorate Environment Kamel Bouguemmara Director Directorate of Hydrology Bachir Lacheari Association Mih-Ouanssa 15 Fellahs Members Ghout NGOs Ouassa Representative Mahda Ali Representative Boudissa Representative Hachifa Representative Ferdjani Representative Ababa Representative Society Errizk Elhacen Ali Menai Director

FAO HQ DG Cabinet April 3rd He Chang Chui Special Adviser to theDG ADG NR April 3rd + follow-up Alexander Mueller ADG NR – Overseeing GIAHS

programmatic aspects NRL April 3rd + follow-up Parviz Koohafkan Director NRL, Global GIAHS

Page 113: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

32

Location and Date Name Function and Role Coordinator

April 2nd + regular follow-up

Mary Jane RamosDelaCruz Technical Officer day to day in charge GIAHS

April 3rd + follow-up Poalo Groppo Strategic back up services to GIAHS

project April 3rd Sally Bunning April 3rd JeanMarc Faures Senior Officer, Water Resources

Management

May 10th Dominique Vallee PES and Bhutan Resource Person NRDD April 3rd MarieAude Even April 3rd Stephane Jost ESA April 3rd

Kinlay Dorjee Economist, Bhutan Resource Person

TCID May 8th + follow-up

Barbara Cooney GEF focal Point FAO

May 8th + follow-up

Rikke Olivera (TCID) GEF focal Point FAO

TCIO May 8th

Katia Medeiros GIAHS Philippines back-stopping

AGND April 2nd Florence Egal Agro-biodiversity and Nutrition AGP April 3rd + follow-up

Caterina Batello Resource Person Agro-biodiversity

April 3rd

Thomas Price GFAR Secretariat , Resource Person Civil Society – Private Sector and GIAHS

FOCM April 3rd

Thomas Hofer Mountain Partnership

FAOSNE May 9th Noureddine Nasr Tunisia FAOR Programme Officer FAO RAP May 10th

Thierry Facon Senior Regional Water Management Officer for Asia and the Pacific

FAO RAP By Phone

Prof. MS Swaminathan Swaminathan Research Foundation, GIAHS Stering Committee Chairman, Resource Person GIAHS India

UNESCO By Phone Mechtild Rossler Key Contact Person GIAHS WB (former GEFSEC) By Phone Andrea Kutter GEF project officer during PDF-B start

FSP UNU Japan By Phone Kazem Vadafari Eco-tourism GIAHS, Philippines,

Japan Luo Hui Liang PDF-B involvement China, now Japan

Page 114: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

33

Location and Date Name Function and Role ILEIA (former) By Phone Frank van Schoubroeck Former GIAHS consultant on behalf of

the Gov. of the Netherlands University of Florence By Phone Prof. Mauro Agnoletti, Resource Person GIAHS Italy,

Environmental History University of Tuscia By Phone Prof. Gregio Resource Person GIAHS Italy

Page 115: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

34

Annex 5: Evaluation Matrix

The evaluation matrix below served as a general guide for the evaluation. It provided directions for the evaluation; particularly for the collect of relevant data. It was used as a basis for interviewing people and reviewing project documents. It also provided a basis for structuring the evaluation report as a whole.

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

Evaluation criteria: Relevance - How does the Project relate to the main objectives of the UNCBD and to the conservation of globally and nationally significant (agricultural)

biodiversity?

Is the Project

relevant to

UNCBD

objectives?

� How does the Project support the objectives of the UNCBD?

� How does the Project support the strategic priorities for biodiversity of the GEF?

� Does the Project participate in the implementation of the UNCBD in the countries were it conducts project interventions ?

� Is the GEF incremental cost principle being respected?

� Level of coherence between project objectives and those of the UNCBD

� Degree of coherence between the project and national priorities, policies and strategies in the area of biodiversity conservation

� Level of coherence between the project and National specific legislation and regulations

� UNCBD status in countries harbouring GIAHS sites

� Extent to which the project is actually implemented in line with incremental cost argument

� Project documents � National policies and

strategies to implement the UNCBD. and environment more generally

� Key government officials and other partners

� UNCBD web site

� Documents analyses � Interviews with

government officials and other partners

Is the Project

relevant to FAO

Strategic and

Functional

Objectives?

� How does the project relate to and support FAO’s SO and FO

� Existence of a clear relationship between the project objectives and relevant elements of the FAO SO and FO

� Project document and country specific intervention GIAHS site intervention proposals

� FAO Strategic Framework 2010-2019

� National policies and strategies to implement the UNCBD and environment more generally

� Key government officials and other partners

� Documents analyses � Interviews with

government officials and other partners

Is the Project � How does the Project support the environmental � Degree to which the project support � Project documents � Documents analyses

Page 116: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

35

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

relevant to the

environmental

objectives of

countries

harbouring GIAHS

sites?

objectives of National and lower level Governments in countries harbouring GIAHS?

� How country-driven is the Project? � Does the Project adequately take into account

the national realities, both in terms of institutional framework and programming, in its design and its implementation?

� To what extent were national partners involved in the design of the Project?

� Were the GEF criteria for Project identification adequate in view of actual needs?

national environmental objectives � Degree of coherence between the project

and nationals priorities, policies and strategies

� Appreciation from national stakeholders with respect to adequacy of project design and implementation to national realities and existing capacities?

� Level of involvement of Government officials and other partners into the project

� Coherence between needs expressed by national stakeholders and FAO-GEF criteria

� National policies and strategies

� Key government officials and other partners

� Interviews with government officials and other partners

Is the Project

addressing the

needs of target

beneficiaries?

� How does the Project support the needs of target beneficiaries?

� Is the implementation of the Project been inclusive of all relevant Stakeholders?

� Are local beneficiaries and stakeholders adequately involved in Project design and implementation?

� Does the project provide tangible benefits to direct beneficiaries which may help ensure their enduring role in agricultural biodiversity conservation?

� Strength of the link between expected results from the project and the needs of target beneficiaries

� Degree of involvement and inclusiveness of beneficiaries and stakeholders in project design and implementation

� Beneficiaries and stakeholders

� Needs assessment studies � Project documents

� Document analysis � Interviews with

beneficiaries and stakeholders

Is the Project

internally coherent

in its design?

� Is there a direct and strong link between expected results of the Project (log frame) and the Project design (in terms of Project components, choice of partners, structure, delivery mechanism, scope, budget, use of resources etc)?

� Is the length of the Project conducive to achieve Project outcomes?

� Level of coherence between project expected results and project design internal logic

� Level of coherence between project design and project implementation approach

� Program and project documents

� Key project stakeholders

� Document analysis

� Key Interviews

How is the Project

relevant in light of

� How is the project functionally linked to other donor supported programme and project

� Degree to which project was coherent and complementary to other donor

� Other Donors’ policies and programming documents

� Documents analyses � Interviews with other

Page 117: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

36

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

other donors

programmatic and

project support ?

initiatives? � How do GEF-funds help to fill gaps (or give

additional stimulus) that are crucial but are not covered by other donors?

� How does this funding ultimately translate in incremental environmental benefits?

programming in countries harbouring GIAHS sites?

� List of programs and funds in which the future developments, ideas and partnerships of the project are eligible?

� Other Donor representatives

� Project documents

Donors

Future directions for similar Projects

� What lessons have been learnt and what changes could have been made to the Project in order to strengthen the alignment between the Project and the Partners’ priorities and areas of focus?

� How could the Project better target and address the priorities and development challenges of targeted beneficiaries?

� Data collected throughout evaluation

� Data analysis

Evaluation criteria: Effectiveness – To what extent are the expected outcomes of the Project being achieved?

How is the Project

effective in

achieving its

expected outcomes?

� Is the Project being effective in achieving its expected outcomes?

o Creation of a system and statutory body to manage GIAHS at global scale

o Adaptation and mainstreaming of significant legislation and regulation in support of GIAHS at individual country level

o Site specific and effective management of relevant agro biodiversity in GIAHS sites, through empowerment of immediate stakeholders and beneficiaries.

o Key principles for conservation of GIAHS documented analysed and ready for up-scaling on the basis of lessons learned from pilot country GIAHS experiences

� Presence of / or progress toward s the establishment of a statutory body and secretariat governing GIAHS at a global scale.

� Documentary evidence of incorporation of GIAHS as part of national legislative and regulatory frameworks

� Demonstration of environmental site management adaptations / innovations as a direct or indirect result of project involvement having a positive effect on maintaining agro-biodiversity , and which are at the same time economically and financially underpinned in the local natural resource management reality.

� Degree of progress towards project and independent scientific documentation and analysis of key requirements and adaptive management principles required for implementation of GIAHS

� Project documents � Key stakeholders � Research findings

� Documents analysis � Meetings with main

Project Partners including FAO, Project Team, National Gov. representatives and other Partners

� Interviews with project beneficiaries

Page 118: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

37

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

How is risk and

risk mitigation

being managed?

� How well are risks and assumptions being managed?

� What was the quality of risk mitigation strategies developed? Were these sufficient?

� Are there clear strategies for risk mitigation related with long-term sustainability of the project?

� Completeness of risk identification and assumptions during project planning

� Quality of existing information systems in place to identify emerging risks and other issues?

� Quality of risk mitigations strategies developed and followed

� Project documents and evaluations

� FAO (including FAO representation ) and project staff and Project Partners

� Document analysis � Interviews

Future directions for similar Projects

� What lessons have been learnt for the project to achieve its outcomes?

� What changes could have been made (if any) to the design of the project in order to improve the achievement of the project’s expected results?

� How could the Project be more effective in achieving its results?

� Data collected throughout evaluation

� Data analysis

Evaluation criteria: Efficiency - How efficiently is the Project implemented?

Is Project support

channeled in an

efficient way?

� Was adaptive management used or needed to ensure efficient resource use?

� Did the Project logical framework and work plans and any changes made to them use as management tools during implementation?

� Were the accounting and financial systems in place adequate for Project management and producing accurate and timely financial information?

� Were progress reports produced accurately, timely and responded to reporting requirements including adaptive management changes?

� Was Project implementation as cost effective as originally proposed (planned vs. actual)

� Was the leveraging of funds (co-financing) happened as planned?

� Were financial resources utilized efficiently? Could financial resources have been used more

� Availability and quality of financial and progress reports

� Timeliness and adequacy of reporting provided

� Level of discrepancy between planned and utilized financial expenditures

� Planned vs. actual funds leveraged � Cost in view of results achieved compared

to costs of similar projects from other organizations

� Adequacy of project choices in view of existing context, infrastructure and cost

� Quality of RBM reporting (progress reporting, monitoring and evaluation)

� Occurrence of change in project design/ implementation approach (ie restructuring) when needed to improve project efficiency

� Existence, quality and use of M&E,

� Project documents and evaluations

� FAO, National and Provincial / District Gov. of GIAHS pilot countries

� Beneficiaries and Project partners

� Document analysis � Key Interviews

Page 119: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

38

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

efficiently? � How was RBM used during program and

Project implementation? � Were there an institutionalized or informal

feedback or dissemination mechanisms to ensure that findings, lessons learned and recommendations pertaining to Project design and implementation effectiveness were shared among Project stakeholders, FAO and GEF Staff and other relevant organizations for ongoing Project adjustment and improvement?

� Did the Project mainstream gender considerations into its implementation?

feedback and dissemination mechanism to share findings, lessons learned and recommendation on effectiveness of project design.

� Cost associated with delivery mechanism and management structure compare to alternatives

� Gender disaggregated data in project documents

How efficient are

partnership

arrangements for

the Project?

� To what extent partnerships/linkages between institutions/ organizations were encouraged and supported?

� Which partnerships/linkages were facilitated? Which one can be considered sustainable?

� What was the level of efficiency of cooperation and collaboration arrangements? (between local actors, FAO/GEF and countries harbouring GIAHS sites and pilot project initiatives

� Which methods were successful or not and why?

� Specific activities conducted to support the development of cooperative arrangements between partners,

� Examples of supported partnerships � Evidence that particular

partnerships/linkages will be sustained � Types/quality of partnership cooperation

methods utilized

� Project documents and evaluations

� Project Partners � Beneficiaries

� Document analysis � Interviews

Does the Project

efficiently utilize

local capacity in

implementation?

� Was an appropriate balance struck between utilization of international expertise as well as local capacity?

� Did the Project take into account local capacity in design and implementation of the Project?

� Was there an effective collaboration with scientific institutions with competence in (agro-) BD conservation?

� Proportion of total expertise utilized taken from N

� Number/quality of analyses done to assess local capacity potential and absorptive capacity

� Project documents and evaluations

� FAO, Project Team and Project partners

� Beneficiaries

� Document analysis � Interviews

Future directions for similar

� What lessons can be learnt from the Project on efficiency?

� Data collected throughout evaluation

� Data analysis

Page 120: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

39

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

Projects � How could the Project have more efficiently addressed its key priorities (in terms of management structures and procedures, partnerships arrangements etc…)?

� What changes could have been made (if any) to the Project in order to improve its efficiency?

Evaluation criteria: Impacts - What are the potential and realized impacts of activities carried out in the context of the Project?

How is the Project

effective in

achieving its long

term objectives?

� Will the project achieve its overall project goal to “protect and encourage customary use of biological resources in accordance with traditional cultural practices that are compatible with conservation or sustainable use requirements

� Will the project achieve its objective to

promote conservation and adaptive management of globally significant agricultural biodiversity harboured in globally important agricultural heritage systems or GIAHS.

� Change in global appreciation of the importance of agro-biodiversity as a heritage important to humankind and future sustainable food production systems

� Change in capacity: o To identify and highlight the

significance of GIAHS o Incorporate multiple stakeholder levels

in a joint and mutually reinforcing conservation system adaptation and management effort (global, national and local)

o For implementation of related laws and strategies through adequate institutional frameworks and their maintenance,

� Project documents � Key Stakeholders � Research findings; if

available

� Documents analysis � Meetings with FAO,

National Project Teams and Project Partners

� Interviews with project beneficiaries and other stakeholders

How is the Project

effective in

achieving the

objectives of the

UNCBD?

� What are the impacts or likely impacts of the Project? o On the local environment; o On poverty; and, o On other socio-economic issues.

� Provide specific examples of impacts at those three levels, as relevant

� List of potential structural funding sources which may help to ensure long term sustainability of UNCBD objectives

� Project documents � UNCBD documents � Key Stakeholders � Research findings

� Data analysis � Interviews with key

stakeholders

Future directions for the Project

� How could the Project build on apparent identified successes and learn from its weaknesses in order to enhance the potential for impact of ongoing and future initiatives?

� Data collected throughout evaluation

� Data analysis

Evaluation criteria: Sustainability - Are the initiatives and results of the Project allowing for continued benefits?

Are sustainability � Were sustainability issues integrated into the � Evidence/Quality of sustainability strategy � Project documents and � Document analysis

Page 121: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

40

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

issues adequately

integrated in

Project design?

design and implementation of the Project? � Evidence/Quality of steps taken to address sustainability

evaluations � FAO personnel and Project

Partners � Beneficiaries

� Interviews

Financial

Sustainability

� Did the Project adequately address financial and economic sustainability issues?

� Are the recurrent costs after Project completion sustainable?

� Level and source of future financial support to be provided to relevant sectors and activities in countries harbouring GIAHS after project end?

� Evidence of commitments from government or other stakeholder to financially support relevant sectors of activities after project end

� Level of recurrent costs after completion of project and funding sources for those recurrent costs

� Project documents and evaluations

� FAO and project personnel and Project Partners

� Beneficiaries

� Document analysis � Interviews

Organizations

arrangements and

continuation of

activities

� Were the results of efforts made during the Project implementation to date well assimilated by organizations and their internal systems and procedures?

� Is there evidence that Project partners will continue their activities beyond Project support?

� What degree is there of local ownership of initiatives and results?

� Were appropriate ‘champions’ being identified and/or supported?

� Degree to which project activities and results have been taken over by local counterparts or institutions/organizations

� Level of financial support to be provided to relevant sectors and activities by in-country actors after project end

� Number/quality of champions identified

� Project documents and evaluations

� FAO and project personnel and Project Partners

� Beneficiaries

� Document analysis � Interviews

Enabling

Environment

� Were laws, policies and frameworks addressed through the Project, in order to address sustainability of key initiatives and reforms?

� Were the necessary capacities for legislative framework enhancing processes and law / regulatory enforcement built?

� What is the level of political commitment to build on the results of the project?

� Efforts to support the development of relevant laws and policies

� State of enforcement and law making capacity

� Evidences of commitment by the political class through speeches, enactment of laws and resource allocation to priorities

� Project documents and evaluations

� FAO and project personnel and Project Partners

� Beneficiaries

� Document analysis � Interviews

Institutional and � Is the capacity in place at the national and local � Elements in place in those different � Project documents and � Interviews

Page 122: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

41

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

individual capacity

building

levels adequate to ensure sustainability of the results achieved to date?

management functions, at the appropriate levels through national strategies, regulatory systems, economic and financial incentive structures and accompanying sustainable fund flows, stakeholder skill development and inbuilt attention for stakeholder communication and coordination on GIAHS related subjects.

evaluations � FAO and project

personnel and Project Partners

� Beneficiaries � Capacity assessments

available

� Documentation review

Social and political

sustainability

� Did the Project contribute to key building blocks for social and political sustainability?

� Did the Project contribute to local Stakeholders’ acceptance of the new practices?

� Example of contributions to sustainable political and social change in support of the UNCBD.

� Project documents and evaluations

� FAO and project personnel and Project Partners

� Beneficiaries

� Interviews � Documentation

review

Replication � Were Project activities and results replicated elsewhere and/or scaled up?

� What was the Project contribution to replication or scaling up of innovative practices or mechanisms that support the UNCBD objectives?

� Number/quality of replicated initiatives � Number/quality of replicated innovative

initiatives � Volume of additional investment leveraged

� Other donor programming documents

� Beneficiaries � FAO and project personnel

and Project Partners

� Document analysis � Interviews

Challenges to

sustainability of the

Project

� What are the main challenges that may hinder sustainability of efforts?

� Have any of these been addressed through Project management?

� What could be the possible measures to further contribute to the sustainability of efforts achieved with the Project?

� Challenges in view of building blocks of sustainability as presented above

� Recent changes which may present new challenges to the project

� Education strategy and partnership with school, education institutions etc.

� Project documents and evaluations

� Beneficiaries � FAO and project personnel

and Project Partners

� Document analysis � Interviews

Future directions for the Project

� Which areas/arrangements under the Project show the strongest potential for lasting long-term results?

� What are the key challenges and obstacles to the sustainability of results of the Project initiatives that must be directly and quickly addressed?

� How can the experience and accumulated project findings and experience influence the

� Data collected throughout evaluation

� Data analysis

Page 123: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

42

Evaluated component

Sub-Question Indicators Sources Data Collection

Method

strategies for GIAHS elsewhere? � Are national decision making institutions in

countries harbouring GIAHS (Parliaments and executive) ready to improve their strategy for agro-biodiversity conservation?

Page 124: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

43

Annex 6: Co-financing details, as provided by the project

Co-Financing as Approved by GEF

Name of Co-Financier

Indicated (USD) Actual (USD) Remarks

In-kind cash combined in kind Cash

FAO 1,832,762 600,000 No estimation

Germany/EU 2,200,033 2,663,811

HEADS 149,457 No estimation 100,000 spent for India activities

The Christensen Fund 1,200,000

IFAD 199,906 200,000

Roman Forum 366,000 2,979,822

10,000,000

thru the Indian Kashmir State for India Saffron

National Governments 2,719,516 2,000,000

10,297,977

Sub Total

4,918,278 9,329,218

14,247,496

23,261,788

Additional Resources After Project Approval, as of January 2012

Countries/Donors

FAO Turkey Partnership Programme for Turkey and Azerbaijan

250,000

Libya for Libyan Mountain Heritage

1,009,000

funds is available waiting for peace and order situation

IFAD (thru Oxfam) for Morocco

200,000

Japan 5,000,000

Page 125: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

44

IFAD Small Grant for India and Sri Lanka

200,000

MSSRF (India) 50,000

United Nations Unievrsity 50,000

ISESCO 220,000

ITPGRFA (for Tunisia 200,000

Sub Total

7,179,000

Distribution of National Governments'Co-Financing Reported Total Co-Financing (USD)

Algeria 70,000

China 2,388,125

Chile 81,852

Peru 2,000,000

Philippines 348,000

Tunisia 5,374,000

Total 10,261,977

Note: as of January 2012

Page 126: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

45

Annex 7: GIAHS Budget Overview June 2012 (Provided by Project)

GCP/GLO/212/GFF GIAHS Project

1. Budget Overview by year and country, as of June 2012

Country

Resources

Allocation

Framework

(USD)

Actual Expenditures (USD)

Total

2008 2009 2010 2011 2012

Algeria

200,000 30,436 17,950 2,027 50,413

Chile

600,000 1,286 101,339 87,442 18,224 208,291

China

500,000 140,815 110,707 152,154 36,010 439,685

Peru

600,000 24,908 25,812 73,685 66,005 190,410

Philippines

500,000 44,088 77,639 53,572 32,942 208,241

Tunisia

100,000 23,592 13,975 32,911 1,618 72,096

HQ

1,000,000 13,296 179,114 213,586 331,363 115,784 853,144

Total

3,500,000

13,296

444,239

561,008

731,127

272,610

2,022,280

Page 127: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

46

2. Overview of Budget Expenditures (in USD) by country and account code, as of June 2012

Account Code Account Description

Indicated

Budget

(USD)

Algeria Chile China Peru Philippines Tunisia HQ Total

5014 Contracts 840,020 18,052 119,135 359,040 64 105,195 29,893 37,003 668,383

5011 Professional Salaries* 1,027,500 - - - - - - 425,730 425,730

5021 Travel 262,982 12,146 31,364 52,187 38,888 48,622 14,858 173,659 371,724

5013 Consultants 509,566 - 51,139 6,789 107,246 8,247 1,995 164,702 340,118

5023 Training 561,150 5,079 6,519 6,618 18,196 24,492 25,233 2,000 88,138

5028 GOE 247,032 2,455 134 5,959 1,754 12,487 117 38,470 61,375

5024 Exp Procurement 2,250 508 - 5,505 19,421 1,121 - 1,842 28,396

5025 Non Exp Procurement 49,500 12,172 - 3,587 4,840 4,636 - 477 25,713

5040/5050 OVH/Chargeback 0 - - - - - - 9,261 9,261

5020 Locally cont labor 0 - - - - 3,364 - - 3,364

5026 Hospitality 0 - - - - 77 - - 77

Total Expenditures 3,500,000 50,413 208,291 439,685 190,410 208,241 72,096 853,144 2,022,280

Resources Allocation Framework (USD) 200,000 600,000 500,000 600,000 500,000 100,000 1,000,000 3,500,000

Current Budget Status (USD) 149,587 391,709 60,315 409,590 291,759 27,904 146,856 1,477,720

Budget expenditure (%) 25% 35% 88% 32% 42% 72% 85% 58%

* Indicated budget for Professional services include budget for professional services and general service as per prodoc.

Page 128: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

3. Overview of expenditures (in USD) by project components

Component

Component 1

Component 2

Component 3

Component 4

Component 5

$-

$500,000

$1,000,000

$1,500,000

$2,000,000

$2,500,000

$3,000,000

$3,500,000

Indicated Budget

47

Overview of expenditures (in USD) by project components, as of June 2012

Component Indicated Budget Actual

Expenditures

Expenditures

Status

(A) (B) (B ÷ A)

Component 1 $ 374,445 $ 198,352 53%

Component 2 $ 534,442 $ 251,309 47%

Component 3 $ 1,108,152 $ 749,509 68%

Component 4 $ 1,172,742 $ 684,150 58%

Component 5 $ 310,220 $ 138,961 45%

$ 3,500,001 $ 2,022,280 58%

Indicated Budget Actual Expenditures

$374,445 $198,352

$534,442 $251,309

$1,108,152

$749,509

$1,172,742

$684,150

$310,220

$138,961

Expenditures by Project Components

Component 5

Component 4

Component 3

Component 2

Component 1

Component 5

Component 4

Component 3

Component 2

Component 1

Page 129: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

48

Annex 8: Objectives / Outcomes by Project Phase

PDF-A, PDF-B and FSP, (copied out of respective project documents) PDF-A PDF-B FSP Objective 1: Improving understanding of globally important ingenious agricultural heritage systems (GIAHS), of their diverse environmental, socio-economic and cultural attributes, their global importance and knowledge systems with regard to agricultural biodiversity as well as their landscape diversity, cultural and natural heritage, and dynamic evolution;

OBJECTIVE 1 - Enhanced global understanding and recognition of GIAHS, by informing, raising awareness and mobilising recognition of the global significance of GIAHS by multiple national and international stakeholders and public, and leveraging sustained institutional, financial and global policy incentives and support for their safeguard and continued evolution.

Outcome 1: An internationally accepted system for recognition of GIAHS is in place (Global) Through this outcome the project will aim to raise awareness at the international and national levels of the intrinsic value of GIAHS and the need to promote their long-term sustainability. The underlying strategy for identifying and managing GIAHS will be to avoid or reverse the loss or degradation of essential features and attributes of these systems especially their biodiversity while allowing their necessary evolution and enhancing the socio-economic development of resource users and national benefits. This will require careful consideration of the critical issue of how to meet often-conflicting goals of conservation and development, for instance avoiding creating “ethno-museums” where preserving the key characteristics of the systems might extinguish their human vitality. This is a challenge that requires innovative and adaptive approaches, which the project will devise, develop and demonstrate in the pilot sites.

Commentary

The PDF-A phrasing in respect of the global project objective speaks about improvement of understanding (highlight) and can therefore be considered as modest in its scope. The PDF-B goes further by using the terms enhancing global understanding and adding the element of recognition. (highlight) Outcome 1 which is the equivalent further increases the stakes for this objective/ outcome by stating that an internationally accepted recognition system is in place (highlight)

Objective 2: Generating increased recognition by multiple stakeholders of the global significance of these agricultural systems and their harvested and non-harvested biodiversity and thereby to leverage policy, institutional and financial support for their safeguard, sustainable evolution and, as appropriate, the replication of valuable attributes.

OBJECTIVE 3: Promotion of conducive legal and policy environments and incentive structures: Enhanced awareness and capacity of national and local policy makers to address the key policy and legal barriers to dynamic conservation and recognition of GIAHS, through identification of targeted innovative policy and legal tools, recommendation of institutional mechanisms for their safeguard, and development of economic and social incentive structures for their enhanced sustainability and viability. The key national

Outcome 2: The conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National) The focus of this outcome will be on ensuring that key sectoral and inter-sectoral policies and plans (such as policies on protected areas, cultural heritage, in situ conservation of genetic resources for food and agriculture, agricultural extension, public participation, indigenous peoples, land-tenure and access to natural resources) take explicit account of the significance of GIAHS.

Page 130: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

49

policies and legal tools will be selected and targeted on the basis of two factors: (a) whether they are key barriers to dynamic conservation in the selected sites; and (b) how they can contribute to increased recognition and political support of GIAHS at the national and global levels. Where possible, these policy and legal tools will be not only developed, but also applied during the life of the project, but given the relatively short period of this Catalytic stage, it is expected that actual policy and legal reform will most likely be felt in the follow-up long term programme.

Commentary

It is worth mentioning that whereas in the PDF-A the national policy and institutional objective is listed as objective 2 the position is changed to objective 3 as presented by the PDF-B document. The original order of presentation, global-national-local, is then again maintained in the FSP PRODOC. In this respect the PDF-B phase appears to have been more realistic in regard of the objective/outcome; a) by listing it later in the project framework, b) by emphasizing the catalytic nature of the project phase itself, and pointing at the project limitations in time. (see highlight) . The end result of this project framework development process, in the FSP formulation takes a serious shift in the direction of a project “ can do “presentation. The realistic limitations of the catalytic project phase are no longer referred to and the document speaks about mainstreaming of sectoral policies in pilot countries (see highlight)

Objective 3: Building the capacity of national and local institutions and providing support to local communities and populations in selected, global priority sites, for the demonstration and development of strategies and management techniques and the creation of opportunities and incentives to promote the preservation of such biodiverse land-livelihood systems. Such support efforts to promote their dynamic conservation and sustained viability would include: (a) the conservation and sustainable use and, where necessary, rehabilitation of their agricultural biodiversity and genetic patrimony, ecosystem services and landscape heterogeneity; (b) the safeguarding and recognition of the dynamism provided by the combination of local knowledge systems, cultural inheritance and social

OBJECTIVE 2 – Demonstration of dynamic conservation in selected, globally important but threatened priority systems, through the development and testing of strategies and participatory methods for their safeguard and sustainable management, the creation of economic opportunities and incentives, and building the capacity of farming communities and populations and local and national institutions, to promote the preservation of GIAHS biodiverse land use-livelihood systems, sustain and enhance the global benefits they produce, and better understanding, assessment and monitoring of GIAHS. Efforts will ensure due responsiveness to gender and other socio-economic differentiation in the society.

Outcome 3: Globally significant agrobiodiversity in pilot

GIAHS is being managed and sustainably used by empowering

local communities and harnessing evolving economic, social,

and policy processes and by adaptation of appropriate new

technologies that allow interaction between ecological and

cultural processes (Local) The strategy for this outcome

explicitly recognizes that change in "traditional" political,

social and economic processes is inevitable; they cannot be

frozen or re-created. Consequently, it adopts the “adaptive

management” approach to explore and develop novel political,

social and economic processes that strengthen the existing

management systems, and which generate the same biodiversity

outcomes as much as possible– that is, maintain the same races,

species and agroecosystems. Thus, the processes may be

different and contain new and modern elements, but the way

they interact with the biophysical world will maintain the

Page 131: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

50

organisation; (c) mitigating threats of degradation and root causes of dysfunction and enhancing environmental and socioeconomic benefits at local and global levels and; (d) adding economic, environmental and cultural value to products, artefacts and knowledge systems of GIAHS through supportive policies and local area development strategies that provide incentives for their sustainability.

Governmental and non-governmental planning, research, and extension workers will be trained in appropriate tools and methods. The results of the demonstration activities will feed directly into selecting key legal and policy tools that need to be developed in Objective 3, as well as into the advocacy tools and categorization of GIAHS of Objective 1.

values of these agroecosystems.

Commentary

By associating national and institutional capacity building with local action the PDF-A phase takes the position of a catalytic agent. The PDF-B phase already adopts a more pro-active position, by making it’s the projects role to demonstrate and test strategies and methods (highlight). Thus there is less talk about project facilitation and more about action undertaken by the project. Outcome 3. Under the PDF-B an explicit interrelationship link between national, global and local is made for the first time. (highlight) . While the PDF-A already lists possible interventions to promote preservation and sustainability this is done in relative general terms and the incentive mechanism proposed relies on supportive (national) policies. (highlight) This can at the same time be seen as a condition for achieving such sustainability and a project caveat in case supportive policies are not in place during the project life time. The equivalent outcome 3 of the FSP further scales up the ambitions of the project. This, takes place by no longer referring to demonstration targets but stating pilot (wide) sustainably managed agro-biodiveristy (highlight). Clearly the demand on the project in respect of its local interventions progressively increases with the design process.

NA NA Outcome 4: Lessons learned and best practices from

promoting effective management of pilot GIAHS are

widely disseminated to support expansion and upscaling

of the GIAHS in other areas/countries and creation of the

GIAHS network (Global, National, Local) In order to

facilitate further replication and expansion of the GIAHS

concept, this outcome will focus on documenting lessons

learned and best practices, and enabling exchange of

experience.

Commentary

Outcome 4 of the FSP does not have its “counterparts “under the formulation of the PDF-A and PDF-B project framework description. Apart from project analytical capacity this outcome relies on the creation of “learning” networks, as a vehicle or tool. Such networks are to be established at all three project tiers, which pre-supposes a solid project presence / influence at all these levels. Further the outcome is clear about the (wide) dissemination of project experience for GIAHS up-scaling into new countries. It will do so during the project lifetime, and it is an implicit assumption for such up-scaling that the preceding 3 outcomes have been realized. Once more, this indicates a growing expectation regarding project achievements.

Page 132: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

51

Annex 9: Logical Framework Analysis

Project Strategy

Objectively verifiable indicators

Goal To “protect and encourage customary use of biological resources in accordance with traditional cultural practices that are compatible with conservation or sustainable use requirements” [cf. CBD: Article10(c)], specifically within agricultural systems

This is a legitimate formulation of the project goal, the goal being larger than the project itself and the project being design to contribute to the longer term achievement of this goal. Moreover the goal is anchored in the CBD providing it with policy a legal cover. Since the terminology refers to encouragement it can logically not be considered as very specific, which again is acceptable as it concerns a goal. Lack of specificity as this level doe however raise the anticipation about how the project contribution shall be formulated and whether a higher level of precision may be expected at the level of project outputs and activities.

Indicator Baseline Target Sources of verification

Assumptions and Risks

Project objective

To promote conservation and adaptive management of globally significant agricultural biodiversity harbored in globally

Establishment of a global enabling environment for GIAHS

CBD Articles 8(j) and 10(c), and the Cultural Landscape Category of World Heritage Convention, provide starting points for an international policy framework, implementation system and funding mechanism for GIAHS

Accepted international policy formulated to recognise and promote the conservation and adaptive management of GIAHS and designate sites.

Creation of an internationally recognised GIAHS interim Secretariat with a statutory mandate by the end of the project that will encourage formal recognition and designation of GIAHS worldwide.

Establishment of a sustainable funding mechanism for the long term program

Documentation from competent international bodies supporting GIAHS designation (CBD, UNESCO, FAO, IUCN, WWF etc).

Existence of GIAHS Secretariat

Audited accounts and reports from financial mechanism

GIAHS is based on a holistic concept of agricultural systems; this carries the risk that its application will be given different interpretations in each of the pilot systems.

Pilot countries are willing to designate, support and promote GIAHS concept in

Page 133: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

52

important agricultural heritage systems or GIAHS15.

The above cannot be considered to be an indicator. If formulate differently it might qualify as an output, but in the current form mostly resembles an activity albeit of a highly non-specific nature.

While the above sketches the international legal environment under which GIAHS might come to fall, it is not the same as a baseline. A baseline is supposed to describe the existing situation at a given point in time. In this case presumably the governing and legal status for GIAHS as the global level. This can be inferred from the targets specified in the next column above

The three specified targets above are closely interlinked: An interim GIAHS secretariat with a statutory mandate acts as a clearinghouse for formal recognition and designation of GIAHS. It is clear that the statutory character of the secretariat will require a formal status upgrade of GIAHS within the programme and governing environment of FAO

To act in this capacity the role of this (semi) permanent entity needs to be covered under an international policy. It is left open here how such a policy shall be governed, although the previous baseline section suggests that this may be through affiliation with existing conventions.

The long term sustainability of GIAHS does depend in part on sustainable funding of the secretariat structure, the recognition and designation process as well as any global support extended to individual GIAHS. This explains the requirement for a funding mechanism for the long term programme.

The sources of verification mentioned lack in precision or in case of “existence of secretariat” is not a source a verification but rather an indicator.

With the same effort precise sources such as FAO Council minutes could have been mentioned as a source of substantiated proof.

their territories

Collaboration among GIAHS secretariat, governments and other stakeholders is achieved in order to create an international policy environment conducive for GIAHS

15 GIAHS are defined as remarkable land use systems and landscapes which are rich in globally significant biological diversity evolving from the co-adaptation of a community

with its environment and its needs and aspirations for sustainable development

Page 134: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

53

Establishment of national enabling environments for GIAHS

Ministries responsible for Environment, Agriculture, Forestry, Fisheries, Water and Rural Development are involved in various aspects of implementation of CBD and NBSAPs with respect to agricultural biodiversity

Project countries have all set up national contact points to promote the GIAHS concept and develop best practice for their designation and management

Project countries have adopted GIAHS considerations in key policies and legislation

Existence of national bodies and meeting reports

Government publications

National Reports to CBD Secretariat with respect to implementation of Article 10(c)

The vagueness of the above indicator does in fact disqualify it as such.

The baseline cannot be referred to as no status is described here but rather a general concept of involvement” which itself cannot be captured or let alone quantified.

The target for national level GIAHS governance and promotion is rather non-specific as it fails to deal with the institutional setting or describes the degree of formalization of national GIAHS secretariats.

In absence of a clear definition of what are precisely GIAHS considerations, it may be difficult to establish whether this target has actually been reached. Moreover a better definition of expectations regarding policy and legislative adoption would be welcome.

Existence of national bodies is not a source of verification.

Improvement of GIAHS conservation and adaptive management

Project pilot sites face three key barriers for their conservation and sustainable management at present: (i) weak local institutions and stakeholder networks; (ii) acquiring new knowledge, methodologies and tools; and (iii) access to markets.

The key barriers to conservation and management in pilot sites are significantly reduced or removed.

GIAHS operate without external financial assistance and key indicators for extent and biodiversity are achieved

Reports from M&E surveys

Case history reports from Outcome 3

Scientific publications from Outcome 4

Page 135: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

54

As with the two preceding sections under this column, the above indicator formulation fails to reach up to what is expected from an indicator, namely to be verifiable, measurable and quantifiable

This is not a baseline, but a status description which mainly serves to juxtapose against the targets in the next column.

Literally interpreted the project intervention sets the target of significantly transforming or upgrading of weak local (read national) institutions, without however specifying which are exactly the institutions concerned. Apart from being imprecise this target is very ambitious for a project to achieve.

The target can be translated in complete financial system sustainability as a result of the project. Apart from a lack of economic understanding of what underpins GIAHS it should be deemed unrealistic for a project to achieve such sustainability as the result of its interventions.

The use of sources of verification at objective level which are in fact derived from the verification sources of underlying outcomes, indicates unwarranted replication in the LF structure.

Apart from this it is interesting that while reference is made to case history and scientific reports under respectively outcome 3 and 4 no such terminology can be found under the sections of the LF pertaining to these outcomes.

Tracking tool BD 2 The 7 project pilot sites cover 120,000 ha of land having significant agricultural biodiversity value

40 other potential GIAHS identified in accordance with internationally accepted criteria Hectares of land managed in accordance with GIAHS definition and criteria: 120,000 ha or more.

Reports from M&E surveys

National Reports to CBD Secretariat with respect to implementation of Article 10(c)

Reports from GIAHS interim secretariat

Page 136: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

55

A tracking tool by itself cannot be an indicator

It is difficult to see how either the number of sites, their combined area coverage (which upon further verification is not accurate) or unspecified levels of biodiversity can act as a baseline against which to measure project progress

There is some odd arithmetic involved in this target setting. The target proposed 33 additional GIAHS sites, however the combined acreage covered by these seems identical to the figure under the baseline.

Reports can of course act as verification sources. However it is not clear what is going to be exactly verified using the here mentioned reports.

Wording like “promoting” leaves a lot of space for (individual) interpretation and should preferably be avoided when used in the context of a logical framework. The purpose of the LF is after all to convey a sense of logic, and to specify concrete achievements which can be independently monitored. Thus the current project objective as outlined does lack in clear definition and precision.

Outcome 1:

An internationally

Number of GIAHS systems receiving international recognition

Nil At least 15 recognised Project reports International policy

processes are influenced by many

Page 137: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

56

accepted system for recognition of GIAHS is in place (Global)

Without exception indicators provided for this outcome are by proxy. While indicators are supposed to substantiate and “internationally accepted system”, the potential shape of this system is at no point alluded to, and therefore left open for (multiple) interpretation. The numeric aspect of the GIAHS systems recognition can at best be seen as a arbitrary figure as it signifies very little about the scope and importance of the underlying content.

factors, and are generally very lengthy. Accordingly, not all international organisations may be able to provide the desired endorsements for GIAHS within the project period. It is assumed, however this will be achieved through the work programme and joint efforts of CBD, UNESCO and FAO.

Official statements from FAO, UNESCO WHC, CBD CoP, CCD, IUCN endorsing the GIAHS concept, definition and identification criteria

Nil By project end all identified institutions issue resolutions / statements supporting the GIAHS concept

Project reports

Copy of the statements

Page 138: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

57

Official statements of endorsement by various UN and non UN bodies, which are moreover targeted at the concept, definition and means of identification are not equal to the institution of a statutory body recognized and referred to by the same.

By itself this target seems to have a forced and unrealistic character. Recognition of GIAHS and its incorporation in existing international legal instruments is a process based affair. It is therefore not correct to aim for end of project resolutions / statements of conceptual support. What is in fact desirable is that international bodies formally adopt amendments, which provide for the recognition of the GIAHS concept and which equally allow for the international legal backing and continuity of it beyond the life of the project.

Establishment of a sustainable financing mechanism and institutional support for consolidating and expanding the GIAHS approach as a long-term open-ended program

US$ 18,000,000 [TBC] Sustainable finance mechanism in place Written commitments by Donors

Apart from the phasing being that of an activity, rather than an indicator, this touches upon an important aspect of (funding) independence and a sustained GIAHS program approach

Quite coincidental the baseline equals the cumulative project funding which was available when the GEF project took off. Subsequently there is no reference to any other amounts or the expected cost of running a permanent program and an accompanying secretariat. The quoted baseline does therefore not really serve a concrete purpose

As mentioned (2nd column of this row) a sustainable financing mechanism supporting GIAHS as a global initiative is crucial to the sustenance of the initiative. Some more detail about the foreseen mechanism and on how to arrive there could have been expected. FAO as the main proponent might have been expected to possess sufficient experience with the creation and financing of statutory bodies, to allow it to provide such (convincing detail) For unexplained reasons the target is formulated as if it is a project output.

If the commitment is related to a sustainable financing mechanism, it would be reasonable to speak about multi-annual funding pledges from donors and eventually international organizations having a stake in GIAHS. The current treatment is too light.

This outcome deals with the global aspect of GIAHS recognition. It is appropriate that this outcome is listed as the first of four. Clearly international recognition providing the essential cohesiveness and defined common interest of an originally FAO formulated and launched concept is key. The point of departure for concept support is a series of projects with a limited funding window, which will themselves be unable to sustain global international support. Participating nations will primarily attach importance to the guaranteed permanency of an international label, which lends justification and visibility to their own national initiatives in support of GIAHS.

Page 139: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

58

Outcome 2: The conservation and adaptive management of globally significant agricultural biodiversity harboured in GIAHS is mainstreamed in sectoral and inter-sectoral plans and policies in pilot countries (National)

Amendments to key sectoral and inter-sectoral policies and plans

Identified policies and plans do not make explicit reference to GIAHS

By project end amendments have been approved to following: Chiloé: NBSAP Protected Area Legislation China: NBSAP Protected Area Legislation Qintiang Provincial Tourism Policy and Plan Peru: NBSAP Protected Area Legislation Land tenure Legislation Philippines: NBSAP Protected Area Legislation Algeria: NBSAP Protected Area Legislation Morocco: NBSAP Protected Area Legislation Tunisia: NBSAP Protected Area Legislation

National govt. official publications

Government changes in pilot countries might delay the adoption of policies. However it is expected that new government fulfil the prior commitments of previous governments.

Page 140: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

59

While the project correctly recognized the multi-disciplinary and inter- sectorial approach required to effectively deal with GIAHS, it is a missed opportunity in the logical framework (LF) that the multi-sectorial nature is not the least specified as part of the same LF. Somehow this also stands in stark contrast with the ample amount of detail provided later on in the LF when it deal with local GIAHS aspects

This baseline is of a somewhat gratuity nature. It is the project ambition to mainstream GIAHS at the national level, and it is therefore quite logical that at the time of the project start, such mainstreaming had not yet been achieved.

For most of these targets their inter-sectorial nature can be questioned. In majority the documents and pieces of legislation here proposed for amendment fall under the jurisdiction of the Ministry of Environment. This leaves aside amendments as well as fundamental support of key ministries and associated sectors such as Agriculture, Tourism, Economic Affairs, Trade and Finance. In general it must be concluded that the LF targets do not correspond to the expressed ambition of the outcome. Apart from the limited inter sectorial nature mentioned above; a major short coming is that there are practically no indications (apart from the desire to see explicit reference) what should be the actual nature of the amendments in national policy, strategy, plans as well as legislation.

Presumably the LF refers here to the National Government Gazettes of the various participating countries

Level of government budgetary support to GIAHS

No government support explicitly to the concept of GIAHS

At least 1-2 government staff per pilot country are dedicated and qualified to champion the concept of GIAHS

National govt. official publications

Page 141: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

60

Description of a level is by itself a neutral statement. Therefore it is not recommended to use level as an indicator, but rather specify a direction of growth or decline. In this case it is obvious desired that as a consequence of the project National Government buy in for GIAHS increases and that this is honoured by a growing contribution to GIAHS conservation and adaptive management by national Governments

This baseline merits the same comments as the preceding one. It is difficult to expect that any form of formalized Government support is extended to a concept which is still in the process of seeking definition and a formalized national status.

While numbers of Government staff are quantifiable, their input, institutional affiliation, qualifications and role as GIAHS champion is neither quantifiable nor possible to describe in qualitative terms. This makes the target setting of a somewhat hollow nature.

It is not clear what kind of official Government publications is referred to in this case. Should one expect an payroll statement with a clear indication that it concerns a GIAHS related position?

This outcome proposes to cover all that is required in terms of policy and planning amendments in relation to the formal recognition and adoption of GIAHS in the national governance framework. Although the outcome itself is not explicit about this the target setting does indicate that also various pieces of legislation are included in this foreseen process. Aside from such legislative amendments, which are relatively uncertain to achieve as the result of project leverage, and should therefore not have been made a target of the project, the recurring adaptations are those of the National Biodiversity Strategies and Action Plans (NBSAPs). It is clear that amendments to NBSAP are not merely part of the national policy and legal adaptation process, that that the main driver here consists of the national reporting obligations held vis a vis the Convention on Biological Diversity. Such reporting is an instrument in not the end of a national GIAHS mainstreaming process.

Outcome 3: Globally significant agricultural biodiversity in pilot GIAHS is being managed

No further decline in land conversion and land abandonment pressures on traditional farms

Chiloé: 10,616 ha China: 461 ha Algeria: 500 ha Morocco: 500 ha Tunisia: 700 ha Peru: 30,798 ha Philippines: 68,416 ha

Chiloé: 10,616 ha China: 461 ha Algeria: 500 ha Morocco: 500 ha Tunisia: 700 ha Peru: 30,798 ha Philippines: 68,416 ha

Annual field surveys using rapid assessment of land cover change methods

Macro-economic drivers and natural hazards, socio-economic and environmental changes (e.g. climate change) may disrupt progress

Page 142: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

61

effectively by indigenous and other traditional communities (Local)

This indicator is not comprehensible the way it is phrased. Presumably the ambition is to call a halt to decline of lands under GIAHS as the result of farm conversions and abandonment of land by farmers. If this is the case one would expect the indicator to refer to rates of decline of concerned systems, rather than giving to areas of GIAHS specific land cover of specific national pilot systems. The current formulation of either baseline or target is made rather meaningless without such detail.

Apart from what is stated in the previous column of this row, what is striking is the uneveness of both the baseline and targets in terms of area coverage. What appears to have taken place here is that in some cases such a the national cases of Chile, Peru and Phillipines either complete systems, rather than pilots have been taken into account, while in other cases the project has settled for specific geographically limited pilots, forming part of a larger system. This is a worrying design mistake as it harbours the risk of introducing unequal treatment of GIAHS in terms of project funding allocation, scientific, managerial and operational attention.

It would have been easy to avoid unevenness in treatment as a consequence of the design specifics contained in this LF. This would have required systematic treatment of GIAHS, not as unique and specific pilots, but rather as samples of larger agro-ecosystems in which these are contained. Thus the geographical limitations of Oasis like those inTunisia and Algeria would have played much less of a role, from the start the emphasis would have been on the importance and future role of the traditional Oasis systems in the Magreb region. A further debate would still be required to define the role of pilot sites in relation to the wider Agro-ecosystems.

As the element of change to be measured is unclear, it becomes equally difficult to define the appropriate method to measure change in land cover.

in some pilot GIAHS. Local communities and key stakeholders will engage in the pilot management projects for GIAHS

Decline in land conversion pressure on surrounding habitats

Baseline to be quantified per country in the first year

Habitat networks surrounding traditional farms remain stable or increase compared to baseline levels

Annual field surveys using rapid assessment of land cover change methods

Page 143: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

62

This indicator is not clear or measurable. It is an open question what should be defined as “surrounding habitats”

Please refer to the previous column of this row

If this refers to the larger agro-eco system in which the pilot is located, it is most doubtful to assume that the pilot will generate such an effect as effectively turning the trend of the larger environment. Especially if the scale of this environment is left undefined.

Larger scale land cover change assessments are likely to fall outside the scope of the project and may absorb a disproportionate amount of resources, without a possibility to attribute effect to the project or parallel initiatives.

Level of understanding and commitment of communities to GIAHS in the pilot sites

90% of farmers are estimated to observe management practices supportive of GIAHS criteria

No decline in percentage Project reports

There are two related issues with this indicator: How can one measure community understanding, apart from the fact that the community is supposed to be the main repository of knowledge of the GIAHS? Commitment is not a measurable “commodity” at best it can be qualitatively judged. Even if people are committed to maintain the system there may be overriding economic forces that prevent them from doing so.

This baseline is project centric rather than respectful of the farmers making up the local community. GIAHS criteria are formulated by an outside entity and may not be broadly defined among the farming community. So asking farmers to “observe” to such criteria appears to fall outside the framework of their day to day reality. Depending on the interpretation of these criteria by national authorities there is moreover a real risk that the space for adaptive management is curtailed in favour of a museum approach.

The likelihood that understanding and commitment of farmers form the principal drivers determining stabilization of GIAHS, totally overlooks the economic, (including market and labour) financial, institutional reality. These forces would appear to be the actual determining factors for positive or negative change in relation to GIAHS.

What is proposed to be measured here is the local impact of project and associated initiatives. It would be recommended that such impact measurement is carried out by independent monitoring rather than by the project itself. This will add important credibility to the collected data.

Page 144: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

63

Number of traditional crops and varieties being cultivated

Chile: 200 varieties of Solanum

tuberosum 1 variety of Ajo chilote China: 20 native varieties of rice 6 native breeds of carp Algeria: 100 date varieties Tunisia 50 date varieties Morocco: 80 date varieties Peru: Baseline Caritamaya: Potatoes (28 varieties). Bitter potatoes (13 var.) Quinoa (43 var.), Kañiwa (8 var.), Oca, Olluco, Llamas, Alpacas (all 24 colors, 3 major breeds) Baseline Microcuenca de San José: Potatoes (80 var.), Mashua (14 var.), Olluco (18 var.), Kañiwa (12 var.) Oca (20 var.) Llamas, Alpacas Baseline Cuenca de Lares: Patatoes (177 var.), Oca (20 var.), Olluco (11 var.), Mashua (17 var.), Maiz (23), Quinoa, Kañiwa, Lupins, Llamas, Alpcas, wild relatives Baseline Micro de Carmen: patatoes (105 var.), Oca (25 var.) Olluco (14 var.), Mashua (20 var.), Maiz (34), Quinoa, Kañiwa, Lupins, Llamas, Alpcas, wild relatives Philippines: 4 endemic varieties of rice 264 indig tree species 10 varieties of climbing rattan 45 medicinal plant species 20 plant species used as ethnopesticides

By project end, numbers are stable or increase over baseline

Annual field surveys GIAHS is based on a holistic concept of agricultural systems; this carries the risk that its application will be given different interpretations in each of the pilot systems. Pilot countries are willing to designate, support and promote GIAHS concept in their territories Collaboration among GIAHS secretariat, governments and other stakeholders is achieved in order to create an international policy environment conducive for GIAHS

Page 145: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

64

The diversity of crops under cultivation is but one element of GIAHS. This indicator does not appear to do sufficient justice to the cultural, technical (agricultural management aspects)

The country figures for crop diversity seem both arbitrary and overly detailed in comparison with other baseline data of this LF. Measuring against this baseline will moreover not provide convincing proof about system viability and sustainability. This is certainly the case over the limited duration of the project

See comments of previous column of this row. It appears that the investment in annual field surveys of sufficient quality and detail is neither advisable nor desirable (from the point of project resource allocation). The focus on such surveys will shift the intervention focus away from communities and towards institutional entities. Meanwhile the relevance of the collected data will be doubtful

This outcome fails to specify the scope of project influence. In other words it is left unclear whether the effect merely concerns pilot locations or larger agro-eco systems in which the pilots are located. Without indicating the resource availability (of the project) or analyse the external drivers influencing GIAHS it is not plausible to assume that the project can have a real or measurable impact on the achievement of this outcome. This makes the outcome over ambitious. The linkage between the local level outcome 3 and the institutional, policy and legal level outcome 2 at National level is moreover unclear. One would at least expect that both outcomes are mutually re-enforcing, and that such a relationship is made (more) explicit in the LF, apart from some reference in the Assumption and risk Column.

Outcome 4: Lessons learned and best practices from promoting effective management of pilot GIAHS are widely disseminated to support expansion of the GIAHS network (Global)

Expressions of interest from other GIAHS from around the world to apply the project approach, along with commitments to provide co-financing

Nil At least 5 proposals by end of year 4 and 10 proposals by end of project

Project reports Project outcomes are achieved and result in demand from other areas

Expressions of interest is not an indicator. The degree of change in interest is

At the start of the project there obviously is an element of interest in the subject, otherwise the original candidate systems could not have been selected as the basis for the project.

It would be good to indicate that the new proposals for GIAHS status accession are incremental to those defined at the start of the project.

And not merely project reports, but importantly also evidence provided by national governments in terms of their formal request to launch additional GIAHS on their territory.

Page 146: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

65

Interest from academic and research institutes in analysing and further study of experience in pilot sites

Nil At least 20 proposals/ scientific publications by project end

Project reports

Again interest as such is not quantifiable and cannot be used as an indicator. An alternative would be the # of expressions of interest by academic institutions or more concretely the # of submitted research proposals related to (each) GIAHS

See comment above. This target does put a figure on the undefined indicator. The number is most likely originating from either a proposal or scientific publication for each expected individual GIAHS by project end. What is not understood about this target is the fact that proposals seems to be interchangeable

Publication references might be a more appropriate MoV

Usage of electronic forum and database by interested stakeholders

Measure usage of website in year 1

Increase in usage by at least 100% Web-site counter

This could be better specified in terms of topic lines and might even contain a qualitative dimension to be assessed, such as the contribution of electronic media (website) in terms of concrete and tangible inputs in GIAHS concept development, publication comments and review.

The usage of the electronic forum is simplified to the extreme, as what is here suggested merely covers the number of annual hits. It is moreover not specified whether these should be unique hits or whether hits are geographically distinguished.

See comments in the previous column of this row.

See comments in the previous column of this row.

By itself the outcome is plausible and could be defined as the core function of a (permanent) GIAHS secretariat. The problem with this outcome is the overall superficiality of the indicators applied to it which by no means cover the ambition of the outcome itself.

Page 147: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

66

Annex 10: Extended MTE Findings

Project formulation

1. Finding 1: The project formulation process16 provides insight into how the project design evolved over time. At the outset,17 the project was seen as a catalytic means to enhance global and national recognition of GIAHS. Central to this objective was mobilization of global support for GIAHS conservation and adaptive management. 2. As the process role of the project was considered catalytic, no explicit project operational role at site level was foreseen18. Such a narrowly defined project concept aligns well with the typical GEF (project) role of creating incremental (global) benefit in relation to an existing baseline (project). 3. Finding 2 : Later during the project formulation process19, the project design broadened its set of objectives (outcomes) to include a much more hands-on involvement at GIAHS site level. The expansion of the project mandate was introduced while maintaining the project goal to act as a catalytic agent. This evolution in perceived project roles happened as the result of a confluence of several ideas. Sources of influence include the immediate FAO GIAHS project management20

, were asserted by project hired consultants21, UNDP22 as well as the GEFSEC.23 4. Resultant, project formulation included substantial implementation involvement at site (local) level24. In fact, project funding allocation for local interventions grew to such an extent that they became the dominant component of the FSP. The FSP formulation made the project an active protagonist of events on the ground. 25 From this, it can be inferred that the project’s role went well beyond that of a catalyst. 5. Unavoidably, this had to lead to considerable conceptual tension between: a) The still remaining catalytic ambitions of the project, consolidated under the national and global project outcomes and, b) The project self-imposed role as a distant agent for (local) organizational, investment and (perceived) problem solving support.

16 The whole sequence from PDF-A to, PDF-B and finally the FSP as presented in the previous chapter. 17 PDF-A 18 At least during the early conceptual and formulation phase. 19 PDF-B and FSP 20 A desire to show a hands on presence, be close to the farming community and generate operational examples,

with extrapolation potential. 21 In particular Miguel Altieri, considered as the “godfather” of agro-ecology, whom gained increasing

involvement outside the academic sphere resulting in strong advocacy for farmer / community driven bottom-up approaches, which are not necessarily in line with institutional, organizational and governance realities affecting GIAHS

22 Until the end of the PDF-B UNDP acted as the intermediary agency and was actively involved in completion of project review matrices.

23 GIAHS conceptual defintions methodological guidelines produced as part of the PDF-B phase consultancies were specifically adamant that the project should be dealing directly with communities at the site level, and stressed the introduction of participatory instruments for this purpose.

24 Participatory project intervention planning of an organizational and physical implementation nature at each of the of the selected GIAHS sites.

25 By extension this also meant the need for a more elevated project operational presence at the national level.

Page 148: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

67

6. Finding 3: A striking formulation feature is that design warrants the project to link individual GIAHS interventions with national level policy and legal framework initiatives and global (institutional) recognition. The principal chosen vehicle to achieve this is knowledge and experience exchange26. As mentioned, such local knowledge and experience27 resulting from project interventions will influence the national and global setting28. 7. Vice versa, contribution to growing national and global recognition may translate into measures conducive to system maintenance and effective use.29 This type of project formulation leaves implementation open to trial and error based interventions, which can take place either simultaneously or in relative isolation. 8. Finding 4: The logical premise by which FAO project management defends the design discussed here is rooted in what it sees as adaptive development30. The approach is also seen as a means to ensure that global attention towards designated GIAHS is kept “alive”. Further it is felt that project presence and engagement along the full spectrum, is needed to emphasize GIAHS’ embedded and “living” character31. 9. The mission equally observes a need to accord the GIAHS site level32 a central place in project formulation and link system information from this level to National and Global outcomes. Beyond this similarity of observation, the view of the evaluation mission differs from the project design. The mission is far from convinced that dropping the original purely catalytic mandate is the appropriate way forward.33 This applies in particular to the formulation decision to have the project become directly involved in community (local) participatory planning and project implementation activities. 10. Finding 5: There exists ample evidence from past global and regional executed FAO projects that experience from small localized pilots is not necessarily suited for extrapolation to national, regional or global levels as relevant experience and lessons learned34. Traditionally, global projects with field-based interventions spread over several countries face considerable coordination and logistical obstacles. This can easily lead to a situation where maintaining internal project balance between global, national, and local outcomes is no longer evident. 11. Finding 6: Project formulation logic based on a premise of closely interlinked project tiers of a mutually influencing nature can become easily defeated. For the current project, de-

26 Project Outcome 4 27 Presumably of a paradigm shifting nature 28 Inform national policy and legal framework requirements and provide designation, support requirement and

technological information of global relevance. 29 This translates in a typical combined bottom-up , top-down scenario 30 Adaptive management being an inbuilt and age old characteristic of GIAHS as well as a development

strategy advocated by the project, which is seen as the best manner to ensure that survival of GIAHS into the future is based on a continuous process of conceptualization, action planning, implementation, monitoring information consolidation and analysis followed by a fresh cycle.

31 As it allows the project / programme to support continued generation of products and service. This while the system adapts itself to changing national and global influences and retains its fundamental characteristics in bio-diversity, cultural and technological terms.

32 As the core subject of the global GIAHS initiative 33 Directly or indirectly through active project implementation of funding support. 34 Outcome 4

Page 149: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

68

synchronized outcomes mean that the holistic project concept of two-directional information flow, system analysis, informed (political) decision making and global recognition is undercut. As a result of increasing de-linkage within the project (conceptual) framework, the overall relevance of project pilot interventions35 become increasingly questionable. 12. Finding 7: The mission has established, beyond reasonable limits of doubt, that the project formulation scenario, as described and the way in which it was adopted, is the result of a particular set of circumstances. These count the influence of delays in project formulation completion36, and the change of GEF CEO at the time37 the project go-ahead had to be granted by CEO decision. Circumstances point towards an aligned GEF and FAO interest favouring a local “dominated” project scenario. The (small scale and geographically spread) piloting approach with its associated assumptions closely conform to standard FAO project approaches to institutional capacity building and generation of policy leverage.38

13. As mentioned, as far as GEF can discern this global (project) initiative was launched at the time of a new incoming GEF CEO. It is understood from interviews of personnel39 closely associated with the formulation phase that the ‘project endorsement climate’ was generally adverse to broad-based global GEF initiatives. The focus under new GEF leadership appears to have shifted to generating national and local impacts. The project characteristic of introducing a hefty quantity of local implementation initiatives may thus have been sufficient to sway the CEO decision in favour of this particular global initiative. 14. Finding 8: The project framework contains an ambitious formulation for outcome and outputs40 dealing with national policy and legal frameworks. The mission agrees with FAO’s main project proponent about the importance of sufficiently elevated project targets to act as a driver of the global and national GIAHS recognition process. Ultimately, a principal achievement41 for the GEF-funded and other associated GIAHS projects is to put the subject and its importance on the global ‘map’. 15. A project42, as the current one, by definition is faced with serious limitations in time and resources to achieve policy and legal oriented outcomes. If only for this reason, the evaluation wants to caution against target formulation, which per definition fall outside the scope of a project. Asserting meaningful leverage on national policies and legal frameworks is considered to fall within this category. Its merely requires a realistic appreciation of sovereign law-making powers and national independence over governance processes to understand that even a project with far more resources has a hard time steering such outcomes. 16. This same issue was criticised at the time of the GEF CEO review, but did not lead to adjustments, such as e.g. a more humble43 formulation of project targets aiming to affect a

35 Outcome 3 36 GEF imposed 37 2007-2008 38 The MTE states this on the basis of a considerable number of years of past association with FAO project and

programmes, both in a managment and evaluative capacity. 39 Including past and current GEF staff. 40 Outcome 2 41 Outcome 1 42 Not a programme and not necessarily nationally owned from the start. 43 And therefore more realistic project target

Page 150: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

69

wide range of GIAHS enabling policy and legal conditions, mostly well beyond the mandate of national counterpart agencies. 17. Finding 9: The eavlauation recognizes there is a certain value in sharing national experiences at the international level in order to broaden the horizon of researchers and practitioners44. However regarding communication of lessons learned,45 the MTE mission questions the premise that nationally and locally generated experience is in most cases of broad universal46 transferability. 18. Actual benefit from local lessons learned is quite different from distilling institutional and management solutions pertaining to GIAHS, and attempting the transfer of these into the international arena. It is, of course, possible to categorize and analyse various intervention models47 and subsequently draw on certain commonalities and values for global information exchange48. 19. However, the majority of actual site management experiences, especially those which may be considered of an actual sustainable character, are per definition embedded in national governance systems49. 20. Local and national systems have their own particular historical and economic character. The unique aspects of institutional and governance cultures50 must be grasped in full detail to understand their impact. Only in this manner is it possible to predict the possible adoption of transfer into similar environments. The MTE does not observe evidence of analytical capacity inbuilt in the project51 to interpret and deal with such crucial parameters. This can only be seen as an oversight in the project formulation / design. 21. Finding 10 (Consolidation): The mission takes into account the consideration of a long and difficult project gestation process and the initial setbacks this resulted in for the launch of the GIAHS initiative. From this perspective, the evaluation the relevance of the project initiative is rated as High. The formulation process has, however, subsequently met with factors which have led to a Reduced effectiveness of the project design. A strong and legitimate desire to keep the initiative alive has led the main project proponent to over-protect both the initiative and its design process. 22. Rather than seeking strength52 in adopting diversity of opinions and views, this has led to selection of collaborators and fundamental design choices pointing towards relative institutional isolation of the Initiative. This has resulted in a self-limiting impact as a direct consequence of the formulation / design. This specifically relates to inclusion of over-

44 The requirement of community involvement in planning and implementation of GIAHS conservation and

adaptive management is one example, approaches to GIAHS research is another. 45 Outcome 4 46 Global 47 State Management, Private Sector, Civil Society Dominated. 48 The requirement of community involvement in planning and implementation of GIAHS conservation and

adaptive management is one example, approaches to GIAHS research is another. 49 And therefore mostly of relevance at this level. 50 Including level of decentralization 51 Of either a centralized or de-centralized nature 52 During the formulation process (and beyond)

Page 151: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

70

ambitious and unrealistic project targets53, insufficient recognition of local, national and global stakeholder dynamics54, and limited specificity in terms of designing mechanisms and tools to ensure timely, logical, effective and efficient information exchange among project tiers.

Selection and Choice of initially designated pilot sites.

23. Finding 11 : GIAHS workshop proceedings, steering committee minutes and follow-up interviews do not provide a conclusive account of how detailed GIAHS selection under the GEF project took place. Relative vagueness surrounding the applied selection criteria should be understood in the proper context. Neither the GIAHS concept, nor its selection criteria, had fully crystalized at the time of project preparation. Studies to define and describe potential candidate systems were on-going and lobbying efforts were underway using various FAO institutional and personal pathways55.

24. The state of conceptual flux may have contributed to selection of pilot systems which could be viewed as somewhat opportunistic56. The mission is aware that this observation is amplified by the benefit of hindsight. Several elements in the selection process seem to be under-developed and remain so to date. E.g. there is no evidence that the project has taken into account such issues as the progress and rate of system decline including historical bio-diversity loss and existing system salvage options available. There is also no systematic analysis of past conservation efforts and formal institutional involvement with sites prior to selection. 25. Clearly, the desire to accelerate implementation and make the GIAHS concept concrete must have played a significant role in the desire to timely settle on a on a number of pilot GIAHS57. Still, investing in selection analysis would, as mentioned, have gone a long way in facilitating global GIAHS management to better prioritize GIAHS engagement from a feasibility angle. The mission is sure this would have led to different and more resource-efficient selection result. 26. Finding: 12: Efforts to develop GIAHS selection criteria have nevertheless been progressing over time. Early on, efforts focused on capturing GIAHS’ composite58 nature through a broad set of selection criteria59. During the early stages of GIAHS conceptual development, selection criteria appear to have been applied in a seemingly contradictory manner60. This has now been eliminated however. GIAHS purposely does not opt for narrow selection definitions, although these would make it easier to pigeonhole the concept. This

53 In relation to secured project resources for local, national and global outcomes. 54 Despite sufficient forewarning about the fundamental importance of this aspect during the extended

formulation phase. 55 Connections such as UNESCO, UNU as well as academic affiliations between FAO personnel and

universities and research institutes. 56 The reason to use this phrasing is that with rigorous systematic pilot system selection, including stricter

comparative analysis among potential systems a solid body of documentation the selection process would have been available.

57 Even when the selection criteria were insufficiently developed to come to fully founded choices. 58 Composite in the sense that it focusses on the broad range of characteristics defining the agricultural system,

which as a result also leads to a higher level of complexity in dealing with selection criteria. 59 Encompassing, historical, physical, technical, cultural, institutional and economic dimensions 60 Selection criteria on the one hand ask for historical ‘prove’ of GIAHS resilience in the environment of their

existence, while threat to the system survival forms another criterion.

Page 152: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

71

more holistic and complex process to describe, classify and give practical meaning to GIAHS must be lauded. 27. However, a flip side exists to the composite and complex nature of GIAHS selection. Experience61 suggest that increased complexity introduced through a broad and variable range of selection criteria is practically more difficult to manage and can become subject to multiple interpretations62. Broad based GIAHS definitions also make it more complex to set baselines against. Baselines are essential to facilitate tracking of the conservation and adaptive management impact. 28. All this demands additional effort of quality control during the selection process and definition of individual GIAHS eligibility. Several interviewees have suggested the need for systematic involvement of scientific committees in the selection, and the MTE mission underwrites this opinion. 29. A simplified set of selection criteria would surely make it easier to place GIAHS under existing conventions and treaties. However, seeking “easy” alignment with a specific existing classification would not do justice to the contemporary, complex and dynamic nature of GIAHS. Rather than equating GIAHS with existing international instruments, this leads to the justified view that it must rather be considered as a broad umbrella concept. Specific affiliations and certifications provided through existing mechanisms may take place under it and may be assisted by GIAHS selection and recognition. 30. Finding 13: A key issue requiring attention is the continuing absence of a proportional weighing mechanism for individual criteria. To effectively manage selection complexity, a set of weighed selection criteria is a must for process transparency. Weighing serves the purpose of qualitative comparative analysis between various potential system representative sites/landscapes, which is ultimately required to decide on candidate system eligibility. 31. Finding 14 (Consolidation): The selection of GIAHS has increased in effectiveness and efficiency over time by reducing process complexity. The project, however, still remains affected by a past selection process which was lacking in transparency and which, in some cases, led to the inclusion of sites where sustainability of project efforts are difficult to guarantee. There are strong arguments for working towards a further honed selection process. The role of an established scientific committee, operating independently, should be central to efficient GIAHS selection. 32. Instead of a project / programme engaging pro-actively in the identification of new GIAHS63 this responsibility should be transferred to individual nations. To enhance the sustainability of national and local efforts, it is important that Member States express an active interest and undertake their own independent steps for accession to the GIAHS64. An

61 Past FAO field programme experience as well as other international development initiatives 62 or worse abuse 63 As is currently the case 64 Including preparation of the burden of proof for GIAHS eligibility, long term action planning and support

scenarios. This has fortunately been taking place in a few participating countries, on their own initiative through establishment of National IAHS (NIAHS), but is an institutional undertaking which has not been actively pursued by the project.

Page 153: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

72

emerging global GIAHS programme needs to provide an advisory and soundboard function in such developments (catalytic) but refrain from direct action.

Choice of project national counterparts

33. Finding 15: For the majority of GIAHS pilot countries65, choosing the national counterpart agency has been a simple affair. By virtue of acting as the National GEF and/or Biodiversity Focal Point, mainly environmental ministries / departments took on this role66. The MTE did systematically pose the question why, other than formal GEF / biodiversity linkage, environmental agencies were made the formal counterpart. After all, GIAHS is an essentially agricultural focused concept. 34. Apparently, institutional selection determinants did not necessarily focus on the most appropriate institutional linkage, which would normally include vertical implementation capacity of the counterpart. Instead, most emphasis was placed on the institutional interest to embrace the GIAHS initiative at the time of the project start. The justification provided for this is that Agricultural Ministries, although technically better placed, were on the whole not fully convinced about the project benefits. Subsequently, this is further explained by a focus which oriented itself more towards production than conservation of Agricultural Ministries. 35. Finding 16: Based on interviews, the MTE has concluded that choice of counterpart agencies was to an extent outside the formal boundaries of FAO’s control. Simultaneous negotiation of counterpart arrangements in 6 countries by a global project secretariat posed a constraining factor, as it exceeded both the capacity and level of empowerment of this secretariat. Aside from such a limitation, the MTE has not received any documented evidence about systematic and fundamental weighing of pros and cons of the project’s counterpart arrangements. 36. MTE concerns about institutional choice of counterpart agencies stem from the fact that environmental agencies cannot be considered in a suitable institutional position to promote GIAHS conceptual mainstreaming. Environmental agencies carry a broad responsibility for a wide range of environmental sub-sectors, but are otherwise not at the centre of national economic governance. Aside from environmental matters, such agencies are not in the position to strongly and effectively influence policy, legal, and operational debates, within other sector ministries. 37. Finding 17 (Consolidation): A GIAHS programme, giving full justice to its composite nature should be integrated in the agriculture sector and its surrounding economic environment. National GIAHS linkage to the environmental sector, just because of its agro-biodiversity aspect, only covers a single, albeit, important GIAHS component. By not placing GIAHS institutionally where it belongs, the effectiveness and efficiency of institutional engagement has been jeopardized. This is directly responsible for a lowering of impact, and institutional sustainability of efforts to date. Analysis of the Logical Framework (LF)

65 Under the GEF project. 66 China is the only formal exception as the Ministry of Agriculture has been the national counterpart from the

beginning. In case of Chile the counterpart role was in practice shifted to the MoA, but the MoE still maintains the formal position.

Page 154: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

73

38. Finding 18: It is understood that the Biodiversity financing window of GEF led to a focus on area coverage and numbers of individual varieties. The underpinning philosophy of the project is, however, about helping to create the conditions of GIAHS “system sustainability”, not about a project itself achieving sustainability for prosperity. Therefore, a systematic conceptual and logical programme framework is at stake here. What is required is the right conceptual (and logical) analysis, clear delineation of institutional and programme responsibilities, and identification of pathways, tools and instruments which can concretely contribute to individual GIAHS survival and evolution. 39. Finding 19: The mission stresses its observation that the LF is a typical project product staying well distant from solid programmatic thinking. This leads to a construction of implementation logic which is little about process management. Unfortunately, the LF focusses on indicators for the achievement of, at times, trivial and difficult or impossible to measure (physical) project targets67. Another problem the LF faces is that there are problems of inequality of scope and scale combined with lack of matching project resources.68 40. The generally poor quality of the LF does expose another worrying issue, namely that most of the project progress reporting is held against the LF, its indicators and means of independently verification (MOVI). Progress reporting is affected by a tendency to respond with information which stays free from analyzing and providing answers to the real issues at stake. A similar observation can be made in relation to the answers received by the mission in response to specific evaluation questions posed to project management. 41. Progress reporting turns out to be less than factual and has too often been able to convey a message as if the project, including its higher level institutional dimensions, were on track. The fact that the GEF PMIS does not contain any fresh additions regarding the project69 is worrying. It indicates that follow-up of this, admittedly by GEF standards small initiative, has not been very thorough or of a fundamental questioning nature. 42. Finding 20 (Consolidation): The LF for a complex multi-tier initiative as GIAHS can fulfill a role of considerable importance in the direction the initiative takes.70 The GIAHS initiative demonstrates that the quality of a LF preparation should be prioritized. Obviously, this ought to be different from preparation of a LF as an afterthought or administrative requirement of project formulation. The MTE argues that considerably more investment of time and effort at the start of the project would have led to a LF of concrete value to take regular stock and help direct the GIAHS recognition process. 43. There is evidence from the evaluation that organizational71 Global and National institutional entry points have been missed due to a lack of strategic priority setting, in house72 platform building and lack of delegation. Admittedly, solving all bottlenecks would

67 Especially the targets set for local conservation achievements expressed in Ha. and # of species and varieties

conserved fall in this category. Setting such targets for a project , although supposed to lead to a programme, however itself of limited duration, is not meaningful.

68 A clear example of this is the unevenness in terms of selected geographical scales for Pilot GIAHS, which in some cases may be confined to systems such as a single oasis and in others cover major transects in the Andes encompassing tens of thousands of Ha.

69 Since 2008 70 Be it positively or negatively. 71 FAO corporate 72 FAO

Page 155: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

74

have been impossible and beyond control of the project73 (as alluded to in previous findings). A well governed strategic and logical design process early during the initiative would however have resulted in a more focused and efficient design.

Stakeholders’ participation

44. Finding 21: The PDF-B phase entered in an unfortunate open ended participatory identification and project planning process. Relatively unchecked occurrence of overambitious local activity design was neither matched by assured adequate financial and human resource allocation nor by a coherent intervention strategy. By doing so the project preparation phase has stepped into a classical trap of creating field level commitment’s beyond what could subsequently be followed through. The acute reality of this sank in following a cut back in expected GEF funding and lower than expected counterpart contributions. The “trust damage” which was created has taken a substantial amount of project energies and time to redress, and lingers up to the present time. 45. Finding 22: In all but a few cases,74 the interventions have been provided with typical external project-funded development support. This leads to an impression of a temporary project-based intervention outside the “normal” economic and public sector environment. Such project activities are rarely conducive to a change in (national) institutional vision, strategy responsibility and attitudes. 46. Despite the fact that project funding is directed in a substantial manner towards local initiatives the fragmented distribution of project pilots has enhanced the logistical hurdles and frequently resulted in under-funded,75 and often cash-strapped local operations. Not only have such pilots resulted in limited localized impact and visibility, the opportunity to draw broad lessons from such experiences was also further diminished. 47. Project resource allocation for local interventions is per definition of a one off nature, and usually not integrated in sector budgets, which themselves are far more substantial, but not specific to GIAHS. Budgets have usually been spent through channels parallel to those established for regular government or private sector support. On the ground management and analytical presence to learn from results is in most cases limited or absent, leaving this role to an already overstretched global coordination unit. 48. Finding 23: In line with the project’s desire (and future programme) to maximize national ownership76 , it is the mission’s strong held conviction that the project should engage at the local level where it can have a catalytic effect. In case there exist convincing arguments to rely on the use of typical development assistance interventions, the project may assist in leveraging such assistance through national or alternative budgetary sources.

49. This approach will effectively confirm or refute the national commitment to GIAHS support, while reserving global resources for broader structural GIAHS support issues (for system upgrading and enhancement)

73 Regardless of any amount of logic and systemic thinking applied. 74 The exception being where national Governments have taken a stronger initiative such as China, Chile and to

an extend Peru, the latter more at regional level the former throughout the institutional chain. . 75 In relation to stated ambitions 76 NIAHS

Page 156: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

75

50. Finding 24: The mission has come across examples of pilot site selection for which the selection choice was driven by a high level of bio-diversity and relatively untouched cultural aspects. The selection of such sites was made in the believe that these more outstanding examples of GIAHS preservation are needed to effectively demonstrate the initiative. Unfortunately the same selection resulted in even more fragmented and widespread project locations at a high logistical cost. 51. Emphasis to select “perfect sites” stands in the way of is the actual purpose of developing pilots; The creation / development of meaningful support measures for GIAHS. Support measures must have wider applicability, take note of a broader eco-system situation than what can be found in the most outstanding examples of the GIAHS. This warrants pilot site selection able to represent a broader cross section of the concerned GIAHS. Moreover the selection of unique but widespread and remote sites limits the opportunity of coming up with relevant support measures and disseminating these into the wider system. 52. Finding 25 (Consolidation): Strategizing stakeholder engagement needs to be matched to available resources, 77 not least of which is time. Prioritizing stakeholder interventions (from local to national and vice versa) ensures that quality prevails over quantity, and the chance of long term sustainability over short term visibility. Custodians of GIAHS, namely local farmers / communities will then be the first to recognize and agree that the cause of GIAHS is better served by structural public and private sector recognition and support, rather than by one off project interventions. By disproportionally focusing and engaging at the local level, the project has rapidly lost steam insofar as the more elevated and more strategic stakeholder processes are concerned. The absence of a scientific approach towards building stakeholder platforms and communication arrangements, has resulted in reduced project effectiveness at the institutional level.

Relation between GEF and baseline projects activities, planning identification of incremental activities for global benefits,..

53. Finding 26: As covered under the section findings about the formulation phase, the project fell within the typical GEF mandate. This mandate would assume use of the GEF funding contribution to provide a specific (global) incremental role78. In fact, there is evidence of instances where the project was instrumental in creating and supporting its own new baseline scenarios79. Part of such “fresh” interventions also took place through project geographical expansion into new (GIAHS) sites. 54. The preceding does not infer that all present GIAHS baseline project interventions are created in function of the project presence. The MTE also came across value added interventions, which are less tangible but appropriately linked to baselines and provide incremental benefit. Such work generally includes the enhancement of information availability and exposure of GIAHS originated produce through local / national fairs, seminars, workshops and publications.

77 Including time. 78 In relation to national and local baseline interventions 79 Implementing Partner Organizations at GIAHS sites, although pre-existing themselves, were not always

involved operationally or geographically and it has been the project itself which has been instrumental in taking up the implementation role they fulfill today e.g. Tunisia, Gafsa.

Page 157: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

76

55. The principle instrument for engagement with / delivery of “baseline” supporting project initiatives is that of contracting through Letters of Agreement (LoA). In some cases GEF-provided financial resources were directed towards collaboration arrangements with already locally involved national (academic) actors. This allowed, among other things, for the expansion of pre-existing university and NGO initiatives. There is ground to consider such project activities, although not incremental per se, adhering to the project baseline. Similar work now being supported though GEF had already been ongoing for a substantial period through the same channels. 56. Certain “fresh” site interventions follow a general development “local problem solving”-oriented approach80. Such interventions, which take place outside regular institutional frameworks, raise questions about organizational and technical impact and sustainability. The observation of the MTE is that problem solving may not actually be what is required, but rather that project emphasis should be directed towards the creation of conditions that will ensure the continuity of GIAHS. 57. “Baseline” interventions were also initiated through pro-active mobilization of a local NGO81. Although the latter had not been previously engaged with GIAHS, but had a good record in social mobilization. This process was facilitated through involvement of national (FAO) staff championing GIAHS82. Again, the aspect of incremental global benefit is hard to identify. 58. Finding 27: Project interventions directed at the “higher” institutional tiers conformed far more easily to the pattern of the GEF incremental value description. These efforts include national and global advocacy83, as well as some institutional support and policy-oriented interventions. 59. From a fundamental point of view, there is little doubt that the project should have been skewed more significantly towards the delivery of global incremental value aspects. The absence of such a focus may be largely accounted for by the project’s design flaws. Aside from the apparent absence of urgency to re-direct project efforts and investments, design opened up the opportunity for the project to engage in non-standard84 local implementation scenarios. The disproportionate focus on local interventions absorbed project (management) resources to an extent, which imposed limitations on balancing the project sufficiently in favor of its more strategic, higher level outcomes. 60. Finding 27 (Consolidation): The project formulation and approval circumstances led to a project approach which is atypical for GEF. The project has drawn on legitimate and less

80 Support to (pre) cooperative establishment for sheep cheese production, rural credit and evolving fund

establishment, compost manufacturing enterprises.. None of these interventions were either well conceived financially and economically sound or promising in terms of their sustainability. .

81 Gafsa 82 Tunisia, where same staff had prior strong affiliation with this NGO. 83 The charter of the Oasis, which was created with project assistance by a legal official of the Tunisian

ministry of agriculture is a good example of a broader based advocacy initiative. When interviewed the concerned official however stated that the charter would only become of real value provided more international effort would be directed towards highlighting the initiative with national authorities (including those of his own ministry)

84 Non standard meaning without due recognition for comprehensive stakeholder arrangements and involvement.

Page 158: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

77

legitimate baselines to produce national and global information materials. Apart from this upward exposure of individual GIAHS, there is no substantial evidence that GEF contributions were systematically leading towards global incremental benefit. Relevance of the project baseline relationship is limited or at best inconsistent with GEF funding objectives. Impact from incremental global support to baselines is variable between satisfactory to low, depending on intervention choice.

GEF Budget

61. Finding 28: The budget situation discussed by the mission reflects the project situation of June 2012. The evaluation benefited from its perusal of an earlier review of project cycle management cost (PCM), carried out in 2011.85 The overall project budgetary picture for the FSP must be considered well-structured and up-to-date. Global project coordination has already carried out a fair amount of budget data analysis. Budget analysis made available to the evaluation is, however, limited to the GEF funding component. Only lump sum reported figures are provided for the RAF allocated national counterpart funding. 62. The quality of the available budget data allow for direct assessment of project progress. Disbursement progress against plan acts as a useful indicator of potential implementation obstacles. For this reason, budget information was used to verify and match MTE observations and working hypothesis. Budgetary data review was conducted only during a later stage of the evaluation and thus mainly served the purpose of triangulation and confirmation. MTE hypotheses themselves were developed during initial document review86 and gradually modified / honed as the evaluation went along. 63. Finding 29: Apart from the FSP to date, the PCM study covers the PDF-A and PDF-B phases of the project87. As such, this study provides a more comprehensive view of the project budget history. Of course, given the particular nature of the PCM analysis, its main focus is to put figures on the cost of project (administrative) management in relation to actual project delivery.

64. While this provides an indication of the effectiveness and efficiency of the project management model, information provided is relatively limited in relation to national and local project delivery performance. The PCM report and associated questionnaire do account for the (long) project gestation period88 and explain reasons behind the considerable budgetary reduction effectuated for the FSP89. The advanced role of the PDF-B in laying the participatory planning basis for FSP implementation is also discussed90. As the current

85 Rideau Strategy Consultants Ltd. 86 And included in the evaluation matrix questions. 87 The PPG amount was: US$725 000 (PDF-A: USD 25,000 approved on February 7, 2002; PDF-B: USD

700,000 approved on February 12, 2004 88 The project preparation was completed for GEF 3 , but was confronted with the depletion of funding sources

under GEF 3. As a consequence PDF-B lasted from early 2004 to mid-2008. Furthermore, intermediate changes in GEF oorganizational access took place with FAO becoming eligible in 2006. This led to an end of UNDP involvement, and entailed the need for considerable document restructuring , to meet FAO formats, and comply with changed GEF 4 eligibility requirements .

89 From US$6 mio anticipated during the PDF_B to US$3.5 mio actual as a result of the more limited resource availability under GEF4

90 The PCM study considers this as a beneficial attribute of the PDF-B in relation to the FSP. The MTE would argue that the advanced investments in local implementation planning contain a rather grave danger of

Page 159: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

78

evaluation is not specifically concerned with details of the PCM cost, it refers to the aforementioned consultancy report. 65. Finding 30: The project budget structure acts as an unintended cause for imbalance between the three project tiers. A project focus skewed in favor of on the ground (local) rather than policy (national) of globally-oriented interventions is reflected in the GEF component wise budget distribution. 66. As supported by the following figures, the budget layout clearly confirms that there is emphasis on national and local expenditures: 1) FAO global component US$ 1,000,000 or 28.5 % ; combined country budgets US$ 2,500,000 or 71.5 % of the GEF allocated budget. GEF country budgets are further sub-divided as follows: Algeria (US$ 200,000); Chile (US$600,000); China (US$500,000); Peru (US$600,000); Philippines (US$500,000); and Tunisia (US$100,000). 67. This high budget allocation for local and national interventions appears to have been an unfortunate resource match for a project with an essentially global ambition. For most participating countries91 national and local delivery rates have been lagging92. The MTE sees this as a confirmation of the predictable management and coordination obstacles which tend to rise when implementing local and national interventions through a non-decentralized coordination structure93. 68. The current project is no exception to the management logic which holds that effective local and national interventions must rely on sufficiently decentralized management. This is because of the short communication lines, and sufficient appreciation of specific national and local details needed for effective decision making. Conversely, the adoption of a global management structure94 with limited delegation results in increased complexity95 and more cumbersome lines of communication. 69. Moreover, global management has an associated price tag, as the proportional cost for management and technical backstopping96 inputs increases. Limitations and / or budgetary caps for global HR inputs and travel automatically leads to limitations in interaction. In all countries visited it was reported that frequency of global coordination interaction was perceived as less than desirable97.

__________________________ entering into commitments, which can subsequently not be honored. As is clearly the case with the FSP following the considerable budgetary reductions which took place between GEF3 and GEF4.

91 See the project expenditure table by country for the GEF budget component 92 The MTE was conducted with about 80% of the project time completed. The same percentage does serve as

an accurate indicator for expected project financial delivery, since slow start-up in delivery should at this stage of the project have largely evened out.

93 Read global management and budgetary controls. 94 The global coordination retained full control over budget expenditure decision making. This within the

framework of agreed annual work plans for each country. 95 Both the global project unit and national FAO Representations must coordinate their dealings with national

and local authorities. Distance of the global unit from the center of action requires added communication requirements or depends on in-frequent national presence of the global coordinator.

96 The global coordination unit is supposed to deal both with technical and administrative project issues, where required supported by global and national technical consultant inputs.

97 This is certainly the case for a novel concept, which simultaneously must be defined, refined, tested and adapted at global, national and local levels.

Page 160: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

79

70. Finding 31: Another factor specific to the project is that the global coordinating staff had no budgetary powers. Throughout the project, budgetary decision making depended on access to the FAO budget holder. The global coordinator / budget holder held a Divisional Director position which is accompanied by numerous managerial responsibilities, therefore limiting the amount of time which can be devoted to a single project. 71. There is evidence that a lack of financial delegation has slowed down project coordination and management decision making. Distant / remote management and the absence of entrusted national and local oversight also led to a sense of heightened budgetary risk aversion98. Combined factors all attributed to slow financial delivery of national and local budget resources. The fact that the project invested to an extent in national coordination has not had a fundamentally altering effect on the disbursement situation. For all purposes, with or without national coordination, the project remained a single unit99. 72. Finding 32: There are two notable exceptions of country specific budget expenditure targets which are close to the mark. These are China and Tunisia with an expenditure rate of 88% and 72% respectively. In China, strong ownership100 combined with substantial co-funding commitments have led to rapid expansion of GIAHS sites. GEF and co-funding budget absorption has been according to plan. In Tunisia, the reasons accounting for the satisfactory financial delivery are different. Here, the limited allocated amount and relative isolated NGO implementation in one single location have helped to increase the level of budget depletion. 73. For the other 4 participating countries national expenditure rates range as follows: 25% Algeria, 35% Chile, 32% Peru and 42% Philippines101. Without exception, the low financial delivery rates for the national and local budget components are explained by slow project start up at country level. Despite the extended project preparation under PDF-B, various reasons for project implementation delays have surfaced. These may be categorized under headings of: a) initial limited conceptual buy-in by national authorities, b) incomplete institutional and legal arrangements and c) the need to re-negotiate local components of project frameworks because of lower than expected GEF budget allocation. 74. Finding 33: By contrast, the (GEF) budget component allocated to FAO HQ has seen a depletion rate well on schedule. In June 2012, its delivery stood at US$853,144 or 85% of the allocated US$1,000,000. Approximately half of this sum or US$425,730 went into meeting salary and benefits of professional staff102, while the remainder was absorbed primarily by travel (US$173,659) and consultants (US$164,702).) The foreseen Travel budget line for the total project duration (US$262,982) has been substantially exceeded with current overall expenditures of US$371,724.

98 Which in turn tends to further slowdown decision making processes 99 There is not foreseen in sub-projects or baby project as part of the budget structure. 100 Conceptual as well as institutional. 101 The slightly higher delivery rate in the Philippines is explained by the fact formally no co-funding is formally

committed. National budget allocations are decided only on an annual basis. There are various regular and project related initiatives on record, which are on-going in parallel in the wider project environment, Some of these are based on discretional funding by congress and senate members. None of this funding is however counted as project co-financing. All expenditures by the national counterpart agency therefore solely rely on GEF budget.

102 The only recruitment under the project was the global coordinator budgeted at P3 level for a 5 year period.

Page 161: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

80

75. Finding 34: The GEF-funded budget line: Consultants (US$509,566) has seen a high levels of expenditure on the side of FAO HQ (global). This expenditure stands out from most of the project’s other global geographical components. The Peru consultant budget (US$107,246 spent) most closely approaches the HQ consultant budget. Not only does this make up for a relatively high percentage (18%) of overall allocated country budget (US$600,000), the quoted sum is even more important when considered as a fraction of the actual national expenditure to date (US$190,410). In this calculation, consultant costs make up 56% of the national budget. 76. The Peru situation can be explained in function of the large and remote project pilot area which covers diverse agro-logical environments. To a lesser extent, the same situation applies for Chile consultant expenditures (US$51,139). Here, the overall budget allocation is also US$600,000, of which consultant costs to date make up 8%,. The actual budget expenditure to date under this line is US$208,291), which translates in 25% of the outlay. In the case of Chile, a complete and relatively large island103 is subject of GIAHS pilot interventions, explaining the quoted numbers. 77. Finding 35: A relatively straightforward picture emerges, supported by the budget figures and the desynchronized manner of their depletion. As discussed, above, for some particular forms of budget use figures and percentages, it is clear that the project has been grappling with a contradictory desire to carry out local interventions in support of, and in interaction with the national level a global vision, but doing so without having had the benefit of decentralized management. 78. This is corroborated by the MTE interviews at country level and specifically with staff of FAO Representations. Because of the light footprint of the project at the national and local level, implementation issues have accumulated and prevented the project from a higher intensity of local and national delivery. This is largely reflected by the progress in attaining outcomes, outputs and activities at these two levels. As a result, the feedback of information from local and national initiatives to the global level has also been slower in arriving. This, in turn, has affected the refinement process of the global GIAHS concept. The added budgetary cost required for this project’s managerial / operational scenario is substantial. 79. With three components of the project initially slowed down, it has become clear that managerial / coordination costs remains fixed. Budget allocation for outcome 4 also did not slow down significantly despite the slower than expected progress of outcomes 1,2 and 3. This is because, regardless of progress on the “ground” or that of policy and legal issues, information and communication materials can be produced. This also demonstrates that the communication of ideas and information on GIAHS is not so closely correlated with project-generated interventions and lessons learned. 80. Finding 36 (Consolidation): GEF budgetary allocation for a combined range of global, national and local outcomes was substantially reduced between the PDF-B and FSP phases of the project104. Additionally, the project management structure and overly strong emphasis on locally directed interventions gave rise to an internal budgetary imbalance within the GEF project component. Global project management has become disproportionally absorbed with issues of a local (and occasionally national) nature.

103 Chiloe 104 The budget was effectively reduced to half of the originally foreseen outlay.

Page 162: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

81

81. As a consequence, global and national outcomes have lagged in proportion to budgetary depletion for these outcomes. The MTE takes into account the inefficiencies imposed by the project design. In this context, implementation effectiveness for global and national outcomes has been constrained. Funding limitations for these components, together with other factors such as HR availability, networking and partnership arrangements have played a contributing role. 82. Meanwhile, as a result of the same inefficiencies, the local outcome has been over-funded, and has seen low budgetary absorption. The project has, in this case, been affected by field-level interaction of a less than optimal efficiency. This may be accounted for by HR limitations, distance and the high cost of travel for the global coordinator105 . A more rational design and budgetary balance would most likely have resulted in substantial gains of budgetary efficiency. 83. There is little doubt that many of the issues of limited budgetary efficiency could have been reduced or circumvented by a management and budget structure allowing for decentralized decision making over the local and national elements of this global project. The global project aspects should have been better resourced from a budgetary perspective. This is essential to ensure adequate impact through FAO staff mobilization and full-fledged GIAHS participation in relevant international forums106.

105 and at national / local level. 106 An example of such international engagement is the need to pursue more comprehensive participation in

meetings of the UNESCO WHC, this in order to forge tangible linkages and working mechanisms.

Page 163: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

82

Annex 11: Background to Project Consultancies

Name Consultant Timing and Purpose Consultancy Natalia Acosta 16 February 2012, 6 months, Assistance with national Work plan consolidation and

project reporting, secretarial, web page maintenance, information collection and screening.

Miguel Altieri ,

February and March 2010, Undertake in-depth studies of GIAHS Andean and Chiloe for the Preparation of the publication for the Globally Important Agricultural Heritage Systems (GIAHS) Initiative.

Miguel Altieri August 8th 2010, systematic study of GIAHS implementation and preparation of the book publication for the Globally Important Agricultural Heritage Systems (GIAHS).

Miguel Altieri, 16 Sept. 2010 until 15 Oct. 2010, to prepare an executive summary of the GIAHS concept aimed for policy makers, donors and governments.

Miguel Altieri 8 June 2011, 10 days, present a paper on dialogue among agricultural civilizations to the international GIAHS Forum, to be held in Beijing from 8-13 June 2011 Conduct site visit in China and apply GIAHS methodological frame work(sustainable livelihood framework with the five capitals) for the assessment of GIAHS-China.

Miguel Altieri 1 September 2011, 20 days, Apply GIAHS methodological framework (sustainable livelihood framework with the five capitals) for the assessment of GIAHS-China and prepare critical review of the method and the system. With reference to GIAHS methodological framework and CBD selected training modules on NBSAPs, prepare GIAHS course/training module applicable for Universities and Institutions (max. 50 pages including illustrations). Participate to OECD-FAO expert meeting for the preparation of Rio + 20 (3 days), Participate to Global Soil Partnership meeting (3 days)

Miguel Altieri 21/11/20111. Finalization of the Chile/Peru assessment technical report; Finalization of the technical report on SLA application - China Rice Fish culture; Final draft of the GIAHS book

Norah de Falco , November 9th 2011, 5 months, Communications Expert - Consultant

Kathleen Marcela

Flores Bulnes

5/12/2011, 1 month, Prepare and submit the global GIAHS project video in 3 languages (English, French and Spanish) both in HD and web format.

Grifi 25 April 2011, 2 months, Web designer

Dr. Lauro

Hernandez

Senior adviser Philippines, national level advisory support meant to accelerate national project progress

Iana Jekova 13 February 2012, 14 days (WAE) over 21 days, Upload all the text of the GIAHS website from Word into the existing TYPO3 test site following the provided navigation outline;Upload photos, documents and other provided materials on the website.

Charles Lilin 15 December 2010, 29 days, GIAHS (SIPAM) Book authoring and redaction, improvement of GIAHS conservation benefit justification based on f.i. biodiversity characteristics, Identification of potential new GIAHS based on library and web research. Knowledge gap analysis for candidate GIAHS

Alfredo Mandl 15 December 2010, 10 days, Spanish Translation of document “CONSERVATION AND ADAPTIVE MANAGEMENT OF GLOBALLY IMPORTANT AGRICULTURAL HERITAGE SYSTEMS (GIAHS)”

Mei 01 February 2011, 5 months, Prepare a consortium document for GIAHS Initiative; Edit, revise and update the GIAHS website text (in collaboration with the communication group for the design and layout); Promote/make GIAHS Initiative available on various online media resources (Wikipedia, facebook, etc); Draft and package technical reports for publication (India report, GIAHS book publication, etc); Prepare a feasibility study for product labelling and certification of pilot countries; Assist in the preparation and organization GIAHS Forum 2011;

Annie Monpetit 5 March 2012, 5 April 2012, Based on the GIAHS template and criteria, identify and

Page 164: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

83

describe potential sites by analyzing the information (reports, background documents) published on World Heritage Sites, Indigenous Conservation Community Areas, Ramsar and other relevant international organisations and GIAHS partners websites; Summarizing the gathered information in a manner that can be readily utilized by GIAHS team to: facilitate future communication with identified sites/partners, and prepare background documents;Drafting operational guidelines for the GIAHS Partnership Initiative and conceptualizing them in a coherent and comprehensive manner

Puzzo 18 January 2011, 6 months , Internship Assist in the preparation of documents for the registration of the GIAHS (banner, logo and other materials for “certification mark”) Prepare a draft agenda/proposal for each country to conduct a training to become fully aware of the WIPO and related agenda for GIAHS needs, Perform other general support to other day to day activities of GIAHS project as they emerge

Sarvi 1 month, Support in consolidating pilot country activity workplans, technical reports; Provide support in preparation, designing and managing/updating information of the global GIAHS webpage;draft information/concept notes, papers and other communication materials for GIAHS e.g. Rio+20, FAO interdepartmental requests, donor’s request, etc.;Carry out other tasks pertaining to NRL-GIAHS Initiative, as deemed necessary.

Prof Mario Tapia

12 days with M Altieri , Documentation/assessment of agricultural systems by agroecological domains, specific constraints and adaptations at cropping and farming systems, social and territorial levels; The “Heritage Concept” in the Latin American Countries – Agricultural Heritage, cultural values, natural cultural landscapes, initiatives and programmes dealing on heritage conservation and management; Provide definition/review of local terminologies and knowledge systems; Verify recent progress in the Andean agricultural systems’ site; Assess/evaluate the impacts of climate change related disasters on the adaptation of traditional technologies.

Kazem Vafadari August 4th 2011, 35 Days, Home-based, with 3-4 times field visit to Ifugao, Philippines : Introduction of the study and pre-assessment of the local community and government officials as regards “GIAHS Tourism”; 2-documentation of the existing level and situation of tourism in IFUGAO rice terraces area, including the market size and marketing activities, infrastructure, attractions, human resource management for tourism and the institutional capacity ; and conduct workshops with local community and officials (two levels) for: (i) human resource capacity building based on the needs and priorities identified in the first two steps of the research; and (ii) develop indicators on measuring sustainability of tourism.

Carlos Venegas

Between February and March 2010, with travel to Peru, The Consultant shall work with Prof. Miguel Altieri on the following tasks: Documentation/assessment of agricultural systems by agro-ecological domains, specific constraints and adaptations, at cropping and farming systems, social and territorial levels; The “Heritage Concept” in the Latin American Countries – Agricultural Heritage, cultural values, natural cultural landscapes, initiatives and programmes dealing on heritage conservation and management ;Provide definition/review of local terminologies and knowledge systems ;Assess/evaluate and report progress in the Chiloe agricultural systems including Andean sites.

Page 165: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

84

Annex 12: List of Project Publications (Selection)

Project Publications:

• A Methodological Framework for the Dynamic Conservation of Agricultural Heritage Systems

• GIAHS: extent, significance, and implications for development • GIAHS – An examination of their context in existing multilateral instruments • Local Knowledge Systems and the Management of Dryland Agro-ecosystems: Some

Principles for an Approach • GIAHS: An Eco-Cultural Landscape Perspective • UNESCO World Heritage Centre: Background document on UNESCO World

Heritage Cultural Landscape • Agricultural Biodiversiy of Global Significance and GIAHS – logical steps • GIAHS Strategic Framework • Globally Important Agricultural Heritage Systems: from a Social-Ecological Systems

Perspective to a Scientific Conceptual Framework and Strategic Principles • A case of Agricultural Heritage Systems • Globally Important Agricultural Heritage Systems: Notes and guides in assessing

Natural Resources Management • 18.05.2012 • Apatani Wet Rice Cultivation: an example of a highly evolved traditional

agroecosystem • GIAHS: Towards analyzing the drivers of change in farming systems

Others:

• Developing governance mechanisms for agricultural heritage conservation, December, 2007, by Frank van Schoubroeck, Luohui Liang, Qingwen Min, Arend-Jan van Bodegom

• Mid Term Evaluation of Supporting Food Security and Reducing Poverty in Kenya and the United Republic of Tanzania through Dynamic Conservation of Globally Important Agricultural Heritage System (GIAHS)- GCP/GLO/198/GER

General Project Related Documents:

• PDF-A Document & Terminal Report : Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS)

• PDF-B Document & Terminal Report: Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS)

• FSP Conservation and Adaptive Management of the Globally Important Agricultural Heritage Systems (GIAHS) Project, GCP/GLO/212/GFF

• National Consultant Report China MTE GCP/GLO/212/GFF

• National Consultant Report Tunisia MTE GCP/GLO/212/GFF

• National Consultant Report Peru MTE GCP/GLO/212/GFF

• National Consultant Report Algeria MTE GCP/GLO/212/GFF

• GCP/GLO/212/GFF Progress Reports

• International Forum on Globally Important Agricultural Heritage Systems (GIAHS),

Beijing, China, 9-12 June 2011

• Proceedings of the Second International Forum on Globally Important Agricultural

Heritage Systems (GIAHS), Buenos Aires, Argentina, 21-23 October 2009

Page 166: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

85

• Summary of Symposium 38 (GIAHS and Biodiversity Conservation)at Eco Summit

2007

• Proceedings of the international forum on Globally Important Agricultural Heritage Systems (GIAHS) A Heritage for the Future, 24-26 October 2006

• Second International Workshop and Steering Committee Meeting, 7-9 June 2004 • First Stakeholder Workshop and Steering Committee Meeting, 5-7 August 2002 • GCP/GLO/212/GFF , FAO/GEF Project Implementation Review (PIR), 01 July 2008

to 30 June 2009 • GCP/GLO/212/GFF , FAO/GEF Project Implementation Review (PIR) 01 July 2008

to 30 June 2010 • Various BTOR • Various Consultant ToRs

China Publications

• Book: Research on Agro-cultural Heritage and Its Dynamic ConservationⅢⅢⅢⅢ. Beijing:

China Environmental Science Press, 2010

China Academic articles

• Analysis on Driving Factors of Land Use Change in Traditional Agricultural Regions: An Empirical Study Based on Household Behaviors. Resources Science. 2010, 32(6):1050-1056. (In Chinese)

• Evaluation Models for Multifunctionality of Agriculture and Their Applications: A Case Study on Qingtian County in Zhejiang Province , China.Resources Science. 2010, 32(6):1057-1064. (In Chinese)

• A Study on the Assessment Index of Agrobiodiversity: A Case on Congjiang County in Guizhou Province. Resources Science. 2010, 32(6):1042-1049. (In Chinese)

• Assessment on Tourism Resources Potential for Agricultural Heritage Sites : A Case Study on Qingtian Count y in Zhejiang Province. Resources Science. 2010, 32(6):1026-1034. (In Chinese)

• Protection and Inheritance of Ethnic Cultures in Agro-Cultural Heritage Tourism Community: A Case Study of Ping’an Village of Guilin City , Guangxi. Journal of Guangxi Normal University : Philosophy and Social Sciences Edition. 2010, 46(4):121-124. (In Chinese)

• A Preliminary Study on Regional Social Responses to Changes in Forest Resources in the Southeast of Guizhou Province throughout the Qing Dynasty. Resources Science. 2010, 32(6):1065-1071. (In Chinese)

• Explanations and Enlightenments of the GIAHS’s Criteria. Resources Science. 2010, 32(6):1022-1025. (In Chinese)

• An Investigation of Residents’ Perception on Tourism in Agro-Cultural Heritage Community:A Case Study on Ping’an Village of Guilin City, Guangxi. Resources Science. 2010, 32(6):1035-1041. (In Chinese)

• Progress and Perspectives of China’s Ecological Agriculture. Resources Science. 2010, 32(6):1015-1021. (In Chinese)

• Study on the Tourism Development of Community Cultural Hereditarily. Journal of Anhui Agri Sci 2010, 38( 22): 12091-12092, 12294. (In Chinese)

• Ecological studies on the food web structures and trophic relationships of multiple species coexistence in paddy fields using stable carbon and nitrogen isotopes. Ac ta Ecologica Sinica. 2010, 30( 24) : 6734- 6742. (In Chinese)

• Forum on Hani Terrace Agriculture Heritage and Dynamic Conservation. (In Chinese)

Page 167: Food and Agriculture Nations Office of Evaluation · report and also contributed review and editing inputs in the course of national evaluation report preparation. The work carried

86

• Evaluation of comprehensive benefit of rice-fish agriculture and rice monocropping——A case study of Qingtian County, Zhejiang Province. Chinese Journal of Eco-Agriculture 2010, 18(1):164-169. (In Chinese)

• Effects of different rice farming systems on paddy field weed community. Chinese Journal of Applied Ecology. 2010, 21(6):1603-1608. (In Chinese)

• Measurement of ecosystem services consumption: a case study of the traditional agricultural area in Congjiang County of Guizhou Province. Acta Ecologica Sinica. 2010, 21(6):1603-1608. (In Chinese)

• Study on the Tourism Resource Characteristics of Agricultural Heritage. Tourism Tribune. 2010, 25(10):57-62. (In Chinese)

• GIAHS Project and Its Implementation in China. Journal of Resources and Ecology. 2010, 1(1):94-96.(In English)

Maroc Publications:

• Promotion des systèmes de savoir et de culture conservation dynamique des systèmes ingénieux du Patrimone Mondial (SIPAM) au Maroc Site pilote dans le Haut Atlas Oriental : Imilchil-Amellago Argumentaire détaillé selon le format SIPAM pour inscription sur la liste FAO de Systèmes Ingénieux de Patrimoine Agricole Mondial.