Ex-Post Evaluation of Projects and Activities Financed ... · Part 6 presents the conclusions and recommendations arising from the evaluation's various components1, including: •

Directorate General Environment, Unit E.4. LIFE

Ex-Post Evaluation of Projects and Activities Financed under the LIFE Programme

Final Report Part 6: Conclusions and Recommendations

July 2009

Directorate General Environment, Unit E.4. LIFE

Ex-Post Evaluation of Projects and Activities Financed under the LIFE Programme Final Report Part 6: Conclusions and Recommendations

July 2009

COWI A/S Parallelvej 2 DK-2800 Kongens Lyngby Denmark Tel +45 45 97 22 11 Fax +45 45 97 22 12 www.cowi.com

Report no. 7-6

Issue no. 1

Date of issue July 2009

Prepared bim, ar, jek, tih, sbj

Checked jek, il, bim, neo

Approved bim

This report has been prepared as a result of an independ-ent evaluation by COWI being contracted by the Director-ate General Environment The views expressed are those of the Consultant and do not necessarily reflect those of the European Commission.


Part 6: Conclusions and Recommendations

O:\A000000\A001146\Final Report for PDF\Part 6_LIFE_finrep_1.doc

1

.

Table of Contents

1 Introduction 2

2 Conclusions 3 2.1 Evaluation question 1: What are the results and

impacts of the LIFE Programme 4 2.2 Evaluation question 2: To what extent are the results

in line with the objectives 7 2.3 Evaluation question 3: To what extent does the LIFE

unit organisational set-up contribute to the effectiveness of the programme? 8

2.4 Evaluation question 4: To what extent are the costs at which results have been achieved reasonable? 9

2.5 Evaluation question 5: To what extent does the LIFE unit organisational set-up contribute to the efficiency of the programme? 10

2.6 Evaluation question 6: To what extent do the results and impacts address the environmental issues facing the EU and the stakeholders involved? 11

2.7 Evaluation question 7: To what extent are the positive results and impacts likely to last after the termination of the projects? 13

2.8 Evaluation question 8: To what extent has the LIFE Programme been complementary/supportive/contradictory to other EU policies and programmes? 14

2.9 Evaluation question 9: To what extent do stakeholders accept the LIFE programme and LIFE projects 15

3 Recommendations 16 3.1 Strategic management and programming 16 3.2 Coordination, dissemination, partnerships 18 3.3 Procedures for selection and monitoring 19




2

.

1 Introduction This document constitutes Part 6 of the final report in connection with the Ex-post Evaluation of Projects and Activities under the LIFE Programme.

Part 6 presents the conclusions and recommendations arising from the evaluation's various components1, including:

• Evaluation of programme management (ref. Part 2 of this evaluation report) • Country-by-country analysis of the implementation of the LIFE Programme comprising 26

country studies (ref. Part 3 in this evaluation report) • Thematic analysis of the implementation of the LIFE Programme comprising ten thematic re-

ports and horizontal conclusions on the LIFE Environment and Nature components (ref. Part 4 of this evaluation report)

• Evaluation of the implementation of the LIFE Third Countries component (ref. Part 5 of this evaluation report)

As is also evident from the individual reports provided in the various parts of this evaluation, the LIFE programme was de-facto three programmes (LIFE Environment, LIFE Nature and LIFE Third Countries) with some similar characteristics in terms of their administrative management but with different objectives. For this reason conclusions most often have to be given for each of these components separately.

• The LIFE Nature component had clear objectives linked to specific legislation and comprised a large number of projects with similar characteristics.

• The LIFE Environment component was multi-thematic, diverse in terms of regulation and project types, with a focus on innovation and demonstration rather than particular environmental themes.

• The LIFE Third Countries component focused on other priorities namely capacity building and policy development.

1 The components and the methodology are presented in Part 1 of this evaluation report




3

.

2 Conclusions The conclusions are presented below according the structure of the seven primary and two secondary evaluation questions together forming the nine evaluation questions that guided the evaluation:

Primary questions 1-7

1. What are the results and impacts of the LIFE Programme?

Effectiveness:

2. To what extent are the results of the LIFE Programme in line with the objectives?

3. To what extent do the LIFE unit organisational setup, and the modalities for selection/monitoring of projects, contribute to the effectiveness of the programme/projects?

Efficiency:

4. To what extent is the cost at which results have been achieved reasonable when considering the costs of comparative interventions?

5. To what extent do the LIFE unit organisational setup, and the modalities for selection/monitoring of projects, contribute to the efficiency of the programme/projects?

6. Utility: To what extent do the results and impacts generated by the LIFE Programme address the environmental issues facing the EU and the stakeholders involved?

7. Sustainability: To what extent are the positive results and impacts generated by the LIFE Programme likely to last after the termination of the projects?




4

.

Secondary questions (8-9)

8. Consistency/coherence: To what extent has the LIFE Programme been complementary/supportive/contradictory to other EU policies and programmes?

9. Acceptability: To what extent do stakeholders accept the LIFE Programme and LIFE projects?

2.1 Evaluation question 1: What are the results and impacts of the LIFE Programme

Summary conclusion:

The three LIFE components have produced projects results that reflect their specific character: LIFE Nature improved the NATURA 2000 network, restoring habitats and strengthening species protection while LIFE Environment across nine environmental themes promoted product and management innovation and demonstration. The environmental effects of LIFE Environment were mainly local. LIFE Third Countries assisted mainly local environmental authorities in 16 beneficiary countries to modernise environmental administration.

LIFE Nature During the period 1996-2006, the LIFE Programme co-financed 771 Nature projects thus accounting for 42% of all LIFE projects. The total LIFE contribution to the Nature projects in the period amounted to EUR 637 million.

LIFE has made a significant contribution towards implementing Natura 2000 in the Member States. The projects are estimated to have covered 8-9% of all Natura 2000 areas and a significant share of the habitats and species listed in the Annexes to the Birds and Habitats Directives. In terms of the area covered, this is estimated to be approximately 3-6% of the entire Natura 2000 network for the Old Member States. The actual coverage seen in relation to area in need of intervention is undoubtedly higher. However, since there is no data on the share of the Natura 2000 area requiring intervention, it is not possible to determine how much higher. Considering the relatively limited budget of approximately EUR 70 million per year, this is considered a significant achievement.

The findings indicate that an area of approximately 320,000 hectares was restored as a result of LIFE projects in the evaluation period. The projects focusing on habitat restoration most often resulted in achievement of favourable conservation status and ensured the continued management of the area and thus the long term impacts of the projects were either that the area remained in restored condition or that the restored area was enlarged. A large share of the projects also led to an enlargement of Natura 2000 sites or facilitated the designation of new Natura 2000 sites, thus leading to an enlargement of the network. The types of habitats most typically covered by LIFE projects were grasslands and wetlands as well as raised bogs and mires.

The projects focusing on species protection targeted a very significant share of the bird species listed in the annex to the Birds Directive. This is remarkable as the 195 species listed contains almost 37% of the bird species regularly occurring in Europe. However, the projects have not always




5

.

encompassed the individual species according to their threat status. Some rare species were not targeted, whereas others were targeted by numerous projects. In terms of coverage of species listed in the Annex II of the Habitats Directive about half of the animals species (especially mammals) have been targeted by LIFE projects, whereas the coverage for plants is lower. The evaluation shows that approximately half of the projects aiming at species protection or reintroduction achieved favourable conservation status for one or more species in the long term.

The LIFE projects have played a very important role in increasing the general level of awareness concerning biodiversity, Natura 2000 and the related policies and regulatory requirements among national, regional and local authorities. Also, through the LIFE projects, capacity to implement and manage the interventions required to implement the Birds and Habitats Directives was built up - both within relevant authorities and NGOs. This pump priming effect affected in a positive direction the general implementation of the Directives within the individual Member States.

At the same time, there is evidence that only a minority of the projects were seen and used as demonstration projects. Many projects have tended to live their own life and there has been little incentive and direct opportunity to share knowledge, especially at the supranational level. At the same time, the projects represented many parallel initiatives across Europe with a significant overlap in project scope, methods and achievements thus indicating an unused potential for cross-project fertilisation.

LIFE Environment The project portfolio of LIFE Environment in the period 1996-2006 comprises a total of 1076 projects.

LIFE Environment differs from LIFE Nature as the environmental issues addressed by the component are much broader and more varied. The multi-thematic composition of the LIFE Environment component implies that there are relatively few projects under each theme and this made it difficult to validly establish the results and impacts at thematic level.

Across the LIFE Environment themes the projects had the following main objectives (some projects having multiple objectives):

• Environmental improvement - 29 % • Management - 19 % • Awareness raising - 17 % • Demonstration - 42 % • Innovation - 31 % The direct or immediate environmental results and impacts were generally small and typically restricted to the project area/site itself, which in most cases mean quite local. Due to the many different types of projects and subjects supported it was not possible to quantify and aggregate environmental results from the project level to the thematic / programme level. Many projects did not have direct environmental improvement as their main objective (this is only the case for 29 per cent of the projects) but have rather aimed to demonstrate the feasibility of new technologies or other approaches to improvement of environmental conditions, i.e. the projects have often not been full-scale but pilot-scale projects.

The information available points at a number of cases of increased management capacity of, in particular, local and regional authorities as a result of the projects undertaken. The additional




6

.

management capacity thus created is often assessed not to be restricted to the issue studied but to have a wider applicability. Increasing awareness on environmental issues was a main objective only in few projects and awareness was typically created locally in the project area.

The innovation 'content' is generally found to be highest in technology projects undertaken by private enterprises (and research institutions) while many projects with public entities or NGOs as beneficiaries only to a more limited extent comprised truly innovative elements. A significant part of the projects, especially those of the management type, seem not to be genuinely innovative but rather adaptations or optimisations of existing systems, approaches and methods to a particular geographic setting or other specific conditions.

The wider impact arising from the possible replication of the projects is uncertain and difficult to assess, but it is assessed that it is only occurring in approximately 10-20 per cent of the projects. Many projects have been found to demonstrate technical feasibility and a number even additionally to be economically viable, however there is evidence that the demonstration potential being present was often not fully released or exploited because the necessary dissemination of activities and results did not take place or at least only to a limited extent. Further, the demonstration power within a certain theme or issue may have been insufficient due to too few projects i.e. not sufficient weight of evidence produced.

LIFE Third Countries During the period 1996-2006, 179 projects were implemented in 16 beneficiary countries with a total budget of EUR 93.25 million of which EUR 61.9 million was co-financed by the EC. A third of the projects have been financed in the new Member States (Cyprus, Malta) and the candidate countries (Croatia and Turkey), and 28 per cent in Russia, Bosnia and Herzegovina, Israel and Lebanon. The 8 remaining countries had 30 per cent of the projects

The main results relate to an improvement of the beneficiaries’ capacity to undertake environmental protection, environmental management and policy-making (while the projects to a limited degree have produced new environmental legislation per se). The most successful elements of LIFE Third Countries were transfer of experience and know-how between the local and international partners and beneficiaries, highlighting the environment at a national policy level and raising awareness. However, the projects generally suffer from a lack of involvement of national partners and EC Delegations in the projects.

It is finally seen, that the results and impacts of LIFE Third Countries depend strongly on the countries in question. Significant impacts were found in New Member States (Cyprus, Malta) and candidate countries (Croatia and Turkey) while the impact of the projects were modest in the other countries.




7

.

2.2 Evaluation question 2: To what extent are the results in line with the objectives

Summary conclusion:

The specific objectives guiding LIFE Nature were relatively clear focusing on implementation of the Habitats and Birds Directives. Effectiveness is assessed as high as the projects have clearly made a significant contribution to the implementation of these Directives in the Community.

The specific objectives for LIFE Environment were less clear which provides a weak basis for assessing effectiveness. Since it was the objective of the LIFE programme to fund environmental projects and since this evaluation documents that eligible projects were selected and subsequently have been effectively implemented, a satisfactory relation between results and objectives can be noted. However, due to a diffuse programming strategy for LIFE Environment that did not couple specific environmental priorities with selection criteria it is not possible to assess at a more specific level the degree to which the results are in line with the objectives.

LIFE Nature The project-level effectiveness of the Nature projects is assessed as high. Most projects reached their objectives. Considering programme-level effectiveness, the projects have made a significant contribution to implementation of the Birds and Habitats Directives in the Member States and the results were thus well in line with the specific objectives of the Nature component.

LIFE ENV The project-level effectiveness of the LIFE Environment projects is assessed as high as more than 80 per cent have reached all or nearly all of their objectives. The success rate was irrespective of theme.

The assessment of programme-level effectiveness of the LIFE Environment component is hampered by the fact that the LIFE Environment programme did not have a strong strategic focus in terms of environmental policy priorities. The objective of LIFE Environment was to support the development of innovative and integrated techniques and methods within the broad framework of the general Community environmental policy and the criteria for project selection were designed accordingly. Although the innovative content of some projects may be discussed, the majority of the projects can be argued to support innovative and integrated techniques/methods at the very general level. As such results are in line with the objectives. At the more specific level, there is a diffuse relation between specific environmental policy priorities and LIFE Programme objectives and criteria for project selection for which reason the effectiveness cannot be assessed.

LIFE Third Countries The Third Countries component aimed at supporting capacity building and developing environmental policy in third countries bordering on the Mediterranean and the Baltic Sea. The projects contributed to this end and there was good correspondence between objectives and results. It is also a significant evaluation result that few of the projects (with some variance between the countries) arguably would have been implemented without the LIFE Programme because of a lack of local funding and the low prioritization of environmental issues or they would have been implemented later or on a smaller scale and with a different structure. LIFE was thus a facilitator for prioritisation of the environment in countries with limited environmental policy capacity - which is fully in line with the objectives of the LIFE Third Countries.




8

.

2.3 Evaluation question 3: To what extent does the LIFE unit organisational set-up contribute to the effectiveness of the programme?

Summary conclusion:

The organisation of the LIFE Unit as a functional, project management secretariat did to a high extent contribute to an effective selection and monitoring of individual projects. The set-up was less effective concerning the strategic management of the programme. Although managed in accordance with the objectives set out in the respective LIFE Regulations, little interest was shown on the part of the LIFE Unit and DG Environment as a whole to further target the programme to specific policies. While the monitoring of the individual projects was very detailed, there was no reporting on how the programme was performing at programme level.

The LIFE programme was managed by DG Environment's LIFE Unit which had as its sole responsibility to select - on the basis of criteria specifying eligibility - nationally submitted projects and to monitor the implementation of the projects selected for co-financing. The LIFE Unit was created in 2001 when there was a restructuring of the organisation, shifting the responsibility for managing the LIFE Programme from the policy Units in DG Environment to a Unit dedicated to this purpose.

The evaluation shows that the LIFE Unit was highly effective as concerns the management and monitoring of the individual projects. The Unit was organised as a functional, project management secretariat with officers each responsible for a portfolio of projects and with standardised procedures and rules in the form of SAP and CP and this facilitated the high degree of effectiveness.

The level of effectiveness is assessed to be considerably lower concerning the strategic management of the programme. The LIFE Programme was managed in accordance with the objectives set out in the respective LIFE Regulations, but there was little effort (on the part of the LIFE Unit and on the part of DG Environment as a whole) to further target the programme to support the implementation of specific policies. While the monitoring of the individual projects was very detailed and effectual, the monitoring and assessment of programme performance against its objectives was virtually non-existent. The evaluation further shows:

• Budget execution was high during the evaluation period, nearing 100%. Data on projects selected show that generally almost all projects passing the award stage in the selection process were financed and there was only a small surplus of qualified projects.

• During the evaluation period, the LIFE Unit became increasingly pro-active in marketing the LIFE Programme and managing relations with the Member States. Given that the LIFE+ annual budget is considerably higher than under LIFE III an extra effort to attract qualified projects and applicants and to improve application procedures and guidance is relevant.

• The selection procedure and criteria were very focused on project feasibility and the projects were thoroughly scrutinised to verify that they were realistic and implementable and that beneficiaries had the necessary technical and financial capacity to manage the projects. The




9

.

selection criteria also included elements to ensure coherence with Community environmental policy (especially for NAT) and European value added (especially for ENV).

• During the evaluation period, several steps were taken to improve transparency and equal treatment in the selection process, and the majority of beneficiaries and national focal points express satisfaction with the system. However, the analysis shows that: 1) there are some ambiguities in the definition of criteria and scoring systems concerning mainly examples of: a) guides to applicants mentioned priorities which were not reflected in the criteria of the evaluation guide, b) lack of clarity in how the sub-criteria/questions mentioned under each criteria are to be weighted to arrive at a score for the criteria; 2) the Unit was very involved the evaluation procedure on the basis of which the impartiality of the evaluation may be questioned as the same officials who are responsible for monitoring the projects are also involved in selection; 3) the Member States provided varying degree of assistance to project applicants, which made the playing field uneven.

• There is a high degree of satisfaction on the part of beneficiaries with the way the LIFE Unit has handled the monitoring tasks and with the assistance provided from the monitoring team. The reporting requirements for beneficiaries, coordination mechanisms between the Unit and the monitoring team, and the policy for monitoring visits contribute to providing the LIFE Unit with very detailed information about each project. BUTLER is assessed as a well-functioning tool for individual project monitoring and management.

• Since the restructuring in 2001 the coordination between the LIFE Unit and the policy units in DG ENV has been limited, particularly in relation to LIFE ENV. It mainly took place in connection with the selection phase and when preparing larger thematic publications on LIFE projects. The units appreciate the information they get in the form of cases with interesting projects and similar, but also call for more information on results and outcomes, while at the same time recognising that they have limited resources for LIFE Programme issues.

2.4 Evaluation question 4: To what extent are the costs at which results have been achieved reasonable?

Summary conclusion:

Since the LIFE Programme 'produces' not easily measurable 'products' as environmental improvements, innovation and capacity-building, a precise cost-effectiveness assessment cannot be given. But the evaluation has whenever possible constructed comparisons and have compiled several cost-benefit assessments of a qualitative character. On the basis hereon, the evaluator can conclude that no examples of excessive projects costs were detected and that the efficiency appears to be reasonable.

LIFE Nature The available data indicate a satisfactory level of efficiency of LIFE Nature projects. In terms of the overall efficiency of the programme, a benchmark at the overall level could be the Commission's cost estimate from 2004 indicating an annual expenditure of EUR 6.1 billion required to implement the Birds and Habitats Directives (corresponding to EUR 61 billion during the evaluation period). Seen in this perspective, covering 8-9% of the Natura 2000 framework with a mere EUR 640 million




10

.

indicates a high degree of efficiency, even when considering that it typically represents a co-financing share of 50%. However, the cost estimate includes both "investment costs" and recurrent costs and is not specific with regard to their respective shares. More precise cost estimates related to the implementation of the Natura 2000 framework would allow a more detailed evaluation of the efficiency of the LIFE Nature component.

LIFE Environment The efficiency of the LIFE Environment projects is difficult to evaluate. For most projects the relevant information is not available and/or there is no basis for comparison as many of the projects due to their innovative character are unique, at least in the country or sector in which they are executed. LIFE Environment projects are due to their innovative, development or pilot nature almost bound to be more expensive than the mere implementation of a known process or methodology and the efficiency should of course be viewed in that light. Qualitatively the majority of the stakeholders consulted consider the projects to be reasonably efficient. It should be noted that many projects consist of a number of elements of which some turn out to be feasible and efficient while others are not.

LIFE Third Countries LIFE Third Countries projects became more cost effective over time following improvements to the monitoring system. The projects were generally cost effective when one compares the limited budget involved with the positive local impacts. In addition, the costs consisted mostly of personnel costs (often of local or NGO staff), which are much less expensive than EU consultant wages.

2.5 Evaluation question 5: To what extent does the LIFE unit organisational set-up contribute to the efficiency of the programme?

Summary conclusion:

The evaluation shows mixed results. Administration costs relative to commitments have increased slightly over time indicating reduced efficiency and a LEAN analysis of work procedures documented potentials for streamlining of certain processes. A comparative study of LIFE Unit administrative costs with an EU Executive Agency (EACEA) however indicated a comparatively satisfactory level of efficiency.

The following elements of the LIFE Unit organisational set-up contribute to efficiency:

• A high degree of standardised procedures, which minimise transaction costs • A high degree of outsourcing of tasks which can be carried out at less cost by contractors • A continuous focus on improving procedures and processes to optimise efficiency

However, despite the efforts of the LIFE Unit to streamline and increase efficiency, the calculation of administrative costs compared to commitments made indicate a slight annual increase in the period 2002-2006.

In comparison with a similar cost ratio for the Education, Audiovisual and Culture Executive Agency (EACEA), which manages a range of financing programmes with similar modalities to the LIFE




11

.

Programme, the efficiency of the Unit is at par with the efficiency of this executive agency, which indicates a high degree of efficiency.

While there is generally a high degree of satisfaction among the stakeholders, areas for improvement have also been put forward. A LEAN inspired analysis provided a number of areas where further efficiency gains could be obtained. These concerned most notably a complete outsourcing of the selection activities and by improving the coordination with the monitoring team. Further elaboration is provided under recommendations below.

2.6 Evaluation question 6: To what extent do the results and impacts address the environmental issues facing the EU and the stakeholders involved?

Summary conclusion:

The utility of LIFE Nature is high. LIFE Nature made a significant contribution towards addressing the challenges related to nature conservation in the Community and reaching the objectives of the Biodiversity Action Plan. However, there is also room for continued development of the LIFE Nature instrument to respond to developments in biodiversity policy and the future challenges.

LIFE Environment contributed to addressing local needs and priorities but the bottom-up programming strategy did not ensure that the most important/urgent environmental problems from a Community perspective were addressed.

The value of LIFE Third Countries was that it was a useful instrument to tackle local needs of the beneficiaries whilst not necessarily addressing the most important priorities of the country or region.

LIFE Nature Utility is assessed as high. The findings on results and impacts demonstrate that the LIFE instrument has played a key role in respect to both designation of Natura 2000 sites as well as their protection and management. The majority of the LIFE projects would not have taken place without LIFE funding or would have been implemented at a lower scale or at a later stage. LIFE has thus contributed significantly to addressing this core element in the Biodiversity Action Plan. On the other hand, biodiversity is in many respects still in decline in Europe and the LIFE projects have thus not reversed this trend. However, the LIFE instrument must be seen as one among a number measures required to fulfil the overall objective of halting the loss of biodiversity in the EU. It is assessed that without LIFE, the decline in biodiversity would have been more severe, and in that respect LIFE has made a contribution towards reaching the objective. The Natura 2000 areas affected by LIFE projects experienced a significant improvement due to the high success rate of the project. However, areas not involved in LIFE projects only benefitted indirectly to a limited extent.

• There has been a limited focus on disseminating the results. While a number of high quality publications on LIFE projects and lessons learnt have been issued during the evaluation period and the LIFE web-site was also regularly updated, there was less pro-active effort to share knowledge, e.g. in the form of conferences, seminars, training events at regional or EU level.




12

.

• LIFE addressed Natura 2000 in a narrow implementation-oriented way with many similar initiatives and parallel projects in different geographical locations. This had limited European added value in the sense that the learning potential from these projects was less than if projects to a larger extent had focused on testing and demonstrating new approaches. On the other hand, an increased focus on demonstration / new approaches would probably have been more risky and have had a pay-off in terms of reduced effectiveness at project level.

• Through a strong focus on feasibility in project selection and complex application procedures, LIFE built the capacities of a relatively limited group of beneficiaries, who became familiar with LIFE scheme and have implemented a number of projects over the years.

• During the evaluation period, LIFE has had a narrow focus on Natura 2000 implementation, while biodiversity policy has broadened considerably.

LIFE Environment Among the beneficiaries and stakeholders in general there is a general appreciation of the funding opportunities provided by LIFE Environment, which, to many, represent the sole option for external funding of the phase between technology innovation and full-scale implementation. Beneficiaries and Member States also appreciate the flexibility and openness of LIFE Environment in responding to local needs and requirements. As mentioned before, many projects are very local and their utility is correspondingly local.

In terms of utility at the European level, the projects by and large have a link to one or more pieces of EU or national legislation and/or policies, and their objectives and scopes are in line with that legislation. However, the open programming strategy not specifically focused on particular environmental priorities means that relevant environmental issues are addressed by the projects but these are not necessarily the most important problems in the Community or in the Member State. Even within the nine themes or sub-issues within a given theme doubt can be raised as to whether the composition of the project portfolio is adequately balanced against the environmental issues of concern within that area. An example is the Natural Resources & Waste theme in which some of the large categories of waste in Europe and the environmental issues related to the management of them are only addressed by few LIFE projects.

Furthermore, the broad coverage in terms of different themes and sub-themes in many different countries implies that a significant risk of "dilution" is present. I.e. that the total project portfolio becomes so scattered/fragmented that LIFE Environment with its relatively limited funds is not really able 'to make a difference' within the specific sectors or themes covered. This aspect and the lack of focusing of the LIFE projects has consistently been mentioned during interviews with policy unit staff and national focal points as a limitation in the possibilities of the programme to support implementation and development of EU environmental legislation and policies at a larger scale.

LIFE Third Countries The point made about LIFE being a demand-driven or re-active mechanism is made concerning LIFE Environment above is also relevant in relation to the Third Countries component. It was an instrument to show to the beneficiary how they could approach and tackle environmental problems in an effective way and how to involve stakeholders. The value of LIFE Third Countries was that it is a reactive mechanism, which addressed the local needs of beneficiaries, whilst not necessarily addressing the most important environmental priorities of the country or region. For many beneficiaries, it was a unique opportunity without which they would not have implemented their




13

.

projects. For the EC Delegations, LIFE Third Countries was a complementary tool: open, flexible and diverse in terms of themes and actors.

2.7 Evaluation question 7: To what extent are the positive results and impacts likely to last after the termination of the projects?

Summary conclusion:

The sustainability of Nature projects is high with a high number of projects sustaining positive results after termination of the projects. The sustainability of Environment projects is medium to high with the majority of projects sustaining positive results after termination of the projects. For TCY projects, sustainability is low.

LIFE Nature The level of sustainability is assessed as high. Some projects requiring less intensive follow-up and recurrent funding were by nature more sustainable than others. Among the key factors explaining the high level of sustainability are well-designed projects, attention to building the necessary management and monitoring capacity to continue activities after project completion and ability of the projects to secure recurrent funding after project completion.

The challenges to ensure project sustainability were more significant in the cases where the projects involve private land as opposed to public land already designated for conservation. In these cases, a number of projects have been successful in preparing the ground for continued management through agri-environmental measures under the Guarantee section of the European Agricultural Guidance and Guarantee Fund, which also indicates a high level of consistency between these types of funding. The findings indicate a low level of knowledge about different sources of financing (other than LIFE) for nature conservation / Natura 2000 implementation among key stakeholders. Further initiatives on the part of the Commission (through LIFE and/or other mechanisms) could potentially increase the cross-fertilisation between LIFE and other funding mechanisms and thus increase sustainability.

Three interrelated key threats to sustainability have been identified:

• The project manager often leaves the project immediately after termination taking with him/her a large bulk of knowledge

• The projects are not always sufficiently anchored with the range of stakeholders which in one way or another have a role to play in relation to the continued conservation activities.

• Competing interests from agriculture and (to a lesser extent) forestry pose a significant threat to sustainability.

LIFE Environment The sustainability is in general assessed as reasonably good but with some variation between themes. On the average probably more than 50 % of the projects were continued, fully or partially, at the site or in the organisation where they were executed, especially if they were closely linked to




14

.

requirements or obligations in existing EU legislation. Considering the innovative nature of many projects, this is regarded as a satisfactory level of sustainability. Sustainability of projects in private companies was linked to technical feasibility and economic viability while for authorities and other public entities it depended on the political will and/or public finances.

LIFE Third Countries According to the EC Delegations and the monitors, one of the biggest gaps in the LIFE Third Countries component is the lack of sustainability of the projects. On the basis of interview data, it is assessed that sustainability was low because of the low operational capacity of the administration at the central level and the inadequate institutional framework in the countries. Some countries, in particular the EU candidate countries, saw significant improvements in term of institutional capacity building during the evaluation period and sustainability of the projects in these countries was consequently higher. In the other countries, the LIFE Third Countries component often faced the problem of "isolated projects". A lot of good and successfully implemented pilot projects were considered to give momentum to the whole process. However, they were not able to generate long terms effects because of the lack of institutionalisation of the project; the low level of involvement of crucial public partners and the limited capitalisation and dissemination at government level.

The main factors contributing to generation of long term effects were ownership of the projects by the government (i.e. Ministry of Environment) and political will to put in place the necessary legislation to sustain the results of projects. Moreover, financial support via international or national funds was essential in order to ensure the implementation or replication of results.

2.8 Evaluation question 8: To what extent has the LIFE Programme been complementary/supportive/contradictory to other EU policies and programmes?

LIFE Nature and Environment The coherence of the LIFE Programme objectives and selection criteria with Community environmental policy was stronger for LIFE Nature than for LIFE Environment. While LIFE Nature can mainly be characterised as a European programme supporting Community-defined objectives (related to Habitats and Birds Directives), LIFE Environment has been implemented much more as a bottom-up programme aiming to motivate Member States / individual entities to initiate environmental improvements in line with EU environmental policy generally. The findings show a limited level of "strategic intent" on the part of DG Environment and the LIFE Unit to scope and target the LIFE Programme in relation to specific Community policies. The LIFE Regulations have been followed loyally but the potential room for manoeuvre to further narrow the scope, particularly in relation to LIFE Environment, was utilised only to a limited extent.

For LIFE Environment, the "European element" becomes clearer when analysing the extent to which the objectives and selection criteria emphasise European value added. The analysis shows a strong focus on innovation and demonstration at the European level, i.e. selecting projects which provide new solutions relevant to all Member States. A similar focus was not found in respect to LIFE Nature. Generally for LIFE Environment, stakeholders have commented that the funding is complementary to other funding mechanisms and a unique funding opportunity for pilot testing of new technologies. In the new Member States, stakeholders have indicated that the funding opportunities through LIFE have received less attention due to many other funds being channelled into the environment sector.




15

.

For LIFE Nature, there is a level of contradiction between the Common Agricultural Policy and nature conservation policy, which also affects the LIFE Programme. On the other hand, LIFE has proven to be a good front-runner for agri-environmental schemes implemented under the Guarantee section of the European Agricultural Guidance and Guarantee Fund. In general, the knowledge among LIFE beneficiaries of other sources of financing for nature conservation could be improved potentially leading to further improvement of this positive supportive role of the LIFE Programme.

At the concrete project level, the LIFE Unit has put in place procedures to ensure that projects selected are not also receiving funding from other EU instruments. This is done by sending the list of projects awarded to relevant Commission services for review.

LIFE Third Countries At present there is no alternative to funding such as LIFE Third Countries and the countries now experience a vacuum in funding projects in the environmental sector. The new AIDCO instrument for the environment is Environment and Sustainable Management of Natural Resources including energy (ENRTP). It is also based on calls for proposals, but it is much more competitive (EUR 30 million per year worldwide) and the beneficiaries will need to be much more innovative if they are to gain access to this fund.

2.9 Evaluation question 9: To what extent do stakeholders accept the LIFE programme and LIFE projects

LIFE Nature The level of acceptability follows the pattern established for sustainability to some extent. Acceptability increases with the awareness raising efforts made during the project and is generally supported by continued dialogue and involvement of relevant stakeholders. LIFE Nature projects involving private land owners in general enjoy a higher degree of acceptability if the land owners become subject to direct and relevant funding aimed at their core activities, such as agri-environmental schemes for farmers.

Acceptability has also shown to be strengthened by the presence of LIFE Unit representatives or monitors with a 'foreign' background, as local communities may regard to international attention to their local area a credit to the values they posses and do efforts to preserve. Thus, in some cases by "flying the EU flag" the local acceptability may indeed be augmented. Examples of LIFE Nature projects where this has been expressed as a success factor are the wetlands projects from northern Italy.

LIFE Environment The results of the evaluation indicate that acceptability has not been a major concern in most LIFE Environment projects. Many of the projects are very specific, technological projects, which do not depend much on acceptability of a broad range of stakeholders. A number of stakeholders mention that LIFE is seen as a sort of quality stamp for the projects, because the requirements to project design and feasibility are so high and because of the communication value of having EU funding.

LIFE Third Countries According to the project managers, the monitors and the EC delegations, the bottom up approach has clearly been the best approach to better reflect the needs and problems of the beneficiaries. It allowed beneficiaries to develop projects on their own initiative and to create a feeling of ownership.




16

.

3 Recommendations Seen as a new generation of the LIFE Programme is now implemented in under the LIFE+ Regulation, it is relevant to provide the recommendations arising out of the analysis. There a number of differences between the set-up of the LIFE Programme in the period studied and the set-up under LIFE+. As the present evaluation did not include an analysis of LIFE+, these differences are not taken into account in the recommendations.

The recommendations are structured in three parts:

• Strategic management and programming

• Coordination, dissemination, partnerships

• Procedures for selection and monitoring

3.1 Strategic management and programming This evaluation has shown that the LIFE Unit has administered the LIFE Programme in accordance with the LIFE Regulations in an effective and efficient manner. Having a unit dedicated to programme management has contributed to streamlining and standardising the administration of the programme to the benefit of the Commission as well as beneficiaries. However, the disadvantage of this management model is that programme management becomes removed from the policy development functions. It therefore requires a special effort to ensure that the programme is in line with the strategic policy priorities of DG Environment as a whole. The following three recommendations provide suggestions for the future management of the programme at the strategic programming level.

1 Clarify the role and objectives of the LIFE Programme in relation to implementation of EU environmental policy and creation of European added value. At present the role of the LIFE Programme, in particular LIFE Environment, is unclear. There is a need for a management dialogue in DG Environment to specify the role more clearly and set targets and indicators for the implementation of the Programme, so that it is also possible to determine the degree to which the Programme is successful in achieving those targets. The LIFE Unit should be more active in seeking this dialogue and should increase coordination with policy units. This could for example be done by establishing a strategic steering committee for the LIFE Programme within DG Environment. Seen as the policy units are organised according to themes, it could also be considered to reorganise the LIFE Unit in thematic desks (rather than country desks, which is the current organisation).




17

.

There are some strategic options for the future development of both the Nature and the Envi-ronment components to consider:

In relation to LIFE Nature, the current budget is limited considering the estimated financing needs although current Natura 2000 cost estimates are uncertain and imprecise. The role of LIFE Nature needs to be considered within the framework of the debate on financing Natura 2000. Two strategic paths are outlined here - not exclusive to each other:

A) Continue with LIFE Nature as an implementation-oriented tool implementing relatively stan-dardised projects but with a more extensive outreach, i.e. more/larger areas of Natura 2000 in-volved. This entails an increased budget and possibly a lower co-financing rate. The Community and Member States co-share the responsibility to implement Natura 2000. This will require sig-nificant capacity building among potential beneficiaries in order to be able to absorb more LIFE resources.

B) Focus LIFE Nature more on new conservation methods and approaches, international pro-jects, projects outside Natura 2000 areas and combine this with activities to disseminate and share knowledge. Such a strategy could be accommodated within the current LIFE Nature budget. The Community acts as a source of inspiration providing strategic guidance to the Member States on how to implement Natura 2000 and the Biodiversity Action Plan.

Today, the requirements to the feasibility of the Nature projects are very high in order for the projects to be selected. This has contributed to a very high level of success of the projects. For both the above options, it is suggested to place less emphasis on the feasibility of individual ac-tions and more emphasis on the learning and capacity building benefits achieved.

In relation to LIFE Environment, the objectives are related to Community environmental pol-icy but the strategic link to policy is weak in the programming strategy. There is a need to clarify objectives and role - is LIFE Environment primarily a policy instrument for the Community or is it an instrument to motivate Member States / individual entities to initiate local processes and technologies? A more policy oriented programming strategy would benefit from a clearer focus-ing of environmental issues and policies to be addressed.

2 Regular monitoring and reporting on programme performance. At present the monitoring and reporting systems in place are completely focused at the project level. Monitoring and reporting of programme performance at the overall level is irregular in the form of the evaluations that have been completed (including the present one). This means that at a given point in time, there is very limited knowledge about programme performance against objectives. It is suggested that the Unit should develop systems and procedures for regular monitoring and reporting about programme performance on the basis of objectives and indicators (ref. recommendation 1). It is suggested that this be included as an integral part of the existing monitoring system applying a uniform set of indicators across all projects (Nature and Environment, respectively). The data could be integrated as a component in BUTLER but other complementary solutions may also be necessary (see recommendation 3). This could also include a regular annual status report on programme performance.

3 Develop support systems to improve the basis for strategic management. Provided more precise objectives and indicators are developed, the systems containing the project level information should be further developed so as to provide information on project objectives and achievements in line with the indicators for the Programme. For LIFE Nature, which already has




18

.

quite specific policy related objectives, there is a potential to develop a geographical database on the projects, which could be comparable to the overall database on Natura 2000, which DG Environment is in the process of developing. This would provide a unique opportunity to monitor the achievements and progress of the LIFE projects in relation to supporting the implementation of Natura 2000 - and to assess the need for targeting the LIFE Programme to more specific (geographic or thematic) areas.

3.2 Coordination, dissemination, partnerships The evaluation shows that coordination and knowledge sharing has generally worked effectively at local, regional and, most often, national level. Through the web-site and publications the LIFE Unit provided information about the LIFE projects and the projects were required to issue information as well. The evaluation also pointed to a need to focus more on dissemination and knowledge sharing to fully exploit the learning and replication potential of the LIFE projects and thereby enhancing the impact, sustainability and utility of the programme. The four recommendations below contain sug-gestions to this end.

4 Promote active knowledge sharing at European level. Events such as thematic conferences at EU or regional level should be used not only for knowledge sharing between beneficiaries but also to involve a broader range of stakeholders and thereby increase the general interest and awareness of the LIFE Programme with a view to attracting more project applications in the fu-ture. External partners such as industry organisations, research institutes and other interest groups could be invited. It is suggested to coordinate such events closely with the policy units in DG Environment and possibly arrange co-managed events dealing with policy aspects, financing opportunities and LIFE projects together, which could lead to synergy effects.

5 Target dissemination efforts to reach the right audience. The LIFE projects which succeeded in finding the appropriate communication channels to reach the target group, which could potentially benefit from the experience acquired, represent the most positive results with regard to dissemination and replication. In many projects, however, considerations with regard to key messages, target groups and communication channels were few. In the follow-up done by the LIFE Unit and the monitoring team, it is suggested to urge the beneficiaries to think along the lines of a communication strategy. This could be supplemented by requiring beneficiaries to submit a (1 page) communication strategy along with their interim report and to follow up on this strategy in the final report. It is also recommended that the LIFE Unit develop relations to some of the key actors at European level, e.g. industry organisations, and involve them as sparring partners in connection with conferences, thematic publications or other dissemination events.

6 Build capacities of potential LIFE beneficiaries. There is a need to broaden the client-base of the LIFE Programme to ensure a sufficient number and quality of applications. In particular for LIFE Nature, there is a need to bring on board less experienced organisations to extend the over-all base of organisations capable of instituting and managing nature conservation actions in the Member States. It is recommended that the LIFE Unit should be more pro-active in building re-lations through the national focal points and in arranging information meetings, trainings and seminars in project design and management - not only for existing beneficiaries but also for po-tential applicants.




19

.

7 Involve the Member States in dissemination and capacity building activities. The national focal points can potentially play a key role as access points to potential beneficiaries, in marketing the LIFE Programme and in providing guidance to applicants. This requires that the national focal points are engaged in the programme to a higher extent and that the Unit plays a more pro-active role in involving the national focal points by providing more information and by arranging joint activities, e.g. training in project preparation, thematic workshops, etc. This would also contribute to a more uniform approach among the Member States to guiding potential applicants, which would provide a more level playing field than is the case today.

3.3 Procedures for selection and monitoring During the implementation period, the LIFE Unit has continuously improved procedures with a view to achieving higher effectiveness and efficiency. The evaluation showed some areas where this proc-ess could be further continued as shown in the six recommendations below.

8 Improve application guidance and application forms. The data shows that many beneficiaries find the application procedure to be complex and slow. This may deter potential applicants from applying. The guidance to applicants should be improved e.g. by arranging training sessions in project preparation and management. Application forms should be simplified and digitalised to the extent possible, e.g. so that eligibility is checked in an inter-active way while the applicant is on-line registering the application. This should also help to minimise the time required for assessing the applications. Abolishing the procedure by which the Member States (the LIFE Committee) approve the final list of projects to be co-financed would significantly reduce the length of the selection period.

9 Outsource the receipt, eligibility check, selection and award phases of the selection procedure. During the evaluation period, the selection procedure was increasingly outsourced. The tasks involved in the selection procedure were abnormal compared to the standard monitoring tasks performed by the Unit which made it difficult to optimise efficiency. Outsourcing is assessed as the appropriate management decision and it is therefore suggested to continue along this line and outsource the entire process from receipt of applications to the award phase.

10 Further clarify criteria for selection and scoring system. While at the overall level, the framework of criteria and scoring systems guiding the selection process are assessed as clear, some ambiguities have been identified in the definition of criteria for scoring the projects during the award phase concerning mainly examples of: a) guides to applicants mentioned priorities which were not reflected in the criteria of the evaluation guide, b) lack of clarity in how the sub-criteria/questions mentioned under each criteria are to be weighted to arrive at a score for the criteria. The evaluation guide should be clarified with regard to the evaluation of the award criteria in order to ensure equal treatment of the applications. Such a clear specification of the evaluation task is also regarded as a precondition for successfully outsourcing this task.

11 Further clarify tasks of the monitoring team. The system where certain monitoring tasks are outsourced to an external team is effective. There has been an increasing centralisation during the evaluation period going from several monitoring teams to one team. This has clarified rela-tions, requirements and tasks. However, the dialogue is still to a large extent decentralised to the individual TDO/FDO and monitoring expert in connection with specific countries. It is sug-gested that a broader dialogue with the monitoring team is initiated. This could be done by an in-




20

.

tensified dialogue between the LIFE Unit management and the central monitoring team as well as focusing more on common training seminars, conferences, etc. with the monitoring experts and LIFE Unit staff. This dialogue should particularly seek to clarify tasks and quality criteria for the evaluation of project reports so as to avoid overlapping of tasks to the extent possible. Further, the procedures for project visits and risk assessment should be a focus area.

12 Apply a prioritised approach to individual project monitoring based on risk assessment. Visits to projects and face-to-face contact with beneficiaries were important parts of the effec-tive monitoring system in place. By taking a more strategic approach focusing on visits to higher-risk projects and replacing some visits with cross-project (thematic) conferences and workshops, the Unit could reach a larger number of projects and still achieve similar benefits for each project. A thematic approach also offers opportunities in relation to networking between projects and with national focal points.

13 Optimise document flow and clarify the role of the verifier in the Unit. During the LEAN workshop, it was found that up to five days of sending papers back and forth between staff can be saved – for every project and for every step in the monitoring process (reports, request for additional information, amendments etc.) by preparing the signataire when receiving the evalua-tion from the beneficiaries. Furthermore, the role of the verifier and the level of detail required of the verification were unclear. The system for verification should be clarified so that unneces-sary time used on verification can be eliminated.

Documents

Ex-Post Evaluation of Projects and Activities Financed ... · Part 6 presents the conclusions and recommendations arising from the evaluation's various components1, including: •