TEMPLATE FOR

FP6 PROJECT REVIEWS

This template can be downloaded from the Internet at ftp://ftp.cordis.europa.eu/pub/fp6/docs/template_review_report_1105.doc

xIndividual reportConsolidated report

Thematic Priority/Activity

Instrument type

Project no and acronym

Project full title

Project start date

Project duration

Project coordinator name and organisation

Period covered by report from/toDate of (review) meeting

Name(s) of reviewer(s)

Name of reviewer drafting the report

Sustainable Energy

Integrated Project

502687 NEEDS

New Energy Externalities Development for Sustainability

1 September 2004

48 months

Dr Andrea Ricci, ISIS Institute of Studies for the Integration of Systems

13-24 months

9-10 November 2006

Drda-Kuehn, Casey, Lucas

Nigel Lucas

1

Introduction

This template provides the structure for the review report that needs to be prepared after the project review. Alternatively, the template can be found in Word format at http://www.cordis.lu/fp6/project-management.htm and be completed off-line. In completing Sections 2-8 of the report, independent reviewers should keep in mind that, in case they feel that they do not have the competence or the information to answer a question, they do not need to tick any of the boxes ‘Yes’, ‘Partially’, ‘No’ for that question, but they must complete the ‘Comments’ box.

If several reviewers are involved, it is preferable that a consolidated report be prepared by one reviewer chosen as ‘rapporteur’.

The reporting requirements for FP6 projects are described in detail in the “Guidance notes on Project reporting in FP6” (downloadable from http://cordis.europa.eu/fp6/project-management.htm and http://cordis.europa.eu/fp6/find-doc-management.htm#reporting )

Questions to be answered by the reviewer(s)

2

1. OVERALL ASSESSMENT

a. Executive summaryPlease follow the order of the individual sections of this report

Comments:

The project is important in establishing an integrated approach in dealing with energy externalities at a national, European and in some cases global level. It is working to a well-structured work plan and has plentiful resources. The work plan for the coming 18 months is sound and the consortium is well managed – technically and administratively.

It has progressed well since the last annual review. The final tests will be whether the project succeeds in melding together the multiple techniques to produce results that are not otherwise available and that are novel and useful. It must also engage policy makers, persuade researchers in the field that it is the reference for all future work and establish itself as the global leader in the field.

Such impact is by no means guaranteed, but is within reach. The key to success lies in integration and in dissemination/communication. Integration is foreseen as an important component of the project and there is a research stream dedicated to it. Technically the integration is proceeding well; templates have been developed to link the models and these appear to work. There is also a good spirit of common and constructive endeavour among the project teams. The main risk is that the teams will not be able to develop a common vision of the final product and will therefore not be able or be prepared to make the compromises within their own work that will be necessary for the overall success. This should be an important concern over the next year. A draft of the final tool; the so-called “structured protocol; should be available by then as a guide to all concerned.

The dissemination and communication has improved since last year. There is now some good engagement with policy makers and a better sense of purpose in Stream 3b. The website functions well as a project management tool, but needs to be improved for wider dissemination. The extensive dissemination potential of this project is not adequately reflected in the work plan, yet. Dissemination activities as well as the creation of a distinctive exploitation plan need a more strategic approach and additional efforts but can still catch up if seriously treated.

The African partners have not contributed adequately because of lacking financial transfers. However, negative effects are not expected yet by the responsible RS coordinator.

The project has the potential to become a “good to excellent” project. To achieve this is needs to demonstrate useful and novel results, to develop a clear and practical vision of the final output and to improve communication and dissemination.

x

3

Good to excellent project (The project has fully achieved its

objectives and technical goals for the period and has even exceeded expectations)

Acceptable project (The project has achieved most of its objectives and technical goals for the period with relatively minor deviations)

Unsatisfactory project (The project has failed to achieve critical objectives and/or is not at all on schedule)

4

b. Recommendations

Coherent vision: It is recommended that the project develop within the next year through a transparent and cooperative process a shared vision of the final tool and the acceptance of the need to contribute effectively to that shared vision. This should be available at the latest by the next review period.

Global leadership: The project should develop a vision statement along the lines that it “will bring to the attention of the Global community of politicians, policy makers, researchers and interested individuals, the importance of and the important advances and findings in energy externalities research. In fulfilling this objective, it will provide Global leadership in this research and analysis and will welcome and support the participation in these activities of researchers from across the world.” Within this context, the web site should transform itself into a welcoming host, which recognises the needs of not only researchers and high-level users, but also the rights of individuals, schools, parents, and even children to have access to the results of research. This website should employ the most advanced technologies and design approaches to deliver its content. Indeed, it will also become a global leader in bringing research results to society, and will support this imperative as it has already been expressed in the FP7 Energy Work Programme.

Dissemination: The project needs substantial improving of the dissemination as well as the preparation of the exploitation of the achieved results as well as the future results. There needs to be a more strategic approach than at present; professional support might be desirable. Further suggestions are provided later in the review.

Handling of late results for dissemination activities: A detailed strategic dissemination plan which describes interdependencies between Research Streams and offers ideas and an operational concept for generating dissemination activities before the end of the project is necessary.

Financial transfer to African partners: The coordinator should investigate mechanisms within the partnership to ensure that adequate funds are assigned to the African partners in order to get the work going. This is within the legal rights of the coordinator but should be discussed with the other partners for reasons of a trustworthy partnership.

A longer term strategic plan: It would useful, despite being a 4 year project, to have and to work within a longer term strategic framework which foresees (however tentatively) raising additional funding and future research needs with the expectation of continuing this undoubtedly important research. In this context, the researchers (and particularly their home research institutions) should consider an immediate, explicit commitment to continuing this group-research effort and in the medium term stabilise the financial and administrative base of this important area of research. The formation of a European Technology Partnership might be a useful step in this process.

5

2. OBJECTIVES

a. Have the objectives for the period been achieved?

x Yes Partially

No

Comments:

6

With some exceptions, the objectives have been achieved.

Integration seems on track – the technical issues of making the models work together seem to have been solved. The management of the integration process is effective. The route to producing a structured protocol to further users is less clear and needs thought.

The LCA work is important to capture externalities of capital-intensive technologies; it is proceeding without evident problems. Innovative aspects are centralization of data and generation of matrices for future years.

The improved methodology and measurement of externalities is also on track. The atmospheric modeling has advanced through hemispheric models. Source-receptor matrices have been developed in 44 areas. Exposure-response functions have been extended to radiation and heavy metals. The project is examining the impact of PM10 components. There are innovations on indicators and valuation of bio-diversity and advance in the valuation of mortality. For soil and water the project has identified through a screening process the most important impacts through soil and water of energy cycles. The key now is to focus and ensure that a useful tool is created within two years – must avoid the temptations of the search for perfection.

Work on externalities of transport is proceeding well. There are nice studies on oil and gas flows. Will make own assessments of hydrogen penetration.

The Extension of geographical coverage has been delayed apparently because MEDA partners cannot work with the long payment delays of the Commission.

National models have been developed, tested, calibrated and training initiated. Pan-European model developed based on the national models. Scenario-definition is close to completion. Objectives of year 2 have been met. Main problems arose from inadequacy of published data sources; substantial effort to refine data was needed.

The Energy Technology Roadmap and Stakeholder Perspectives Stream is moving more slowly than other streams. It is an approach to decision-making that complements the optimisation of economic costs.

Transferability and generalisation is an important practical issue. The stream has only just begun and there is little to report; there are no evident difficulties at this stage. It is a concern that there are no deliverables up to Month 42. This should be revised. Apparently, some technical papers are foreseen before then.

The Plan for Dissemination and Use of Knowledge is greatly improved in terms of engagement with the policy community, but more is needed. It needs more comprehensive dissemination of technical results especially peer-reviewed; it needs to aim at making the output of NEEDS the basis for the bulk of future externality work in the EU. Engagement with policy community is one part of that, but need also to think about the physical deliverable to users. What will it be? How will they be influenced?

7

b. Are the overall objectives (i) still relevant and (ii) still achievable within the time and resources available to the project?

(i) x

Yes Partially No

(ii) x

Yes Partially No

Comments:The results still seem achievable within the available time and resources, but it will need a degree of focus. Researchers need to focus on the need to have a working, integrated system up and running and delivering useful results. They should not get bogged down for the search for perfection where it is not possible.

c. Do you recommend changes in objectives in order to keep up with the current state-of-the-art?

Yes Partially

x No

Comments:The teams are aware of the state-of-the art and incorporating such advances as are required.

The dimension of the dissemination of the project results should be more strongly considered in every single research stream. An overall consideration by the coordinator is difficult as the research is of high complexity and needs overview knowledge in many relevant energy fields. However, this could be managed by a close look at the end of every single work package at dissemination matters.

8

3. WORKPLAN AND RESOURCES

a. Has the project as a whole been making satisfactory progress in relation to the Description of Work (Annex I to the contract)?

x

Yes Partially No

Comments:The project is broadly on track, well managed and likely to deliver. There needs to be even more focus on delivery and dissemination even to the prejudice of interesting scientific alleys.

Nature and timing of interfaces between streams has been difficult to achieve, but is largely managed. There are specific problems with matching technical progress locally to progress globally, but this has been achieved by defining exogenous scenarios for technical progress.

b. Has each work package (WP) been making satisfactory progress in relation to the Description of Work (Annex I to the contract)?

x

Yes Partially No

Comments:

9

RS 1a + 1 b are acceptable in the implementation, the reports available, and the RS show good and interesting results.

RS 1c The research stream is in good progress and the work packages are sufficiently reported. The delay of year 1 has been caught up.

RS 1d: The RS has started in 03/2006; the 3 WPs are in progress. The African partners have not contributed adequately because of lacking financial transfers. However, negative effects are not expected yet by the RS coordinator.

RS 2a: The objectives of the RS have been met.

RS 2b: Research Stream 2b appears to be on a different schedule from other Streams; there are reasons for this, but the incorporation of this work in the final product requires thought. Only 1/3 of work has been done so far, most of work will be done in year 4; social scientists and NGOs are included. The following points may be considered:

- Given the awareness of methodological problems in this RS, the very small database is not really representative on a European level, although this is methodologically acceptable.

- The connection with 3b is not given yet and should be developed strategically.

- The results of this RS are foreseen for month 48. This will be too late for being integrated in other results and for dissemination activities. This should be rethought in close cooperation with the responsibilities in dissemination.

- Indicators will be available until month 39.

The collected stakeholder database is concentrated on one topic only: Electricity. This limits the outcome. The RS should consider how representation can be secured for other fields.

RS 3a: No great problems; in progress.

S Integration: Communication and political integration: a strategic plan is missing for communication, a structural protocol is needed.

S 3b Dissemination: The recommendations of the first annual review have been considered. This must be appreciated. More can and should be achieved and suggestions are given later in this review.

c. Have planned milestones and deliverables been achieved for the reporting period?

x Yes Partially

No

Comments:With some small delays, all milestones have been passed and deliverables met.

10

d. Have resources been deployed as foreseen in Annex I, overall and for each participant?

x

Yes Partially No

Comments:As far as this can be seen, the resources have been deployed as foreseen, other than for the shift in Research Stream3 that reflected comments from the previous evaluation. The comparison between planned expenditure and actual was made available to the reviewers after the evaluation meeting.

e. Have costs incurred, i.e., personnel costs and other major cost items, been 1) necessary for the implementation of the project and 2) economic. Note that both aspects 1) and 2) have to be covered in the answer.

x

Yes Partially No

Comments:The costs have been incurred for work that was necessary to the project.

Generally, and without prejudice to this project, we note that the efficiency of the use of resources cannot be accurately assessed under the current financial reporting system required by the EC. To measure the efficiency of work it would be helpful that costs involved in the individual WPs and deliverables are reported separately.

11

4. WORK PLANNED FOR THE NEXT 18-MONTH PERIOD (NoEs and IPs only)

Is the proposed update to the Implementation Plan (IPs) for the next 18-month period satisfactory?

a. from a scientific/technical point of view?

x Yes Partially

No

Comments:

12

The work plan is largely adequate.

Understandable there is internal debate on scenarios and methodological assumptions. It is important to sustain the debate, but also to ensure that it does not hinder successful completion of results. Large degree of practical compromise will be needed.

Stream 1a on LCA has been expanded to include Ocean energy, but is still on track and proposed plan for the next period and is appropriate to achieving remaining objectives. Final outputs are defines including a book.

Work Packages 1 to 5 in Stream 1b on externalities are winding up in this period. The work programme to achieve this has been defined. Plan for WPs 6 to 8 is acceptable. Web-based tool included in output.

Stream 1c on externalities of transport of energy is largely on track; initial delays recouped. WP1 will focus on representative transport routes. Largely completed by Month 42.

Work plan for Stream 1d is appropriate, but needs a solution to the financing problem for Mediterranean partner countries. Cooperation with other research Streams is appropriate.

Stream 2a will be very active over the period: some reports envisaged for third year will now be brought forward to be available for third year review. Work programme envisaged further testing, tuning and refining and the analysis of defined scenarios ant national and European levels. Objectives for the period should be achievable under the work proposed. Modelling suite is very adaptable to a range of problems in energy policy.

Work programme for Stream 2b is internally consistent, but scope has increased somewhat and may mean that deliverables will not be available on the same schedule as other streams. To some extent, this reflects the nature of the WP as the final step on the decision-making route. Output not absolutely on critical path to operational modelling tool and protocol, but would be important enhancement. There is a need to think how the approach will be incorporated into the structured protocol. The methodological tools will probably be available in time; it is the experience from application that may be more difficult to capture.

There is no deliverable from Stream 3a in the next year, but a series of technical reports are planned to ensure proper documentation. Work programme is well directed at the key topics.

Stream 3b envisages 3 stakeholder meetings of different types; 3 newsletters and improvements to the website; topics and locations for the events are appropriate. Later a third policy workshop is foreseen to discuss inter alia the structured protocol. Other aspects of this stream need strengthening.

Integration. An outline of the structured protocol will be available from Month 36 – appropriate. The delivery of a coherent tool is a key performance criterion and it is important that all partners are aware well in advance of the nature of this output and their contribution to it. Final joint conference with CASES envisaged. Not entirely clear, that this is such a good idea as may obscure the objectives of each project. Four-month extension requested. No objection.

b. from a management point of view including use of resources?

x Yes Partially

No

Comments:

13

Management is exercised through a hierarchical structure, from the IP Coordinator through a Project Management Committee comprising the Stream leaders, subsequently through the Stream Management Committees incorporating the contractors in each stream.

This seems to work well; all team-members seem to have a good idea of what is expected of them and of the overall shape and character of the project. Deliverables has mainly been on schedule and the content has been mainly acceptable. Integration of the various work-packages and the models has been technically successful under the guidance of the project manager. These arrangements are continued in the next period and no problem is envisaged.

A detailed financial plan for months 24 – 42 was presented, although some forms (A2 and A3) were missing.

c. concerning non-scientific activities (dissemination, exploitation, training, science-society issues, further integration etc)?

Yes x

Partially No

Comments:Dissemination has improved, but still needs a strategic approach. It requires to be situated within a completely new vision as to the Global importance and Global leadership potential of this project.

From this vision should be derived a strategy for dissemination 1) at a Global level and 2) moving from a position where the project tries to “sell” its wares to where there is an anticipative audience waiting for each report.

The website has improved, much richer in materials; but there is still much to do; newsletter is a step forward.

Some training undertaken, there is still not enough interaction with users and no clear strategy by which the work will become a future reference

14

5. CONSORTIUM PARTNERSHIP

a. Has the collaboration between the participants been effective?

X Yes Partially

No

Comments:Collaboration is good. All partners have overall view and recognise the role of their work in it. With some exceptions, work has not been impeded by late delivery of other streams.

Coordinator has been effective in promotion of consensus.

b. Have the partners contributed as planned to the project and tasks assigned to them?

x

Yes Partially No

Comments:Overall, yes.

c. Do you identify any conflicts or evidence of underperforming partners, lack of commitment or change of interest of any partners? Do you recommend any changes in responsibilities?

Yes Partially

x No

15

Comments:All partners appeared committed and are contributing as foreseen. The problems in dissemination are not a consequence of any lack of commitment, but of lack of experience.

6. MANAGEMENT

a. Has the scientific/technical management been performed as required?

x Yes Partially

No

Comments:The project management and coordination is good. The project manager has a good overview of the scientific issues and the members of the consortium have a clear understanding of their tasks. The production of deliverables has mainly been on schedule and the content has been mainly acceptable. Integration by the project coordinator of multiple tools appears successful: ExternE, MARKAL-EFOM; LCA; MCDA. There is a question over whether it is going to work – it is very complex and we will need to see, but so far, it looks on track.

Analytical overviews have been added as outputs that provide high-level review of the technical achievements; this was recommended in the first annual review and has proved useful.

b. Has the administrative and financial management been performed as required ((including proper handling of contractual matters, maintenance of the consortium agreement, intellectual property rights, technical collective responsibility, sub-contracting, competitive calls)?

x

Yes Partially No

Comments:There is rather little IPR; it is largely a public good. There have been no competitive calls.

Financial reporting not fully completed within allotted 45-day period.

52% of budget remains, which is appropriate for this stage in the project. Percentage utilisation by research stream accurately reflects the degree of completion of their work.

Some changes from original allocation of funds, but no deviations that are inexplicable or that cannot be managed.

c. Have (electronic) information and communication networks been established as required to support interactive working between the teams involved (if relevant)?

x

Yes Partially No

Comments:The project has put in place an excellent information and communication web-based management tool. Used regularly by stream-leaders for communication. It serves also as project archive and memory.

Financial software has been developed at project coordination level to synthesise financial information. Operates through the website.

d. Is the consortium interacting in a satisfactory manner with other related 5th and 6th Framework projects or other R&D national/international programmes (if relevant)?

Yes x

Partially No

Comments:According to recommendations of the 1st annual review, this was taken into consideration. However, it is not visible, what the real outcome of these activities will be. Interactions with related projects on European and national level should be integrated part of a dissemination strategy in order to target possible synergies and other kinds of cooperation. Here the work seems to be at the very beginning and NEEDS needs additional efforts.

Connections with existing (technology) platforms should be taken into consideration.

7. USE AND DISSEMINATION OF KNOWLEDGE

a. Does the project have significant use potential (if applicable)?

x Yes Partially

No

Comments:The dissemination and communication potential of the NEEDS project is very high and offers outstanding opportunities on internal (inside the consortium) as well as external (different target groups like stakeholders, NGOs, policy makers) level. It offers the seldom opportunity for starting communication from the very beginning and to creating and maintaining a community and therefore generating a unique opportunity to influence policy-making.

b. Is the Plan for the Use and Dissemination of Knowledge developing in a satisfactory manner?

Yes x

Partially No

Comments:The project has accepted recommendations of first review and dissemination to policy makers has been much improved. A plan for dissemination which identifies target groups, instruments for dissemination activities and a clear schedule is still lacking. Therefore, a strategic approach to dissemination is lacking as well. Much of the input into this strategy is evident in the thinking of the team though not in the publications.

A transfer description from one RS to the other from dissemination point of view is also lacking. It is not considered how the results of single Research Streams are communicated to other RSs (internal) and communicated to different target groups (external). There is a danger that the results stay inside the research community of the project, even only in the single RSs and do not go further. The proposal to organise some systematic internal training of 66 partners gets to a large part of the relevant research community and is to be encouraged.

c. Have the contractors disseminated project results and information as foreseen by the contract and the plan for dissemination and use of knowledge (publications, conferences…)?

Yes x

Partially No

Comments:The main performance criterion for this project is to produce results that would not be otherwise available and that are useful in policy support. Also to influence policy makers and analytical researchers to adopt the approach. These aims are only partially completed.

The visibility of the project is not proportional to the landmark function it could have according to the given resources in terms of knowledge, expertise and finances. The available NEEDS publication does not reflect the importance of this project, neither does the website. The NEEDS website covers only very basic information.

d. Are potential users and other stakeholders (outside the consortium) suitably involved (if applicable)?

Yes x

Partially No

Comments:

Progress has been made with the policy community, but there is more to be done to make the modelling-suite the tool of choice among analysts. Much effort has been put into communication with other projects; this is to be commended. A database of 700 users has been created. A book is envisaged from LCA stream. Stakeholders are involved in RS 2b but the given description of the research work already done raises the question if this involvement is relevant in any way as the number of involved stakeholders is too small for any scientific conclusion or even a reliable database.

A plan of engagement for users and peer-groups requires:

Next steps

draw up a dissemination plan within 2 months identify target groups, instruments, synergy possibilities and a time table relaunch the website as the most important instrument for dissemination

within 2 month: the interactivity is essential as well as portal functions and a search machine optimisation

organise the operational level for the implementation of dissemination activities within the next 4 months

get visible by implementing the dissemination plan with as much resources as possible

start preparing an exploitation plan at the end of project month 35 that it can be presented in the next annual review for sustainability reasons of project results

Develop dissemination expertise. There are three possibilities:

1. Send the responsible persons on short notice to an effective training course for disseminating research results. The training should concentrate on research dissemination, not on Public Relations in general. Or,2. Bring a professional as an accompanying expert into the project in order to get temporary advice and support for setting up a dissemination plan. Or,3. Engage professional help by outsourcing the dissemination to a professional company completely. This will cost some money but will bring quick effects to the project.

Financing of dissemination:

Options to increase funding might be considered:

1. Agreement between all partners that some of their budget (which can be very small compared with the big number of partners) should be allocated to dissemination. Every single partner will benefit from this solution in the end.

2. The coordinator should check in which budget part there are back flows possible which can be allocated to dissemination. This should be possible at mid term of a project.

3. According to information given by the coordinator the auditing costs will not be as high as calculated. It should be possible at the time being to calculate the auditing costs anew with the support of the single RS coordinators, so the money which will not be spent should be allocated at once to dissemination.

8. OTHER ISSUES

a. Have policy-related and/or regulatory issues been properly handled (if applicable)?

x Yes Partially

No

Comments:Policy issues are now being addressed

b. Have ethical issues been appropriately handled (if applicable)?

x Yes Partially

No

Comments:n.a.

c. Have safety issues been properly handled (if applicable)?

x Yes Partially

No

Comments:n.a.

d. Has progress on the Gender Action Plan been satisfactory (if applicable for this reporting period)?

? Yes Partially

No

Comments:No Gender Action Plan tabled

Name (s) of the reviewer(s): N J D Lucas

Date:

Karin Drda-Kühn

Date:

Thomas Casey

Date:

APPENDICES to the reviewers report (optional)

a. Status of project reports and deliverables (columns (a)-(h) to be pre-filled by the Project Officer; columns (i)-(m) to be completed by the reviewer)

Del no.(a)

Deliverable name

(b)

WP

no.(c)

Lead participant

(d)

Estimated indicative person-

months*)(e)

Nature(f)

Dissemination level(g)

Delivery date from DoW

(proj. month)(h)

Used indicative person-months

(i)

Actual/ Forecast delivery

date(j)

Status(k)

Resubmission date(l)

Remarks(m)

*) if available

b. Visibility Actions (to be completed by the Project Officer)

Mark which actions would be appropriate for follow-up by EC programme policy units:

Exploitation Strategy Seminar Contact the Innovation Relay Centres Promote / highlight as a success/case story

Flag this project for in case the programme looks for projects with certain characteristics:

high visibility/media attractive project; project with an impact on EU policies; project with a major role for women; project with a significant impact on health, safety, environment; project with ethical issues associated. substantial breakthrough character significant impact on employment significant participation from outside EU involvement of the top researchers in the field involvement of the top economic actors in the field