Future and Emerging Technologies · 2017-11-21 · Future and Emerging Technologies Giornata Nazionale di Lancio dei Bandi 2018-20 in Horizon 2020 Roma, 20 Novembre 2017 Nicola Ciulli

Proposal evaluation in FET

Future and Emerging Technologies Giornata Nazionale di Lancio dei Bandi 2018-20 in Horizon 2020

Roma, 20 Novembre 2017

Nicola Ciulli Head R&D @ Nextworks External expert @ EC, REA, EASME, Eureka, EIT

We are here in the FET map

FET-Open – Novel ideas for radically new technologies FET-Open Challenging Current Thinking

FET-Open Coordination and Support Actions

FET Innovation Launchpad

FET Proactive – Boosting emerging technologies FET Proactive: emerging paradigms and communities

Community building in Neuromorphic Computing Technologies

FET ERA-NET Cofund

FET Proactive – High Performance Computing International Cooperation on HPC

Extreme scale computing technologies, methods and algorithms for key applications and support to the HPC ecosystem

HPC PPP - Co-design of extreme scale HPC systems and applications

FET FLAGSHIPS – Tackling grand interdisciplinary science and technology challenges Preparatory Actions for new FET Flagships

ERA-NET Cofund for FET Flagships

FET Flagship on Quantum Technologies

Quantum Flagship – QuantERA II ERA-NET Cofund

FET Giornata Nazionale di Lancio dei Bandi 2018-20 H2020 2

FET-Open vs FET Proactive

FET-Open Challenging Current Thinking FET Proactive: emerging paradigms and communities

Goals • to lay the foundations for radically new future technologies of any kind from visionary interdisciplinary collaborations that dissolve the traditional boundaries between sciences and disciplines, including the social sciences and humanities

• to explore and consolidate a new technological direction in order to put it firmly on the map as a viable paradigm for future technology

• to foster the interdisciplinary communities that are able to drive this forward, extending from the participating consortia to a wider European pool of expertise

Focus • fully bottom-up, with GKs • specific sub-topics identified

Impact sought

• Scientific and technological contributions to the foundation of a new future technology

• Potential for future social or economic impact or market creation

• Building leading research and innovation capacity across Europe by involvement of key actors (excellent young researchers, ambitious high-tech SMEs or first-time participants)

• Scientific and technological contributions to the foundation and consolidation of a radically new future technology

• Potential for future returns in terms of societal or economic innovation or market creation

• Spreading excellence and building leading innovation capacity across Europe by involvement of key actors that can make a difference in the future, for example excellent young, researchers, ambitious high-tech SMEs or first-time participants to FET under Horizon 2020

• Build-up of a goal oriented interdisciplinary community (within and beyond the consortium)

• Emergence of an innovation ecosystem around a future technology in the theme addressed from outreach to and partnership with high potential actors in research and innovation, and from wider stakeholder/public engagement, with due consideration of aspects such as education, gender differences and long-term societal, ethical and legal implications


FET-Open vs FET Proactive: criteria

FET-Open Challenging Current Thinking FET Proactive: emerging paradigms and communities

Excellence (Threshold: 4/5, Weight: 60%)

• Clarity of the radical vision of a science-enabled technology and its differentiation from current paradigms.

• Novelty and ambition of the proposed science-to-technology breakthrough that addresses this vision.

• Range of and added value from interdisciplinarity for opening up new areas of research; non-incrementality of the research proposed.

• High-risk, plausibility and flexibility of the research approach.

• Clarity of long-term vision of a science-enabled technology.

• Concreteness and ambition of the proposed science-to-technology breakthrough that addresses this vision.

• Range and added value from interdisciplinarity, novelty and non-incrementality of the research proposed.

• High-risk of the research proposed and plausibility and flexibility of the approach.

Impact (Threshold: 3.5/5, Weight: 20%)

• The extent to which the outputs of the project would contribute to the expected impacts listed in the work programme under this topic.

• Effectiveness of measures and plans to disseminate and use the results (including management of IPR) and to communicate about the project to different target audiences.

• The extent to which the outputs of the project would contribute to the expected impacts mentioned in the work programme under the relevant FET topic.

• Effectiveness of measures and plans to disseminate and use the results (including management of IPR) and to communicate the project to different target audiences.

Implementation (Threshold: 3/5, Weight: 20%)

• Coherence and effectiveness of the research methodology and work plan to achieve project objectives and impacts, including adequate allocation of resources to tasks and partners.

• Role and complementarity of the participants and extent to which the consortium as a whole brings together the necessary expertise.

• Coherence and effectiveness of the work plan to achieve project objectives and impacts, including adequate allocation of resources to tasks and partners.

• Appropriateness of the research and innovation management structures and procedures.

• Role and complementarity of the participants and extent to which the consortium as a whole brings together the necessary expertise.


Focus on FET-Open

The competition in FET-Open is usually extremely high

It’s not enough to prepare a good proposal; it must be perfect under all aspects

i.e. C.1 Excellence is key, but far from enough

To do so,

it is very important to know what will be evaluated by which expert profile,

according to which specific implementation of the evaluation process


A glimpse of the evaluation process [1]

Usually the QC is performed on a CR draft

i.e. after the individual assessments have been finalized and a consensus have been achieved

In FET-Open, QC is performed more upstream…

on the IER, i.e. at the root of the assessment

QCs cannot challenge the individual opinions of the experts, but

How they are expressed in the comments

Their pertinence in each subcriterion

Their completeness according to the evaluation subcriteria

Their balance vs. the proposed scores

Even if the expert evaluators might be biased towards some specific aspects, the VC check brings everything back in focus


crossreading panel

A glimpse of the evaluation process [2]

E1 E2 E3 E4

E1 E2 E3 En

QC

• Comments can change • Scores can

change

• Comments and scores might change, • are endorsed • and proposals

ranked


Criteria & subcriteria

Excellence Impact Implementation

• Clarity and novelty of long-term

vision, and ambition and

concreteness of the targeted

breakthrough towards that vision.

• Novelty, non-incrementality and

plausibility of the proposed research

for achieving the targeted

breakthrough and its foundational

character.

• Appropriateness of the research

methodology and its suitability to

address high scientific and

technological risks.

• Range and added value from

interdisciplinarity, including

measures for exchange, cross-

fertilisation and synergy.

• Importance of the new technological

outcome with regards to its

transformational impact on

technology and/or society.

• Impact on future European scientific

and industrial leadership, notably

from involvement of new and high

potential actors.

• Quality of methods and measures for

achieving impact beyond the

research world and for establishing

European though leadership, as

perceived by industry and society.

• Soundness of the workplan and

clarity of intermediate targets.

• Relevance of expertise in the

consortium.

• Appropriate allocation and

justification of resources (person-

months, equipment).

Threshold: 4/5, Weight: 60% Threshold: 3.5/5, Weight: 20% Threshold: 3/5, Weight: 20%


Excellence [1]

Clarity and novelty of long-term vision, and ambition and concreteness of the targeted breakthrough towards that vision.

Is there a clearly defined novel long-term vision?

Is there a clear description of the proposed breakthrough research? Is the research well targeted towards an attainable goal or represents continuation of research along various parallel paths already being followed?

Relevance of the targeted breakthrough for the long-term vision – it should be outlined how the breakthrough would lead to the long-term vision.

Are the objectives well described in the proposal? Are they specific, measurable, achievable, relevant, and time-bound within the duration of the project?

Novelty, non-incrementality and plausibility of the proposed research for achieving the targeted breakthrough and its foundational character.

Is the analysis of the state-of-the-art comprehensive and complete?

Is the proposed research novel? – It should be more than continuation of research being already done or pursued currently.

Is the research proposed ambitious and far-reaching, not just another small incremental step along a path already followed?

Is the research plausible?

Does the research proposed have a potential to start a new line of investigation, which can be followed for years and can lead to completely new technologies


Excellence [2]

Appropriateness of the research methodology and its suitability to address high scientific and technological risks.

What is the relevance of the chosen methods for the achieving the proposal objectives?

Is the methodology "risk-friendly"? The details of risk analysis and mitigation plan should be addressed under Implementation.

Range and added value from interdisciplinarity, including measures for exchange, cross-fertilisation and synergy.

To what extent the main idea requires involvement of knowledge and methodology from different disciplines?

How "distant" are these disciplines involved? (There are disciplines, which traditionally work together, while others speak different languages and use different methodologies. The expectation is that putting together distant disciplines may bring qualitatively new solutions).

How are these different disciplines intertwined and how the ideas from different disciplines support the scientific breakthrough?


Impact

Importance of the new technological outcome with regards to its transformational impact on technology and/or society.

Is the long-term technological outcome clearly defined?

Is the transformational impact on technology, on society or on both convincingly argued?

It is not sufficient to discuss only the impact based on peer reviewed publications!

Impact on future European scientific and industrial leadership, notably from involvement of new and high potential actors.

Is the general impact on future European scientific and industrial leadership properly discussed?

Below are some examples of involvement of new and high potential actors.

• Are there researchers at the start of their career, especially principle investigators, who would keep up the research in a long run? or

• Are there SMEs in the consortium with sufficient budget, which would take up the results from the project and would eventually carry them over towards innovation and market realization? or

Quality of methods and measures for achieving impact beyond the research world and for establishing European through leadership, as perceived by industry and society.

Are the "standard" measures (scientific publications, website) properly described and planned?

Does the proposal go sufficiently far beyond the "standard" measures? For example, are there additional measures, based on new social media included?

Is it clearly explained how the dissemination measures would support achieving the expected impact?


Implementation

Soundness of the workplan and clarity of intermediate targets.

Is there a clear description of the tasks that will be performed, with specific deliverables allowing the verification of the execution of the tasks? Are there well defined work packages with relevant interdependencies among them? Is the time-dependence of the different tasks meaningful and convincing?

Are the intermediate targets (milestones) properly defined and timely? These should be major achievements serving as verification points if the work should continue according to the original plan or alternative paths should be followed.

Relevance of expertise in the consortium.

Is the expertise in the consortium broad enough in order to tackle all proposed tasks and is it of high enough quality?

Is the interdisciplinarity of the ideas in the proposal (if present) properly reflected in the composition of the consortium?

Is the consortium well-balanced, without redundancies?

Appropriate allocation and justification of resources (person-months, equipment).

Do the person-months allocated correspond to the proposed tasks to be performed?

Is the necessary equipment present or properly described and budgeted?

Is the overall budget comprehensive, well-balanced, and convincing?


Generic golden rules in proposal writing

Avoid using excess wording

It’s either illegal or counterproductive

Improve readability by editing tricks

The reading & discussion time is so short

Write the proposal with the evaluation sub-criteria clear in mind

Avoid writing what’s not needed, and avoid missing what’s needed

Do not overlook any of the subcriteria and template items

Have the proposal proof-read once ready

by someone not participating in the process,

with some experience of the evaluation criteria,

and of the technology involved


Nicola Ciulli Head of Research & Development External expert @ EC, REA, EASME, Eureka, EIT [email protected]


Documents

Future and Emerging Technologies · 2017-11-21 · Future and Emerging Technologies Giornata Nazionale di Lancio dei Bandi 2018-20 in Horizon 2020 Roma, 20 Novembre 2017 Nicola Ciulli