ERCEA WEB EVALUATION TOOL Expert USER'S GUIDE

ERCEA WEB EVALUATION TOOL

Expert USER'S GUIDE

European Research Council Executive Agency

Page 2 of 18

ERC Web Evaluation Tool V3.0-User's Guide

Index 1. Introduction......................................................................................... 3 2. Login to Web Evaluation Tool ........................................................... 3 3. Navigating in the main screen ............................................................ 3

3.1 Part 1-top area of the screen......................................................... 4 3.2 Part 2-filtering area........................................................................ 4 3.3 Part 3-statistics’table ..................................................................... 5 3.4 Part 4- download document .......................................................... 5 3.5 Part 5 -proposals ‘data .................................................................. 5

4. Actions on Proposal ........................................................................... 6 4.1 Downloading documents (arrow icon ): .................................... 6 4.2 Reviewing proposals (pencil icon ):............................................. 6 4.3 Signing and submitting proposal (lock icon ): .......................... 12 4.4 Declaring conflict of interest on proposal (Cross icon ) ............ 13 4.5 Suggesting Remote Referees ..................................................... 14

5. Reviews’ status ................................................................................ 18 6. Exporting Reviews............................................................................ 18

Page 3 of 18


1. Introduction This document is the User’s Guide of ERC Web Evaluation Tool version 3.0. It will explain to you how to use Web Evaluation Tool. This document is intended to the expert’s evaluators (Panel member or Panel chair or External reviewers) working in ERC calls (Consolidators, starting, Advanced and Synergy) in step1 and/or step2.

2. Login to Web Evaluation Tool To access Web Evaluation tool, you have to authenticate yourself in the ECAS "European Commission Authentication Service" You will find more details by clicking on this link: http://erc.europa.eu/space-erc-reviewers -click or Copy/paste the link in your browser, - follow the steps in the document. Note: this link is also incorporated in the invitation communicated to you by ERCEA.

3. Navigating in the main screen

After successful login to ERC Web Evaluation Tool, the main screen with the proposals assigned to you is displayed. The main screen is divided into five parts. Below, an example of the main screen of ERC Advanced grant-6th calls 2013.

http://erc.europa.eu/space-erc-reviewers

Page 4 of 18


3.1 Part 1-top area of the screen

User guide: if you click on the link, the user’s guide in PDF format is displayed Log out: if you click on the link, you will quit Web Eva Tool, you will be then redirected to a new login page. 3.2 Part 2-filtering area At the top of this section, the experts ‘name is displayed.

"Showing the evaluation" is a drop-down menu for selecting the call (starting, consolidators advanced, synergy or Poc -proof of concept) and the steps (step1 or step2). Below an example.

If you are working in a Synergy call, the steps are the following:

Page 5 of 18


3.3 Part 3-statistics’table The table of statistics gives you an overview of your reviews (number of reviews not started, started, completed, signed and submitted, Coi declared). Below, an example.

3.4 Part 4- download document

Download documents:

• All the panel’s proposals, only if you are a panel’s chair • All the assigned proposals to you • ERC Work Program • Guide for peer reviews

3.5 Part 5 -proposals ‘data In the middle of the screen, a table shows the proposals assigned to you, by default, the proposals are assigned by panel. The table can be sorted out in ascending or descending order by using any column as criterion. The column used for sorting the proposals will have its header highlighted. Below, an example of proposals 'table of ERC Advanced grant

Page 6 of 18


4. Actions on Proposal The last column in the proposals’ table of the main screen shows the actions that you might Execute on each proposal.

4.1 Downloading documents (arrow icon ): If you click on this icon, the application opens a new browser window to download a proposal’s document or the whole proposal.

4.2 Reviewing proposals (pencil icon ): Note: If you have not yet signed your AOT’s and you click this icon, you will see this message and you cannot access the proposals.

This section is described for 2013 evaluations only. If you click on the icon, you will access the proposals' review page.

Page 7 of 18


Below an example of review 'screen of ERC Advanced grant 6th call 2013.

The content of this page is different for 2013 evaluations in comparison with the previous evaluations. The main differences found in 2013 evaluations page are: The score is in descendent order, the content of the criterion 1 is new and the criterion 2 has new layout. The page is divided into 4 sections: 4.2.1. Proposal summary This section consists of the proposal summary, the review status and the reminder message. The availability of the actions on this page depends on the review status. Here is the mapping between the available actions and the status:

Page 8 of 18


Review Status/ Available actions

Not started Started Signed and submitted

Deadline passed

Edit reviews X X

Suggest Remote Referee

X X X(*) X(*)

Sign and submit reviews

X X

(*) Except for Synergy, this action is only available in step1. When the panel is finalized this action is not available. Below is the screenshot of the proposal summary with "Not started" status and the available Reminder to regularly save the work.

4.2.2. Proposal reviews This section contains evaluation criteria and expert's reviews. There are 2 criteria for 2013 evaluations. Each criteria contains an overall score and sub criteria.

• Criteria1- Overall score Score radio buttons are shown in descendant order (4.0 to 1.0 with 0.5 intervals).

Page 9 of 18


For each criterion, you should select only one score from the table. If you select a score between 4.0 and 2.0, a red message "above threshold" is displayed on the left side of the selected radio button (below, an example).

If you select a score between 2.0 and 1.0, a red message "below threshold" is displayed on the left side of the selected radio button. Below an example.

Normally the reviews in the sub criteria should be coherent with the overall score of the criteria. If the reviews are very negative then the overall score should be below threshold and vice versa.

• Criteria 1 - Sub criteria Each sub criteria of criterion 1 contains a set of questions that your reviews should answer. By default they are displayed, but it is possible to hide them by clicking on "Hide full criteria descriptions" link. Below an example of sub criteria of criterion 1.

Page 10 of 18


4.2.3 Review’s comments You can write your reviews in the textboxes (white boxes). The length of your reviews is shown below each textbox. This length cannot be lower than the minimum characters and cannot be higher than the maximum characters defined on the top of the textbox. If you do not respect the minimum length, you are allowed to save your reviews, but not to submit them. If you do not respect the maximum length then your reviews will be truncated. Below an example.

4.2.4 Criteria 2- Over all score &Sub criteria Sub criteria of criterion 2 are presented in table where you can express if you "Fully agree", "Agree partially", "Disagree partially", "Strongly disagree" with each sub criteria. You should select one ratio button per sub criteria. Otherwise, your review is considered to be incomplete.

Page 11 of 18


Below an example of criterion 2 for Advanced Grant 2013, 6th call in step 1,

4.2.5 Save and Submit reviews At the bottom of the review 'screen, there are three buttons:

Save – Save the review and maintains the window open. Save & Exit - Save the review and close the window.

Page 12 of 18


If you click on “save” button and the review is completed (all the comments are entered With minimum of 50 characters' and the scores selected), the "sign and submit review" Window is displayed.

If you click on "yes, sign and submit" button, the review will be signed and submitted. If you click on “save” button and the review is not completed, the application shows you a warning message. Cancel – cancels all changes and closes the window.

4.3 Signing and submitting proposal (lock icon ): Once you have sign and submit the review, the review can be processed for the panel meeting. The” lock” icon in the column “action” in the main screen is inactive and no further changes will be allowed.

"Save" and "Save & Exit" buttons are automatically disabled in the review ‘screen.

Page 13 of 18


4.4 Declaring conflict of interest on proposal (Cross icon ) This icon allows you to declare a potential Conflict of Interest (CoI) with a specific proposal, if the review is 'signed and submitted ' you can declare a Coi but before the review deadline is passed. These are the steps to follow in order to declare a CoI:

• select the proposal in the main screen and click button in “Actions” column ,

• The application will display the following window,

Page 14 of 18


• Select the "reason" in the drop down tag, this field is mandatory and the application

will not allow declaring a CoI unless a value is provided, • Write a comment on the reason using the text area tagged as "Specify the reason",

• Its optional, but mandatory if the reason you have selected is "6 Other",

• Click on "Declare CoI ", • A warning message is displayed

• If you click on “OK “button, a notification e-mail is automatically sent to the Panel coordinator.

How will the system behave after you have declared a CoI? The proposal with "CoI declared" has an orange background, and the Cross icon is disabled.

The only action available on a proposal with a "CoI declared" status is "Download proposal".

Page 15 of 18


4.5 Suggesting Remote Referees

You have the possibility to suggest, during the remote evaluation in step 1, new remote referees by searching in ERC database, or by adding new ones.

At the bottom of the review 'screen, just above the “save" button), you have the button "add" to suggest new remote referee.

Below an example of the “Suggesting Remote Referees” screen.

Note: if you are working on the Synergy call, you can suggest a new RR in both steps "1A" & "1B". If the review is closed (save and save & exit are disabled), you can suggest the remote referee's names during the panels meeting. The maximum number of suggested remote is 5, and the minimum number is zero, you can "Sign and submit" a proposal without any suggested RR.

Page 16 of 18


Below and example of suggested RR.

These are the steps to follow to search for Remote referees in the ERC Database.

• Click on 'add' button1 in the bottom of the review screen, • The 'add a remote referee' window opens, see example below, • Enters a last name or the Keywords and click on search , • The application shows the result , • Tick the button to select the expert and click 'Add' button,

• The RR is added in the list at the bottom of the review screen.

Page 17 of 18


These are the steps to follow to add new Remote referees

• Encode, at any order, the last name, the first name the Email and the institution in the 'add a remote referee' window, see example,

• Click on 'Add new' button,

• The result(s) are displayed at the bottom of the review screen; the “Add” button is inactive.

You can remove the RR from the list by clicking on in the “actions” column.

Page 18 of 18


5. Reviews’ status There are six possible statuses for each review:

Not a single comment or score has been introduced in the review

The review has started but scores and/or comments are missing

Review is ready to be finalised

Review finished and blocked and field in white colour

The deadline for sign and submit the proposal has been passed. The proposal cannot be submitted

The Coi has been declared by the expert on the proposal

6. Exporting Reviews At the bottom of the main screen, you can download and save the expert's reviews in RTF and PDF format.

Documents

ERCEA WEB EVALUATION TOOL Expert USER'S GUIDE