Changes in peer review v3

Office of the Vice President for Research

andDivision of Sponsored Programs

April 7, 2010

New Investigators and Early Stage Investigators

Enhanced Review Criteria Template-Based Critiques Scoring Order of Review Summary Statements

New Investigator (NI)- PD/PI who has not yet competed successfully for a substantial NIH research grant (for multiple PIs, all must meet requirement for NI status).

Early Stage Investigator (ESI) - PD/PI who qualifies as a New Investigator AND is within 10 years of completing their terminal research degree or within 10 years of completing their medical residency (or equivalent).

Generally only applies to R01 applications.

Overall Impact - Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved (think paradigm shifting).

New Core Criteria Order Significance Investigator(s) Innovation Approach Environment

ImpactOverall Impact is not a sixth review criterion and is not

necessarily the arithmetic mean of the scores for the scored review criteria. Overall Impact is the synthesis/integration of the five core review criteria that are scored individually along with the additional review criteria which are not scored separately.

SignificanceDoes the project address an important problem or critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

Provide concise, explicit and evaluative statements

Identify strengths and weaknesses for each criterion

Provide criterion scores1. Significance Limit text to ¼ page

Strengths Weaknesses

Scoring scale 1-9, whole numbers only Impact score is NOT average of criterion

scores Applications reviewed in order of

preliminary overall impact scores Applications with initial scores between

7-9 are unlikely to be discussed All applications receive criterion scores Criterion scores are not discussed at

review

Score

Descriptor

Additional Guidance on Strengths/Weaknesses

1 Exceptional Exceptionally strong with essentially no weaknesses

2 Outstanding Extremely strong with negligible weaknesses

3 Excellent Very strong with only some minor weaknesses

4 Very Good Strong but with numerous minor weaknesses

5 Good Strong but with at least one moderate weakness

6 Satisfactory Some strengths but also some moderate weaknesses

7 Fair Some strengths but with at least one major weakness

8 Marginal A few strengths and a few major weaknesses

9 Poor Very few strengths and numerous major weaknesses

Applications with best preliminary scores are reviewed first

NI/ESI R01s clustered at beginning of review

Other types of grants may be clustered if included in the same review meeting (R03, R15, R21, etc)

Final scores of discussed applications may differ from preliminary scores (happens >50% of the time)

Only 50-60% of applications at review meeting will be discussed

Reviewers can request that any other application also be discussed

The remaining applications will NOT be discussed (applications receive criterion scores only). These applications are considered to have low impact.

Assigned reviewers (typically 3 primary) define the range of scores for each specific application

At the time of the review, scores are provided by all present. Any reviewer who plans to score outside the range established by the assigned (primary) reviewers needs to declare their intention, which is based on their determination of overall impact.

Discussed applications receive an overall score from each eligible (non-conflicted) panel member. Scores are averaged to one decimal place and multiplied by 10. Hence, there are 81 possible priority scores that range from 10 to 90.

Percentiles are reported as whole numbers

Summary statement is short and focused Only discussed applications receive a

summary of the panel’s deliberation Applications not discussed receive only

criterion scores and primary reviewers’ critiques

Critiques: The Critiques below were prepared by the reviewers assigned to this application. These commentaries do not necessarily reflect the position of the authors at the close of the group discussion, nor the final majority opinion of the group, although reviewers are asked to amend their critiques if their position changed during the discussion. The resume and other initial sections of the summary statement are the authoritative representation of the final outcome of group discussion. If there is any discrepancy between the peer reviewers’ commentaries and the action statement for a given component, or the overall score, the action statement for the component, the overall critique, and the overall numerical score on the face page of this summary statement should be considered the most accurate representation of the final outcome of the group discussion.

Read RFA and explicitly follow directions Write to the review criteria New page limitations require that you

focus your writing limit jargon clearly define significance and overall impact articulate the big picture in lieu of minutiae