How to Review (…and Read…and Write) Scientific Papers

How to Review (…and Read…and Write) How to Review (…and Read…and Write) Scientific PapersScientific Papers

Peer ReviewPeer Review

BackgroundBackground

• Editor, Journal of Vascular and Interventional Editor, Journal of Vascular and Interventional Radiology (since January 2006)Radiology (since January 2006)

• Have served as a reviewer for JVIR, AJR Have served as a reviewer for JVIR, AJR (including editorial board), Journal of Clinical (including editorial board), Journal of Clinical Ultrasound, Journal of Interventional RadiologyUltrasound, Journal of Interventional Radiology

Reviewer’s RoleReviewer’s Role

• The reviewers are extremely important to any The reviewers are extremely important to any journal. journal.

• Arbiters of quality of submitted manuscriptsArbiters of quality of submitted manuscripts• Role twofoldRole twofold• Does the manuscript merit publication?Does the manuscript merit publication?• Advisory role: Provide constructive criticism (help make a Advisory role: Provide constructive criticism (help make a

manuscript better)manuscript better)

Provenzale and Stanley, AJR 2005

Does a manuscript merit publication?Does a manuscript merit publication?

• AcceptAccept• Rare decision, nearly every manuscript could use some workRare decision, nearly every manuscript could use some work

• Accept with RevisionAccept with Revision• If authors address queries and criticisms, will generally be acceptedIf authors address queries and criticisms, will generally be accepted

• [Conditional Acceptance] (Editor only)[Conditional Acceptance] (Editor only)• Will accept but authors MUST address certain key issuesWill accept but authors MUST address certain key issues

• Reject Request ResubmissionReject Request Resubmission• Not suitable now, and may not ever be, but could be worth a second look--Not suitable now, and may not ever be, but could be worth a second look--

undergoes repeat formal peer reviewundergoes repeat formal peer review

• RejectReject• Poor science, report without value, redundant or duplicate publication, “me too” Poor science, report without value, redundant or duplicate publication, “me too”

reports, clear bias, too esoteric even for lab rats, or good/great paper but wrong reports, clear bias, too esoteric even for lab rats, or good/great paper but wrong audienceaudience


• Caveat: the tips and suggestions I will Caveat: the tips and suggestions I will bring up are based on my own personal bring up are based on my own personal preferences and biases, and these may preferences and biases, and these may not be applicable to all journalsnot be applicable to all journals

Common Sense RulesCommon Sense Rules

• Be polite. Avoid insults, sarcasm, demeaning Be polite. Avoid insults, sarcasm, demeaning statements (the Editor will generally remove statements (the Editor will generally remove these anyway)these anyway)

• If there is something that the reviewer feels If there is something that the reviewer feels needs to be transmitted to the Editor but not the needs to be transmitted to the Editor but not the authors, there is generally a “comments to authors, there is generally a “comments to Editor” boxEditor” box

• Direct comments to “the authors” or even better Direct comments to “the authors” or even better “the manuscript” (I avoid “you” sentences)“the manuscript” (I avoid “you” sentences)


Systematic ApproachSystematic Approach

• To what category does the manuscript belong?To what category does the manuscript belong?• Clinical StudyClinical Study• Laboratory StudyLaboratory Study• Brief ReportBrief Report• Letter to the EditorLetter to the Editor• Other (special communication, standards, editorial, etc.)Other (special communication, standards, editorial, etc.)

• Is there potential for reviewer bias (positive or negative)?Is there potential for reviewer bias (positive or negative)?• Does the reviewer have sufficient expertise for this topic? Does the reviewer have sufficient expertise for this topic?

nb: to me this is important, but NOT criticalnb: to me this is important, but NOT critical• Does the reviewer have time to do the review?Does the reviewer have time to do the review?


How to “Read” a ManuscriptHow to “Read” a Manuscript

• No set right answer, depends on your styleNo set right answer, depends on your style• Many do a quick survey (e.g. read abstract, Many do a quick survey (e.g. read abstract,

skim remainder) to start to address the skim remainder) to start to address the following generaly questionsfollowing generaly questions• Why are the authors reporting this? What was their Why are the authors reporting this? What was their

intent?intent?• Is the topic of interest to readers of the journal?Is the topic of interest to readers of the journal?• Does the study try to answer important questions Does the study try to answer important questions

that have not been answered adequately?that have not been answered adequately?


How to Review A ManuscriptHow to Review A Manuscript

• As an Editor I prefer a structured, numbered approachAs an Editor I prefer a structured, numbered approach• Highlight very important points with asterisksHighlight very important points with asterisks• [Brief Summary][Brief Summary]• General comments General comments • Section by Section Review (Title, Abstract, Introduction, Section by Section Review (Title, Abstract, Introduction,

M&M, Results, Discussion/Conclusion, Tables, M&M, Results, Discussion/Conclusion, Tables, Figures/Illustrations/Graphs and Legends, References)Figures/Illustrations/Graphs and Legends, References)

• Summary: Why you feel this article should be Summary: Why you feel this article should be accepted/revised/rejectedaccepted/revised/rejected

**Provenzale and Stanley, AJR 2005


• The most important criteria are The most important criteria are

1.1. The importance of the study or report The importance of the study or report to the existing body of knowledgeto the existing body of knowledge

2.2. The scientific merit of the studyThe scientific merit of the study

The Scientific ManuscriptThe Scientific Manuscript

Section by SectionSection by Section

• (Title)(Title)• AbstractAbstract• IntroductionIntroduction• Materials and Methods Materials and Methods

• Combined with Results for Brief ReportsCombined with Results for Brief Reports

• ResultsResults• DiscussionDiscussion• Figures/GraphsFigures/Graphs• TablesTables• ReferencesReferences



TitleTitle

• Should truly reflect purpose and findings of Should truly reflect purpose and findings of studystudy

• Not too longNot too long

• Watch abbreviationsWatch abbreviations


AbstractAbstract

• The summary of the manuscript’s most important featuresThe summary of the manuscript’s most important features• For many readers, this is the ONLY part that they will read.For many readers, this is the ONLY part that they will read.• Therefore, the abstract should be able to stand alone as a Therefore, the abstract should be able to stand alone as a

summary of the worksummary of the work• Ideally, should contain a well-articulated purpose and Ideally, should contain a well-articulated purpose and

hypothesis (not just “report our experience with…”)hypothesis (not just “report our experience with…”)


AbstractAbstract

• Purpose (avoid vague objectives)Purpose (avoid vague objectives)• Materials and MethodsMaterials and Methods• ResultsResults• ConclusionConclusion


AbstractAbstract

• There should be no major discrepancies between the There should be no major discrepancies between the body of the paper and the Abstract!body of the paper and the Abstract!

• The most important features of each section should The most important features of each section should be the focus of the Abstractbe the focus of the Abstract

• There are word count limits to the Abstract for each There are word count limits to the Abstract for each type of manuscript--should not be overly lengthytype of manuscript--should not be overly lengthy


AbstractAbstract

• Purpose should mirror end of IntroductionPurpose should mirror end of Introduction• Actual data with P values should be included in Results as Actual data with P values should be included in Results as

appropriateappropriate• Conclusions should be justified by and follow directly from Conclusions should be justified by and follow directly from

Methods and Results and NOT simply be a reiteration of Methods and Results and NOT simply be a reiteration of ResultsResults

• Conclusions should not be overstated, esp. on the basis of a Conclusions should not be overstated, esp. on the basis of a small number of patients and observationssmall number of patients and observations


IntroductionIntroduction• Explains, via background information, why the authors Explains, via background information, why the authors

bothered to perform the studybothered to perform the study• Should be brief (save rest for Discussion)Should be brief (save rest for Discussion)• What is the rationale of the study?What is the rationale of the study?

• Show that an important problem or question existsShow that an important problem or question exists• Show that prior published work has failed to adequately address the Show that prior published work has failed to adequately address the

problemproblem

• What are the goals of the study?What are the goals of the study?• Introduce any unusual terms used for the studyIntroduce any unusual terms used for the study• Make sure all data/claims referencedMake sure all data/claims referenced


Materials and MethodsMaterials and Methods

• The “blueprint” of the study (What subjects were The “blueprint” of the study (What subjects were included? How was the study performed?)included? How was the study performed?)

• Should give sufficient information to allow another Should give sufficient information to allow another investigator to repeat the studyinvestigator to repeat the study

• Also provides an outline of statistical methods used, Also provides an outline of statistical methods used, if appropriateif appropriate

• Give definitions (e.g. outcomes measures such as Give definitions (e.g. outcomes measures such as patency results)patency results)

The Scientific ManuscriptThe Scientific ManuscriptMaterials and MethodsMaterials and Methods

• Subsection headings may be useful for major papers (Patient Subsection headings may be useful for major papers (Patient group, technique, group, technique, study endpoints and definitionsstudy endpoints and definitions, statistical , statistical analysis)analysis)

• Patient group: demographics, comorbidities, proof of disease, Patient group: demographics, comorbidities, proof of disease, etc. Technically, for prospective studies, should be in Results, etc. Technically, for prospective studies, should be in Results, but I (and many others) prefer this in the M&M anyway for but I (and many others) prefer this in the M&M anyway for readability/clarity/flow of the paper.readability/clarity/flow of the paper.

• Make sure numbers add up (here and in results)Make sure numbers add up (here and in results)• IRB approval or equivalent (or statement that it is not necessary IRB approval or equivalent (or statement that it is not necessary

at the authors’ institution for this type of study)at the authors’ institution for this type of study)


Materials and MethodsMaterials and Methods

• Details of imaging techniques may be important; routine Details of imaging techniques may be important; routine steps of procedures generally aren’t, but be sure there is no steps of procedures generally aren’t, but be sure there is no question regarding major technical aspects esp. if unusual question regarding major technical aspects esp. if unusual or importantor important

• There should be corporate attribution for devices, etc.There should be corporate attribution for devices, etc.• Complications should be defined (ideally categorized per Complications should be defined (ideally categorized per

SIR standards)SIR standards)• Stats: Worthwhile to develop a working knowledge of Stats: Worthwhile to develop a working knowledge of

important statistical testsimportant statistical tests


ResultsResults

• Should follow directly from the Materials and Methods Should follow directly from the Materials and Methods section (the Materials and Methods section should tell section (the Materials and Methods section should tell what types of Results are to be looked for); no “new” what types of Results are to be looked for); no “new” or unanticipated results should be presented that or unanticipated results should be presented that don’t follow from the M&Mdon’t follow from the M&M

• The order of presentation of results should parallel the The order of presentation of results should parallel the order of presentation of the methodsorder of presentation of the methods

• Section headings may be useful if lots of complex dataSection headings may be useful if lots of complex data


ResultsResults

• Follow guidelines for significant figuresFollow guidelines for significant figures• Patency and survival data should be done by Patency and survival data should be done by

Kaplan Meier analysisKaplan Meier analysis


DiscussionDiscussion

• State whether hypothesis was verified or proven untrue, or (if no State whether hypothesis was verified or proven untrue, or (if no hypothesis) what questions were answered or why the report is hypothesis) what questions were answered or why the report is importantimportant

• Should comment as to whether the results are in line with prior Should comment as to whether the results are in line with prior studies. If not, an attempt should be made to explain the studies. If not, an attempt should be made to explain the discrepancies.discrepancies.

• Review only those parts of the medical literature relevant to the Review only those parts of the medical literature relevant to the study.study.

• Note (preferably in a separate paragraph) Note (preferably in a separate paragraph) limitationslimitations of the study of the study• Should have a concluding paragraph that summarizes the studyShould have a concluding paragraph that summarizes the study


Figures and GraphsFigures and Graphs

• Should illustrate important features of the methods and resultsShould illustrate important features of the methods and results• Many authors have trouble limiting figures to those essential for Many authors have trouble limiting figures to those essential for

the understanding of the MS--should ask yourself whether each the understanding of the MS--should ask yourself whether each figure necessaryfigure necessary

• Watch for HIPAA issues/patient identifiersWatch for HIPAA issues/patient identifiers• Decide also whether color necessary (color expensive in print, we Decide also whether color necessary (color expensive in print, we

can choose color on web, gray scale in print for JVIR)can choose color on web, gray scale in print for JVIR)• Should be highest possible quality and should have figure legends Should be highest possible quality and should have figure legends

that adequately explain the meaning (supplemented by appropriate that adequately explain the meaning (supplemented by appropriate arrows)arrows)


TablesTables

• A Table summarizes complex collections of data in A Table summarizes complex collections of data in order to make it more understandable and in order order to make it more understandable and in order to allow the reader to more easily make to allow the reader to more easily make comparisonscomparisons

• Tables are not necessary if the information can be Tables are not necessary if the information can be adequately presented in the text (the latter is adequately presented in the text (the latter is preferable)preferable)


ReferencesReferences

• Should be timely, accurate, and should follow the journal Should be timely, accurate, and should follow the journal citation formatcitation format

• Should support claims made in the text of the manuscriptShould support claims made in the text of the manuscript• The support for the study should be based on evidence that The support for the study should be based on evidence that

is as strong as possibleis as strong as possible• Evidence given from articles should be cited accurately--in Evidence given from articles should be cited accurately--in

other words, the results of other articles should not be other words, the results of other articles should not be misinterpreted to buttress the authors’ casemisinterpreted to buttress the authors’ case

• Be sure in journal formatBe sure in journal format

Why Do Articles Get Accepted?Why Do Articles Get Accepted?

• The study is considered timely and relevant to a The study is considered timely and relevant to a current problemcurrent problem

• The manuscript is well-written, logical, and easy to The manuscript is well-written, logical, and easy to comprehendcomprehend

• The study is well designed with appropriate The study is well designed with appropriate methodologymethodology

Bordage G, Acad Med 2001

Why Do Articles Get Rejected?Why Do Articles Get Rejected?

• Incomplete or insufficiently described statisticsIncomplete or insufficiently described statistics• Overinterpretation of the results (e.g. stating that a Overinterpretation of the results (e.g. stating that a

technique is “safe and effective” on the basis of a single technique is “safe and effective” on the basis of a single case report)case report)

• Suboptimal or insufficiently described means of measuring Suboptimal or insufficiently described means of measuring data (again, could another investigator duplicate the study?)data (again, could another investigator duplicate the study?)

• Sample population too small or biasedSample population too small or biased• Text difficult to follow (grammar/syntax vs. complex, highly Text difficult to follow (grammar/syntax vs. complex, highly

specialized language insufficiently explained for readers)specialized language insufficiently explained for readers)• Insufficient problem statementInsufficient problem statement

Bordage G, Acad Med 2001

How to Get Your Paper PublishedHow to Get Your Paper Published

• Assuming that you have a good study, a Assuming that you have a good study, a number of other steps can be taken to help number of other steps can be taken to help improve the likelihood of acceptanceimprove the likelihood of acceptance

• Some of these are remarkably easy to Some of these are remarkably easy to implement (yet even more remarkably, often implement (yet even more remarkably, often ignored by authors) ignored by authors)

How To Get Your Paper PublishedHow To Get Your Paper Published

Tip 1Tip 1

• Highlight the Importance of the Highlight the Importance of the ManuscriptManuscript


Highlight the Importance of the ManuscriptHighlight the Importance of the Manuscript

• Each acceptable study or report should add Each acceptable study or report should add to the literature in an important and unique to the literature in an important and unique way. Don’t make the reviewer guess this.way. Don’t make the reviewer guess this.



• Focus the Introduction on the background leading to the Focus the Introduction on the background leading to the study and the report, and finish it with a clear statement study and the report, and finish it with a clear statement of purpose, ideally a hypothesis.of purpose, ideally a hypothesis.

• Focus the Discussion on an explanation of why the Focus the Discussion on an explanation of why the conclusions and the purpose served by the manuscript conclusions and the purpose served by the manuscript are valuable, and place this explanation in the context of are valuable, and place this explanation in the context of pre-existing literature (Does it corroborate existing pre-existing literature (Does it corroborate existing literature, perhaps making certain conclusions more literature, perhaps making certain conclusions more firm? Does it refute existing literature? If so, why?)firm? Does it refute existing literature? If so, why?)



• Avoid direct statements of primacy (“This is the first report Avoid direct statements of primacy (“This is the first report of…” or “This has not previously been described…”). of…” or “This has not previously been described…”).

• Statements of primacy are difficult to sustain.Statements of primacy are difficult to sustain.• Statements of primacy add little to a manuscript otherwise Statements of primacy add little to a manuscript otherwise

deemed worthy of publication.deemed worthy of publication.• Statements of primacy sometimes border on the absurd Statements of primacy sometimes border on the absurd

(“This is the first description of percutaneous biopsy of a (“This is the first description of percutaneous biopsy of a left patellar angiosarcoma in a 41 year old commercial left patellar angiosarcoma in a 41 year old commercial fisherman”)fisherman”)



• Avoid statements of purpose that simply reiterate Avoid statements of purpose that simply reiterate what you are describing. E.g. “We describe our what you are describing. E.g. “We describe our experience with the Ultimate Stent”. Add a experience with the Ultimate Stent”. Add a statement that describes why the reader should statement that describes why the reader should care about your experience with the “Ultimate care about your experience with the “Ultimate Stent”Stent”


Tip 2Tip 2

• Follow Directions!Follow Directions!


Follow DirectionsFollow Directions

• Each scientific journal has very clearly articulated Each scientific journal has very clearly articulated “Instructions to Authors”“Instructions to Authors”

• Such instructions detail the types of articles a Such instructions detail the types of articles a journal considers, the acceptable format for such journal considers, the acceptable format for such articles, and the content requirements for the articles, and the content requirements for the individual sections of the articles.individual sections of the articles.

Types of ManuscriptsJVIR publishes several types of articles, each of which has a distinct format. Clinical and Laboratory Investigations are the central focus of the Journal and are based on original clinical or experimental studies. The complete format is described below. Brief Reports include descriptions of a new or modified interventional procedure or device and small clinical studies or case reports. A brief one paragraph abstract (less than 100 words) should be included. In general, limit the paper to six pages of text, 15 references, and no more than eight figure parts. Review Articles are generally invited by the Editor. Specific instructions are provided at the time of invitation. Letters to the Editor can be used to offer commentary on any material published in JVIR. Letters may also be used to convey material of more general interest to the interventional radiology community. On occasion, the Editor may offer such space for submitted case reports that do not receive high enough priority for publication as such. Letters should be no longer than three pages with no more than four references. Only one figure (with no more than four figure parts) can be submitted. Letters to the Editor are accepted for publication at the discretion of the Editor and may be copyedited for content and length.

Manuscript PreparationThe preferred word processing program is Microsoft Word. Manuscripts must be written with 12 point font, double-spaced throughout (including tables, references, and figure legends), and have at least 3 cm margins. The text should be ragged right (no right justification). Embedded instructions (eg, italics, underlines, boldface) should not be used or kept to a minimum Do not use coding for centering. Insert only one space after punctuation marks. Sequential page numbering should begin with the text. The order of sections is Abstract, Text, Acknowledgements, References, Tables, and Figure Legends. To ensure blinded peer-review, no direct references to the author(s) or institution of origin should be made anywhere in the text of figures.


Follow DirectionsFollow Directions

• ““Instructions to Authors” also frequently provide Instructions to Authors” also frequently provide additional suggestions for enhancing the value of additional suggestions for enhancing the value of submitted manuscriptssubmitted manuscripts

• For example: JVIR lists (and gives access to) For example: JVIR lists (and gives access to) published reporting standards that, if relevant to published reporting standards that, if relevant to the paper, should be followed.the paper, should be followed.

Reporting Standards: In order to assure consistency in reporting of results of clinical research, the Society of Interventional Radiology has developed a number of reporting standards documents that authors should follow when submitting manuscripts for consideration. Links to these documents are given below. Adherence to relevant reporting standards will be taken into account in the review process.

Haskal, Ziv J., Rees, Chet R., Ring, Ernest J., Saxon, Richard, Sacks, David Reporting Standards for Transjugular Intrahepatic Portosystemic Shunts J Vasc Interv Radiol 2003 14: 419S-426


Tip 3Tip 3

• Make the manuscript as “readable” as possibleMake the manuscript as “readable” as possible


Make the Manuscript ReadableMake the Manuscript Readable

• Reviewers are only human, and errors in grammar, syntax, and spelling are at Reviewers are only human, and errors in grammar, syntax, and spelling are at the very least frustrating and distracting to the reviewer.the very least frustrating and distracting to the reviewer.

• In extreme cases, such errors can confuse the message of even the most In extreme cases, such errors can confuse the message of even the most scientifically sound study.scientifically sound study.

• Also, reviewers may assume (rightly or wrongly) that such errors are reflective Also, reviewers may assume (rightly or wrongly) that such errors are reflective not only of the writing of the manuscript, but the way the study itself was not only of the writing of the manuscript, but the way the study itself was performed.performed.

• Abbreviations should be explained at first “callout” in textAbbreviations should be explained at first “callout” in text• Avoid unconventional abbreviations (abbreviations place the burden of Avoid unconventional abbreviations (abbreviations place the burden of

remembering what they stand for on the reader, and can be extremely irritating remembering what they stand for on the reader, and can be extremely irritating to the reader)to the reader)

• Note: as a reviewer, your choice to point these out, but if there are many, many Note: as a reviewer, your choice to point these out, but if there are many, many such errors simply state--multiple errors of grammar and syntaxsuch errors simply state--multiple errors of grammar and syntax


Make the Manuscript ReadableMake the Manuscript Readable

• The problem of readability is particularly relevant The problem of readability is particularly relevant for authors for whom (at least for JVIR) English is for authors for whom (at least for JVIR) English is not their native tongue.not their native tongue.

• While this problem is completely understandable, a While this problem is completely understandable, a good suggestion is to have a colleague fluent in good suggestion is to have a colleague fluent in English manuscript preparation review the English manuscript preparation review the manuscript prior to submissionmanuscript prior to submission


Tip 4Tip 4

• Accept Suggestions By Reviewers as Ways to Accept Suggestions By Reviewers as Ways to Improve the ManuscriptImprove the Manuscript


Accept Suggestions By ReviewersAccept Suggestions By Reviewers

• Too often, authors respond to reviewers in an argumentative Too often, authors respond to reviewers in an argumentative fashion, as if the reviewer were personally attacking them. fashion, as if the reviewer were personally attacking them. “What in the world does this reviewer want???”“What in the world does this reviewer want???”

• Be respectful of the reviewers: while they may be mistaken Be respectful of the reviewers: while they may be mistaken in some of their suggestions, most are trying very hard to in some of their suggestions, most are trying very hard to help authors recognize weaknesses in the study and its help authors recognize weaknesses in the study and its presentation. Try to respond in a calm, thoughtful fashion presentation. Try to respond in a calm, thoughtful fashion even if you disagree with a suggestion.even if you disagree with a suggestion.


Tip 5Tip 5

• Be Succinct!Be Succinct!


Be SuccinctBe Succinct

• Short, concise articles that “stick to the point” are Short, concise articles that “stick to the point” are far easier to read and review than rambling tomes far easier to read and review than rambling tomes that stray far from the heart of the study.that stray far from the heart of the study.

• It is reasonably easy to add material if an editor or It is reasonably easy to add material if an editor or reviewer believes more detail is necessary.reviewer believes more detail is necessary.

• Delete material extremely familiar to journal readers Delete material extremely familiar to journal readers (e.g. explanation of Seldinger technique)(e.g. explanation of Seldinger technique)


Tip 6Tip 6

• Know the Audience for the Journal You Are Know the Audience for the Journal You Are Submitting the Manuscript toSubmitting the Manuscript to


Know the AudienceKnow the Audience

• A superbly written, scientifically sound manuscript may, A superbly written, scientifically sound manuscript may, nevertheless, not be accepted to a particular journal, if the nevertheless, not be accepted to a particular journal, if the subject matter fails to reflect the practice patterns and subject matter fails to reflect the practice patterns and concerns of the majority of readers.concerns of the majority of readers.

• This is a little difficult in some cases for interventional This is a little difficult in some cases for interventional radiology, which covers such a broad range of practice radiology, which covers such a broad range of practice patterns. However, for example, an article on treating patterns. However, for example, an article on treating coronary artery disease with drug eluting stents would, in coronary artery disease with drug eluting stents would, in general, be better sent to a journal on coronary interventions general, be better sent to a journal on coronary interventions than to JVIR.than to JVIR.


SummarySummary

• Timely, relevant, evidence-based scientific studies that are well Timely, relevant, evidence-based scientific studies that are well designed and well writtendesigned and well written

• Highlight the importance of the study or reportHighlight the importance of the study or report• Follow directionsFollow directions• Make the manuscript clear, logical, and easy to readMake the manuscript clear, logical, and easy to read• Be willing to accept reviewers’ suggestions as ways to Be willing to accept reviewers’ suggestions as ways to

improve the manuscriptimprove the manuscript• Be succinctBe succinct• Know the audienceKnow the audience

Thank You!

Documents

How to Review (…and Read…and Write) Scientific Papers