Standards for the CONDUCT of new Cochrane Intervention

Standards for the CONDUCT of new Cochrane Intervention Reviews (C1-C75) ............................. 2Key points and introduction ........................................................................ 2Developing the protocol of the review (C1-C23) ........................................................... 2Setting the research question to inform the scope of the review (C1-C4) ........................................... 2Setting eligibility criteria for including studies in the review (C5-C13) .............................................. 3Selecting outcomes to be addressed for studies included in the review (C14-C18) ..................................... 6Planning the review methods at protocol stage (C19-C23) .................................................... 9Performing the review (C24-C75) ................................................................... 10Searching for studies (C24-C38) ................................................................... 10Selecting studies to include in the review (C39-C42) ....................................................... 15Collecting data from included studies (C43-C51) ......................................................... 16Assessing risk of bias in included studies (C52-C60) ...................................................... 19Synthesizing the results of included studies (C61-C73) ..................................................... 21Assessing the quality of evidence and summarizing the findings (C74-C75) ........................................ 25Reference ................................................................................. 25Citation ................................................................................... 26

Page 1/26

Standards for the CONDUCT of new Cochrane Intervention Reviews (C1-C75) Key points and introductionKey points:

The conduct Standards should be consulted during preparation of the protocol for a Cochrane InterventionReview.They describe the methods that should be implemented throughout the review process.Few specific methods are mandatory, one notable exception being the Cochrane tool for assessing risk of biaswhen randomized trials are included in the review.

The MECIR Standards for conduct of a Cochrane Intervention Review provide expectations for the general methodologicalapproach to be followed from designing the review up to interpreting the findings at the end. They should be consulted particularlywhen preparing the protocol for the review. The protocol describes the review question, the criteria for considering studies for thereview, and the methods that will be followed to identify, appraise, summarize and synthesize the studies. Cochrane led the way inmaking protocols available to readers of the Cochrane Library. They ensure transparency in how reviews are prepared and allowthe planned methods to be critiqued. Specification of the review question (through setting the review’s objectives) and the criteriafor including studies are critical to the success of the review and the first two sections of the Standards address these tasks. Thefollowing section addresses selection of the outcomes of interest, an important aspect that should be prespecified carefully to avoidthe need for post hoc decisions that could be influenced by the data.

The remaining Standards address the detailed methodology that will be followed during the review, covering the search for studies,selection of studies into the review, data collection, risk of bias assessment, synthesis (including any meta-analysis approaches),and overall assessment of the evidence. With few exceptions (such as use of the Cochrane Risk of Bias 2 tool for randomizedtrials), the precise methods to be used are not prescribed, For example, authors are free to use any meta-analysis method, althoughthere is a potential convenience to both authors and readers if those implemented in Review Manager (RevMan) software areused.

Julian HigginsProfessor of Evidence SynthesisUniversity of Bristol

Developing the protocol of the review (C1-C23)Cochrane Training resource: writing a protocol and common errors and best practice: writing review protocols

Cochrane Interactive Learning (CIL): module 2 - writing the review protocol

Setting the research question to inform the scope of the review (C1-C4)

Setting the research question(s) to inform the scope of the review

Cochrane Training resource: defining the review question

Cochrane Interactive Learning (CIL): module 1 - introduction to conducting systematic reviews

Standard Rationale and elaboration ResourcesC1 Formulating review questions Mandatory Ensure that the review question

and particularly the outcomes ofinterest, address issues thatare important to review userssuch as consumers, healthprofessionals and policymakers.

Cochrane Reviews areintended to support clinicalpractice and policy, not justscientific curiosity. The needsof consumers play a central rolein Cochrane Reviews and theycan play an important role in

See Handbook Section 2.1

Page 2/26

http://training.cochrane.org/resource/writing-protocol

defining the review question.Qualitative research, i.e.studies that explore theexperience of those involved inproviding and receivinginterventions, and studiesevaluating factors that shapethe implementation ofinterventions, might be used inthe same way.

C2 Predefining objectives Mandatory Define in advance the

objectives of the review,including participants,interventions, comparators andoutcomes (PICO).

Objectives give the reviewfocus and must be clear beforeappropriate eligibility criteriacan be developed. If the reviewwill address multipleinterventions, clarity is requiredon how these will be addressed(e.g. summarized separately,combined or explicitlycompared).


C3 Considering potential adverseeffects

Mandatory

Consider any importantpotential adverse effects of theintervention(s) and ensure thatthey are addressed.

It is important that adverseeffects are addressed in orderto avoid one-sided summariesof the evidence. At a minimum,the review will need to highlightthe extent to which potentialadverse effects have beenevaluated in any includedstudies. Sometimes data onadverse effects are bestobtained from non-randomizedstudies, or qualitative researchstudies. This does not meanhowever that all reviews mustinclude non-randomizedstudies.


Cochrane Training resource: adverse effects

C4 Considering equity and specificpopulations

Highly desirable

Consider in advance whetherissues of equity and relevanceof evidence to specificpopulations are important to thereview, and plan for appropriatemethods to address them ifthey are. Attention should bepaid to the relevance of thereview question to populationssuch as low-socioeconomicgroups, low- or middle-incomeregions, women, children andolder people.

Where possible reviews shouldinclude explicit descriptions ofthe effect of the interventionsnot only upon the wholepopulation, but also on thedisadvantaged, and/or theability of the interventions toreduce socioeconomicinequalities in health, and topromote use of theinterventions to the community.


Cochrane Training resources: equity issues and PRISMA-E2012

Page 3/26

Setting eligibility criteria for including studies in the review (C5-C13)

Setting the eligibility criteria for including studies in the review


Cochrane Interactive Learning (CIL): module 2 - writing the review protocol

Standard Rationale and elaboration ResourcesC5 Predefining unambiguous

criteria for participantsMandatory

Define in advance the eligibilitycriteria for participants in thestudies.

Predefined, unambiguouseligibility criteria are afundamental prerequisite for asystematic review. The criteriafor considering types of peopleincluded in studies in a reviewshould be sufficiently broad toencompass the likely diversityof studies, but sufficientlynarrow to ensure that ameaningful answer can beobtained when studies areconsidered in aggregate.Considerations when specifyingparticipants include setting,diagnosis or definition ofcondition and demographicfactors. Any restrictions tostudy populations must bebased on a sound rationale,since it is important thatCochrane Reviews are widelyrelevant.

See Handbook Section 3.2.1

C6 Predefining a strategy forstudies with a subset of eligibleparticipants

Highly desirable

Define in advance how studiesthat include only a subset ofrelevant participants will beaddressed.

Sometimes a study includessome ‘eligible’ participants andsome ‘ineligible’ participants,for example when an age cut-off is used in the review’seligibility criteria. If data fromthe eligible participants cannotbe retrieved, a mechanism fordealing with this situationshould be prespecified.


C7 Predefining unambiguouscriteria for interventions andcomparators

Mandatory

Define in advance the eligibleinterventions and theinterventions against whichthese can be compared in theincluded studies.

Predefined, unambiguouseligibility criteria are afundamental prerequisite for asystematic review. Specification of comparatorinterventions requires particularclarity: are the experimentalinterventions to be comparedwith an inactive control


Page 4/26

intervention (e.g. placebo, notreatment, standard care, or awaiting list control), or with anactive control intervention (e.g.a different variant of the sameintervention, a different drug, adifferent kind of therapy)? Anyrestrictions on interventions andcomparators, for example,regarding delivery, dose,duration, intensity,cointerventions and features ofcomplex interventions shouldalso be predefined andexplained.

C8 Clarifying role of outcomes Mandatory Clarify in advance whether

outcomes listed under 'Criteriafor considering studies for thisreview' are used as criteria forincluding studies (rather thanas a list of the outcomes ofinterest within whicheverstudies are included).

Outcome measures should notalways form part of the criteriafor including studies in a review.However, some reviews dolegitimately restrict eligibility tospecific outcomes. Forexample, the same interventionmay be studied in the samepopulation for differentpurposes (e.g. hormonereplacement therapy, oraspirin); or a review mayaddress specifically theadverse effects of anintervention used for severalconditions. If authors doexclude studies on the basis ofoutcomes, care should be takento ascertain that relevantoutcomes are not availablebecause they have not beenmeasured rather than simplynot reported.

See Handbook Section 3.2.4.1

C9 Predefining study designs Mandatory Define in advance the eligibility

criteria for study designs in aclear and unambiguous way,with a focus on features of astudy's design rather thandesign labels.

Predefined, unambiguouseligibility criteria are afundamental prerequisite for asystematic review. This isparticularly important when non-randomized studies areconsidered. Some labelscommonly used to define studydesigns can be ambiguous. Forexample a ‘double blind’ studymay not make it clear who wasblinded; a ‘case control’ studymay be nested within a cohort,or be undertaken in a cross-sectional manner; or a‘prospective’ study may haveonly some features defined orundertaken prospectively.


Page 5/26

C10 Including randomized trials Mandatory Include randomized trials as

eligible for inclusion in thereview, if it is feasible toconduct them to evaluateinterventions and outcomes ofinterest.

Randomized trials are the beststudy design for evaluating theefficacy of interventions. If it isfeasible to conduct them toevaluate questions that arebeing addressed by the review,they must be consideredeligible for the review. However,appropriate exclusion criteriamay be put in place, forexample regarding length offollow-up.


C11 Justifying choice of studydesigns

Mandatory

Justify the choice of eligiblestudy designs.

It might be difficult to addresssome interventions or someoutcomes in randomized trials.Authors should be able to justifywhy they have chosen either torestrict the review torandomized trials or to includenon-randomized studies. Theparticular study designsincluded should be justified withregard to appropriateness tothe review question and withregard to potential for bias.


C12 Excluding studies based onpublication status

Mandatory

Include studies irrespective oftheir publication status, unlessexclusion is explicitly justified.

Obtaining and including datafrom unpublished studies(including grey literature) canreduce the effects of publicationbias. However, the unpublishedstudies that can be located maybe an unrepresentative sampleof all unpublished studies.


C13 Changing eligibility criteria Mandatory Justify any changes to eligibility

criteria or outcomes studied. Inparticular, post hoc decisionsabout inclusion or exclusion ofstudies should keep faith withthe objectives of the reviewrather than with arbitrary rules.

Following prespecified eligibilitycriteria is a fundamentalattribute of a systematic review.However, unanticipated issuesmay arise. Review authorsshould make sensible post hocdecisions about exclusion ofstudies, and these should bedocumented in the review,possibly accompanied bysensitivity analyses. Changes tothe protocol must not be madeon the basis of the findings ofthe studies or the synthesis, asthis can introduce bias.


Selecting outcomes to be addressed for studies included in the review (C14-C18)

Page 6/26

Selecting outcomes to be addressed for studies included in the review


Cochrane Interactive Learning: module 2 - writing the review protocol

Standard Rationale and elaboration ResourcesC14 Predefining outcome domains Mandatory Define in advance outcomes

that are critical to the review,and any additional importantoutcomes.

Full specification of theoutcomes includesconsideration of outcomedomains (e.g. quality of life) andoutcome measures (e.g.SF-36). Predefinition ofoutcome reduces the risk ofselective outcome reporting.The critical outcomes shouldbe as few as possible andshould normally reflect at leastone potential benefit and atleast one potential area ofharm. It is expected that thereview should be able tosynthesize these outcomes ifeligible studies are identified,and that the conclusions of thereview will be based largely onthe effects of the interventionson these outcomes. Additionalimportant outcomes may alsobe specified. Up to sevencritical and important outcomeswill form the basis of theGRADE assessment andsummarized in the review'sabstract and other summaryformats, although the reviewmay measure more than sevenoutcomes.


Planning GRADE andSummary of Findings tables

C15 Choosing outcomes Mandatory Choose only outcomes that are

critical or important to users ofthe review such as healthcareconsumers, healthprofessionals and policymakers.

Cochrane Reviews areintended to support clinicalpractice and policy, and shouldaddress outcomes that arecritical or important toconsumers. These should bespecified at protocol stage.Where available, establishedsets of core outcomes shouldbe used. Patient-reportedoutcomes should be includedwhere possible. It is alsoimportant to judge whetherevidence of resource use andcosts might be an importantcomponent of decisions toadopt the intervention oralternative managementstrategies around the world.Large numbers of outcomes,


Page 7/26

while sometimes necessary,can make reviews unfocussed,unmanageable for the user, andprone to selective outcomereporting bias. Biochemical,interim and process outcomesshould be considered wherethey are important to decisionmakers. Any outcomes thatwould not be described ascritical or important can be leftout of the review.

C16 Predefining outcome measures Highly desirable Define in advance details of

what will constitute acceptableoutcome measures (e.g.diagnostic criteria, scales,composite outcomes).

Having decided what outcomesare of interest to the review,authors should clarifyacceptable ways in which theseoutcomes can be measured. Itmay be difficult, however, topredefine adverse effects.


C17 Predefining choices frommultiple outcome measures

Highly desirable

Define in advance how outcomemeasures will be selected whenthere are several possiblemeasures (e.g. multipledefinitions, assessors orscales).

Prespecification guards againstselective outcome reporting,and allows users to confirm thatchoices were not overlyinfluenced by the results. Apredefined hierarchy ofoutcomes measures may behelpful. It may be difficult,however, to predefine adverseeffects. A rationale should beprovided for the choice ofoutcome measure.

C18 Predefining time points ofinterest

Highly desirable

Define in advance the timing ofoutcome measurement.

Prespecification guards againstselective outcome reporting,and allows users to confirm thatchoices were not overlyinfluenced by the results.Authors may consider whetherall time frames or only selectedtime points will be included inthe review. These decisionsshould be based on outcomesimportant for making healthcaredecisions. One strategy tomake use of the available datacould be to group time pointsinto prespecified intervals torepresent ‘short-term’, ‘medium-term’ and ‘long-term’ outcomesand to take no more than onefrom each interval from eachstudy for any particularoutcome.

Page 8/26

Planning the review methods at protocol stage (C19-C23)

Planning the review methods at protocol stage

Standard Rationale and elaboration ResourcesC19 Planning the search Mandatory Plan in advance the methods to

be used for identifying studies.Design searches to capture asmany studies as possible thatmeet the eligibility criteria,ensuring that relevant timeperiods and sources arecovered and not restricted bylanguage or publication status.

Searches should be motivateddirectly by the eligibility criteriafor the review, and it isimportant that all types ofeligible studies are consideredwhen planning the search. Ifsearches are restricted bypublication status or bylanguage of publication, there isa possibility of publication bias,or language bias (whereby thelanguage of publication isselected in a way that dependson the findings of the study), orboth. Removing languagerestrictions in English languagedatabases is not a goodsubstitute for searching non-English language journals anddatabases.

See Handbook Section 1.5; 4.3.1.1

Cochrane Training resource: searching studies

CIL: module 3 - searching forstudies

C20 Planning the assessment ofrisk of bias in included studies

Mandatory

Plan in advance the methods tobe used for assessing risk ofbias in included studies,including the tool(s) to be used,how the tool(s) will beimplemented, and the criteriaused to assign studies, forexample, to judgements of lowrisk, high risk and unclear riskof bias.

Predefining the methods andcriteria for assessing risk ofbias is important since analysisor interpretation of the reviewfindings may be affected by thejudgements made during thisprocess. For randomized trials,use of the Cochrane ‘risk ofbias’ tool is Mandatory, so it issufficient (and easiest) simplyto refer to the definitions of lowrisk, unclear risk and high riskof bias provided in the Handbook.


Cochrane Training resource: risk of bias

C21 Planning the synthesis ofresults

Mandatory

Plan in advance the methods tobe used to synthesize theresults of the included studies,including whether a quantitativesynthesis is planned, howheterogeneity will be assessed,choice of effect measure (e.g.odds ratio, risk ratio, riskdifference or other fordichotomous outcomes), andmethods for meta-analysis (e.g.inverse variance or MantelHaenszel, fixed-effect orrandom-effects model).

Predefining the synthesismethods, particularly thestatistical methods, isimportant, since analysis orinterpretation of the reviewfindings may be affected by thejudgements made during thisprocess.


Cochrane Training resources: meta-analysis; dichotomousoutcomes; continuousoutcomes and heterogeneity

CIL: module 6 - analysing thedata

Page 9/26

C22 Planning sub-group analyses Mandatory Predefine potential effect

modifiers (e.g. for subgroupanalyses) at the protocol stage;restrict these in number, andprovide rationale for each.

Prespecification reduces therisk that large numbers ofundirected subgroup analyseswill lead to spuriousexplanations of heterogeneity.


Cochrane Training resource: heterogeneity

CIL: module 6 - analysing thedata

C23 Planning the GRADEassessment and ‘Summary offindings’ table

Mandatory

Plan in advance the methods tobe used for assessing thequality of the body of evidence,and summarizing the findings ofthe review.

Methods for assessing thequality of evidence for the mostimportant outcomes in thereview need to be prespecified.In ‘Summary of findings’ tablesthe most important feature is topredefine the choice ofoutcomes in order to guardagainst selective presentationof results in the review. Thetable should include theessential outcomes for decisionmaking (typically up to seven),which generally should notinclude surrogate or interimoutcomes. The choice ofoutcomes should not be basedon any anticipated or observedmagnitude of effect, or becausethey are likely to have beenaddressed in the studies to bereviewed.


Cochrane Training resource: evaluating evidence

CIL: module 7 - interpreting thefindings

Planning GRADE andSummary of Findings tables

Performing the review (C24-C75) Searching for studies (C24-C38)

Searching for studies

Cochrane Training resource: searching for studies

Cochrane Interactive Learning (CIL): module 3 - searching for studies

Standard Rationale and elaboration ResourcesC24 Searching general

bibliographic databases andCENTRAL

Mandatory

Search the Cochrane ReviewGroup's (CRG’s) SpecializedRegister (internally, e.g. via theCochrane Register of Studies,or externally via CENTRAL).Ensure that CENTRAL,MEDLINE (e.g. via PubMed)

Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible. The minimumdatabases to be covered are


Cochrane Training resource: Register of Studies andRevMan

Page 10/26

and Embase (if Embase isavailable to either the CRG orthe review author), have beensearched (either for the reviewor for the Review Group’sSpecialized Register).

the CRG’s Specialized Register(if it exists and was designed tosupport reviews in this way),CENTRAL, MEDLINE andEmbase (if Embase is availableto either the CRG or the reviewauthor). Expertise may berequired to avoid unnecessaryduplication of effort. Some, butnot all, reports of eligiblestudies from MEDLINE,Embase and the CRGs’Specialized Registers arealready included in CENTRAL. .

C25 Searching specialistbibliographic databases

Highly desirable

Search appropriate national,regional and subject-specificbibliographic databases.

Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible. Databases relevant tothe review topic should becovered (e.g. CINAHL fornursing-related topics,PsycINFO for psychologicalinterventions), and regionaldatabases (e.g. LILACS)should be considered.


C26 Searching for different types ofevidence

Mandatory

If the review has specificeligibility criteria around studydesign to address adverseeffects, economic issues orqualitative research questions,undertake searches to addressthem.

Sometimes different searcheswill be conducted for differenttypes of evidence, such as fornon-randomized studies foraddressing adverse effects, orfor economic evaluationstudies.


Cochrane Training resource: searching for adverse effects

C27 Searching trials registers Mandatory Search trials registers and

repositories of results, whererelevant to the topic, throughClinicalTrials.gov, the WHOInternational Clinical TrialsRegistry Platform (ICTRP)portal and other sources asappropriate.

Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible. AlthoughClinicalTrials.gov is included asone of the registers within theWHO ICTRP portal, it isrecommended that bothClinicalTrials.gov and theICTRP portal are searchedseparately due to additionalfeatures in ClinicalTrials.gov.


C28 Searching for grey literature Highly desirable Search relevant grey literature

sources such as reports,dissertations, theses,

Searches for studies should beas extensive as possible inorder to reduce the risk of


Page 11/26

databases and databases ofconference abstracts.

publication bias and to identifyas much relevant evidence aspossible.

C29 Searching within other reviews Highly desirable Search within previous reviews

on the same topic.Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible.


C30 Searching reference lists Mandatory Check reference lists in

included studies and anyrelevant systematic reviewsidentified.

Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible.


C31 Searching by contactingrelevant individuals andorganizations

Highly desirable

Contact relevant individualsand organizations forinformation about unpublishedor ongoing studies.

Searches for studies should beas extensive as possible inorder to reduce the risk ofpublication bias and to identifyas much relevant evidence aspossible. It is important toidentify ongoing studies, so thatthese can be assessed forpossible inclusion when areview is updated.


C32 Structuring search strategiesfor bibliographic databases

Mandatory

Inform the structure of searchstrategies in bibliographicdatabases around the mainconcepts of the review, usingappropriate elements fromPICO and study design. Instructuring the search,maximize sensitivity whilststriving for reasonableprecision. Ensure correct use ofthe ‘AND’ and ‘OR’ operators.

Inappropriate or inadequatesearch strategies may fail toidentify records that areincluded in bibliographicdatabases. Expertise may needto be sought, in particular fromthe CRG’s InformationSpecialist. The structure of asearch strategy should bebased on the main conceptsbeing examined in a review. Ingeneral databases, such asMEDLINE, a search strategy toidentify studies for a CochraneReview will typically have threesets of terms: 1) terms tosearch for the health conditionof interest, i.e. the population;2) terms to search for theintervention(s) evaluated; and3) terms to search for the typesof study design to be included(typically a ‘filter’ forrandomized trials). There areexceptions, however. Forinstance, for reviews ofcomplex interventions, it may


Page 12/26

be necessary to search only forthe population or theintervention. Within eachconcept, terms are joinedtogether with the Boolean ‘OR’operator, and the concepts arecombined with the Boolean‘AND’ operator. The ‘NOT’operator should be avoidedwhere possible to avoid thedanger of inadvertentlyremoving records that arerelevant from the search set.

C33 Developing search strategiesfor bibliographic databases

Mandatory

Identify appropriate controlledvocabulary (e.g. MeSH,Emtree, including 'exploded'terms) and free-text terms(considering, for example,spelling variants, synonyms,acronyms, truncation andproximity operators).

Inappropriate or inadequatesearch strategies may fail toidentify records that areincluded in bibliographicdatabases. Search strategiesneed to be customized for eachdatabase. It is important thatMeSH terms are ‘exploded’wherever appropriate, in ordernot to miss relevant articles.The same principle applies toEmtree when searchingEmbase and also to a numberof other databases. Thecontrolled vocabulary searchterms for MEDLINE andEmbase are not identical, andneither is the approach toindexing. In order to be ascomprehensive as possible, it isnecessary to include a widerange of free-text terms foreach of the concepts selected.This might include the use oftruncation and wildcards.Developing a search strategy isan iterative process in whichthe terms that are used aremodified, based on what hasalready been retrieved.


C34 Using search filters Highly desirable Use specially designed and

tested search filters whereappropriate including theCochrane Highly SensitiveSearch Strategies foridentifying randomized trials inMEDLINE, but do not use filtersin pre-filtered databases e.g. donot use a randomized trial filterin CENTRAL or a systematicreview filter in DARE.

Inappropriate or inadequatesearch strategies may fail toidentify records that areincluded in bibliographicdatabases. Search filtersshould be used with caution.They should be assessed notonly for the reliability of theirdevelopment and reportedperformance, but also for theircurrent accuracy, relevanceand effectiveness given thefrequent interface and indexingchanges affecting databases.


Page 13/26

C35 Restricting database searches Mandatory Justify the use of any

restrictions in the searchstrategy on publication dateand publication format.

Date restrictions in the searchshould only be used when thereare date restrictions in theeligibility criteria for studies.They should be applied only if itis known that relevant studiescould only have been reportedduring a specific time period,for example if the interventionwas only available after acertain time point. Searches forupdates to reviews mightnaturally be restricted by dateof entry into the database(rather than date of publication)to avoid duplication of effort.Publication format restrictions(e.g. exclusion of letters) shouldgenerally not be used inCochrane Reviews, since anyinformation about an eligiblestudy may be of value.


C36 Documenting the searchprocess

Mandatory

Document the search processin enough detail to ensure that itcan be reported correctly in thereview.

The search process (includingthe sources searched, when, bywhom, and using which terms)needs to be documented inenough detail throughout theprocess to ensure that it can bereported correctly in the review,to the extent that all thesearches of all the databasesare reproducible.


C37 Rerunning searches Mandatory Rerun or update searches for

all relevant databases within 12months before publication ofthe review or review update,and screen the results forpotentially eligible studies.

The published review should beas up to date as possible. Thesearch must be rerun close topublication, if the initial searchdate is more than 12 months(preferably six months) from theintended publication date, andthe results screened forpotentially eligible studies.Ideally the studies should beincorporated fully in the review.If not, then the potentiallyeligible studies will need to bereported, at a minimum as areference under ‘Studiesawaiting classification’ (or‘Ongoing studies’ if they havenot yet completed).


C38 Incorporating findings fromrerun searches

Highly desirable

Fully incorporate any studiesidentified in the rerun or updateof the search within 12 months

The published review should beas up to date as possible. Afterthe rerun of the search, the


Page 14/26

before publication of the reviewor review update.

decision whether to incorporateany new studies fully into thereview will need to be balancedagainst the delay in publication.

Selecting studies to include in the review (C39-C42)

Selecting studies to include in the review

Cochrane Training resources: selecting studies and Covidence webinar (online tool for review production)

Cochrane Interactive Learning (CIL): module 4 - selecting studies and collecting data

Standard Rationale and elaboration ResourcesC39 Making inclusion decisions Mandatory Use (at least) two people

working independently todetermine whether each studymeets the eligibility criteria, anddefine in advance the processfor resolving disagreements.

Duplicating the study selectionprocess reduces both the riskof making mistakes and thepossibility that selection isinfluenced by a single person’sbiases. The inclusion decisionsshould be based on the fulltexts of potentially eligiblestudies when possible, usuallyafter an initial screen of titlesand abstracts. It is desirable,but not mandatory, that twopeople undertake this initialscreening, workingindependently.


C40 Excluding studies withoutuseable data

Mandatory

Include studies in the reviewirrespective of whethermeasured outcome data arereported in a ‘usable’ way.

Systematic reviews typicallyshould seek to include allrelevant participants who havebeen included in eligible studydesigns of the relevantinterventions and had theoutcomes of interest measured.Reviews must not excludestudies solely on the basis of reporting of the outcome data,since this may introduce biasdue to selective outcomereporting and risk underminingthe systematic review process.While such studies cannot beincluded in meta-analyses, theimplications of their omissionshould be considered. Note thatstudies may legitimately beexcluded because outcomeswere not measured.Furthermore, issues may bedifferent for adverse effectsoutcomes, since the pool of


Page 15/26

studies may be much largerand it can be difficult to assesswhether such outcomes weremeasured.

C41 Documenting decisions aboutrecords identified

Mandatory

Document the selectionprocess in sufficient detail to beable to complete a flow diagramand a table of ‘Characteristicsof excluded studies’.

Decisions should therefore bedocumented for all recordsidentified by the search.Numbers of records aresufficient for exclusions basedon initial screening of titles andabstracts. Broadcategorizations are sufficient forrecords classed as potentiallyeligible during an initial screen.Studies listed in the table of‘Characteristics of excludedstudies’ should be those that auser might reasonably expect tofind in the review. At least oneexplicit reason for theirexclusion must be documented.Authors will need to decide foreach review when to maprecords to studies (if multiplerecords refer to one study).Lists of included and excludedstudies must be based onstudies rather than records.


C42 Collating multiple reports Mandatory Collate multiple reports of the

same study, so that each study,rather than each report, is theunit of interest in the review.

It is wrong to consider multiplereports of the same study as ifthey are multiple studies.Secondary reports of a studyshould not be discarded,however, since they maycontain valuable informationabout the design and conduct.Review authors must chooseand justify which report to useas a source for study results.

See Handbook Sections 4.6.2; 5.2.1

Collecting data from included studies (C43-C51)

Collecting data from included studies

Cochrane Training resources: collecting data and Covidence webinar (online tool for review production)

Cochrane Interactive Learning (CIL): module 4 - selecting studies and collecting data

Standard Rationale and elaboration ResourcesC43 Using data collection forms Mandatory Use a data collection form

which has been piloted.Review authors often havedifferent backgrounds and levelof systematic reviewexperience. Using a datacollection form ensures some


Page 16/26

consistency in the process ofdata extraction, and isnecessary for comparing dataextracted in duplicate. Thecompleted data collection formsshould be available to the CRGon request. Piloting the formwithin the review team is highlydesirable. At a minimum, thedata collection form (or a veryclose variant of it) must havebeen assessed for usability.

C44 Describing studies Mandatory Collect characteristics of the

included studies in sufficientdetail to populate a table of‘Characteristics of includedstudies’.

Basic characteristics of eachstudy will need to be presentedas part of the review, includingdetails of participants,interventions and comparators,outcomes and study design.


C45 Extracting study characteristicsin duplicate

Highly desirable

Use (at least) two peopleworking independently toextract study characteristicsfrom reports of each study, anddefine in advance the processfor resolving disagreements.

Duplicating the data extractionprocess reduces both the riskof making mistakes and thepossibility that data selection isinfluenced by a single person’sbiases. Dual data extractionmay be less important for studycharacteristics than it is foroutcome data, so it is not amandatory standard for theformer.


C46 Extracting outcome data induplicate

Mandatory

Use (at least) two peopleworking independently toextract outcome data fromreports of each study, anddefine in advance the processfor resolving disagreements.

Duplicating the data extractionprocess reduces both the riskof making mistakes and thepossibility that data selection isinfluenced by a single person’sbiases. Dual data extraction isparticularly important foroutcome data, which feeddirectly into syntheses of theevidence, and hence to theconclusions of the review.


C47 Making maximal use of data Mandatory Collect and utilize the most

detailed numerical data thatmight facilitate similar analysesof included studies. Where 2×2tables or means and standarddeviations are not available,this might include effectestimates (e.g. odds ratios,regression coefficients),confidence intervals, teststatistics (e.g. t, F, Z, Chi2) or Pvalues, or even data forindividual participants.

Data entry into RevMan iseasiest when 2×2 tables arereported for dichotomousoutcomes, and when meansand standard deviations arepresented for continuousoutcomes. Sometimes thesestatistics are not reported butsome manipulations of thereported data can be performedto obtain them. For instance,2×2 tables can often be derivedfrom sample sizes and


Cochrane Training resources: dichotomous outcomes and continuous outcomes

Page 17/26

percentages, while standarddeviations can often becomputed using confidenceintervals or P values.Furthermore, the inverse-variance data entry format canbe used even if the detaileddata required for dichotomousor continuous data are notavailable, for instance if onlyodds ratios and theirconfidence intervals arepresented. The RevMancalculator facilitates many ofthese manipulations.

C48 Examining errata Mandatory* Examine any relevant retraction

statements and errata forinformation.

Some studies may have beenfound to be fraudulent or mayhave been retracted sincepublication for other reasons.Errata can reveal importantlimitations, or even fatal flaws,in included studies. All of thesemay lead to the potentialexclusion of a study from areview or meta-analysis. Careshould be taken to ensure thatthis information is retrieved inall database searches bydownloading the appropriatefields, together with the citationdata.

See Handbook Section 4.4.6; 5.2

C49 Obtaining unpublished data Highly desirable Seek key unpublished

information that is missing fromreports of included studies.

Contacting study authors toobtain or confirm data makesthe review more complete,potentially enhances precisionand reduces the impact ofreporting biases. Missinginformation includes details toinform ‘risk of bias’assessments, details ofinterventions and outcomes,and study results (includingbreakdowns of results byimportant subgroups).


C50 Choosing interventions in multi-arm studies

Mandatory

If a study is included with morethan two intervention arms,include in the review only theinterventions that meet theeligibility criteria.

There is no point includingirrelevant interventions in thereview. Authors, however,should make it clear in the‘Table of characteristics ofincluded studies’ that theseinterventions were present inthe study.


Cochrane Training resource: non-standard data and studydesign

Page 18/26

C51 Checking accuracy of numericdata in the review

Mandatory

Compare magnitude anddirection of effects reported bystudies with how they arepresented in the review, takingaccount of legitimatedifferences.

This is a reasonablystraightforward way for authorsto check a number of potentialproblems, includingtypographical errors in studies’reports, accuracy of datacollection and manipulation,and data entry into RevMan. For example, the direction of astandardized mean differencemay accidentally be wrong inthe review. A basic check is toensure the same qualitativefindings (e.g. direction of effectand statistical significance)between the data as presentedin the review and the data asavailable from the originalstudy. Results in forest plotsshould agree with data in theoriginal report (point estimateand confidence interval) if thesame effect measure andstatistical model is used.

Cochrane Trainingresource: common errors

Assessing risk of bias in included studies (C52-C60)Cochrane Training resources: assessing RoB and RoB 2.0 webinar

Cochrane Interactive Learning (CIL): module 5 - introduction to study quality and risk of bias

Standard Rationale and elaboration ResourcesC52 Assessing risk of bias

Anupdate to this Standard ispending

Mandatory

Assess the risk of bias in atleast one specific result foreach included study. Forrandomized trials, the RoB 2tool should be used, involvingjudgements and support forthose judgements across aseries of domains of bias, asdescribed in Handbook version6.

The risk of bias in at least onespecific result for everyincluded study must beexplicitly considered todetermine the extent to whichits findings can be believed,noting that risks of bias mightvary by result. The RoB 2 tool –as described in Handbookversion 6– must be used for allrandomized trials in newreviews. This does not preventother tools being used.

See Handbook Section 7.1.2; Chapter 8

C53 Assessing risk of bias induplicate

Mandatory

Use (at least) two peopleworking independently to applythe risk-of-bias tool to each

Duplicating the risk-of-biasassessment reduces both therisk of making mistakes and the


Page 19/26

included study, and define inadvance the process forresolving disagreements.

possibility that assessments areinfluenced by a single person’sbiases.

C54 Supporting judgements of risk

of biasMandatory

Justify judgements of risk ofbias (high, low and someconcerns) and provide thisinformation in the risk-of-biastables (as ‘Support forjudgement’).

Providing support for thejudgement makes the processtransparent.


C55 Providing sources ofinformation for risk of biasassessments

Mandatory

Collect the source ofinformation for each risk of biasjudgement (e.g. quotation,summary of information from atrial report, correspondencewith investigator etc.). Wherejudgements are based onassumptions made on the basisof information provided outsidepublicly available documents,this should be stated.

Readers, editors and refereesshould have the opportunity tosee for themselves from wheresupports for judgements havebeen obtained.


C56 Ensuring results of outcomesincluded in ‘Summary offindings’ tables are assessedfor risk of bias

An updateto this Standard is pending

Mandatory

Ensure that assessments of riskof bias cover the outcomesincluded in the ‘Summary offindings’ table.

It may not be feasible to assessthe risk of bias in every singleresult available across theincluded studies, particularly ifa large number of studies andresults are available. Reviewauthor should strive to assessrisk of bias in the results ofoutcomes that are mostimportant to patients. Suchoutcomes will typically beincluded in ‘Summary offindings’ tables, which presentthe findings of seven or fewerpatient-important outcomes.


C57 Summarizing risk-of-biasassessments.

Highly desirable

Summarize the risk of bias foreach key outcome for eachstudy

This reinforces the link betweenthe characteristics of the studydesign and their possibleimpact on the results of thestudy and is an importantprerequisite for the GRADEapproach to assessing the

See Handbook Section 7.5; Chapter 8

Page 20/26

certainty of the body ofevidence.

C58 Addressing risk of bias in thesynthesis.

Highly desirable

Address risk of bias in thesynthesis (whether quantitativeor non-quantitative). Forexample, present analysesstratified according to summaryrisk of bias, or restricted tostudies at low risk of bias.

Review authors should considerhow study biases affect results.This is useful in determining thestrength of conclusions andhow future research should bedesigned and conducted.


C59 Incorporating assessments ofrisk of bias.

Mandatory

If randomized trials have beenassessed using one or moretools in addition to the RoB 2tool, use the RoB 2 tool as theprimary assessment of bias forinterpreting results, choosingthe primary analysis, anddrawing conclusions.

For consistency of approachacross Cochrane InterventionReviews, the RoB 2 tool shouldtake precedence when two ormore tools are used forassessing risk of bias inrandomized trials. The RoB 2tool also feeds directly into theGRADE approach forassessing the certainty of thebody of evidence.


C60 Addressing conflicts of interestin included trials.

Highly desirable

Address conflict of interests inincluded trials, and reflect onpossible impact on: a)differences in study design; b)risk of bias in trial result, and c)risk of bias in synthesis result

Review authors should considerassessing whether they judge atrial to be of “notable concernabout conflicts of interest”. Thisassessment is useful forexploration of possibleheterogeneity between trials(e.g. in a subgroup analysis),and for reflection on relevantmechanisms for how conflict ofinterest may have biased trialresults and synthesis results.Concerns about conflicts ofinterest can be reported in the‘Characteristics of includedstudies’ table.


Synthesizing the results of included studies (C61-C73)

Synthesizing the results of included studies

Cochrane Interactive Learning (CIL): module 6 - analysing the data

Standard Rationale and elaboration ResourcesC61 Combining different scales Mandatory

Page 21/26

If studies are combined withdifferent scales, ensure thathigher scores for continuousoutcomes all have the samemeaning for any particularoutcome; explain the directionof interpretation; and reportwhen directions are reversed.

Sometimes scales have higherscores that reflect a ‘better’outcome and sometimes lowerscores reflect ‘better’ outcome.Meaningless (and misleading)results arise when effectestimates with opposite clinicalmeanings are combined.

Cochrane Trainingresource: analysing continuousoutcomes

C62 Ensuring meta-analyses aremeaningful

Mandatory

Undertake (or display) a meta-analysis only if participants,interventions, comparisons andoutcomes are judged to besufficiently similar to ensure ananswer that is clinicallymeaningful.

Meta-analyses of very diversestudies can be misleading, forexample where studies usedifferent forms of control.Clinical diversity does notindicate necessarily that a meta-analysis should not beperformed. However, authorsmust be clear about theunderlying question that allstudies are addressing.

Cochrane Training resources: intro to meta-analysis and exploring heterogeneity

C63 Assessing statisticalheterogeneity

Mandatory

Assess the presence andextent of between-studyvariation when undertaking ameta-analysis.

The presence of heterogeneityaffects the extent to whichgeneralizable conclusions canbe formed. It is important toidentify heterogeneity in casethere is sufficient information toexplain it and offer new insights.Authors should recognize thatthere is much uncertainty inmeasures such as I2 and Tau2

when there are few studies.Thus, use of simple thresholdsto diagnose heterogeneityshould be avoided.


Cochrane Training resource: exploring heterogeneity

C64 Addressing missing outcomedata

Highly desirable

Consider the implications ofmissing outcome data fromindividual participants (due tolosses to follow-up orexclusions from analysis).

Incomplete outcome data canintroduce bias. In mostcircumstances, authors shouldfollow the principles of intention-to-treat analyses as far aspossible (this may not beappropriate for adverse effectsor if trying to demonstrateequivalence). Risk of bias dueto incomplete outcome data isaddressed in the Cochrane'riskof-bias' tool. However,statistical analyses and carefulinterpretation of results areadditional ways in which theissue can be addressed byreview authors. Imputationmethods can be considered(accompanied by, or in the formof, sensitivity analyses).


Cochrane Training resources: assessing RoB included studiesand RoB 2.0 webinar

Page 22/26

C65 Addressing skewed data Highly desirable Consider the possibility and

implications of skewed datawhen analysing continuousoutcomes.

Skewed data are sometimesnot summarized usefully bymeans and standarddeviations. While statisticalmethods are approximatelyvalid for large sample sizes,skewed outcome data can leadto misleading results whenstudies are small.


Cochrane Training resource: analysing continuous outcomes

C66 Addressing studies with morethan two groups

Mandatory

If multi-arm studies areincluded, analyse multipleintervention groups in anappropriate way that avoidsarbitrary omission of relevantgroups and double-counting ofparticipants.

Excluding relevant groupsdecreases precision and double-counting increases precisionspuriously; both areinappropriate and unnecessary.Alternative strategies includecombining intervention groups,separating comparisons intodifferent forest plots and usingnetwork meta-analysis.

See Handbook Section 6.2.9and Chapter 11.

Cochrane Training resource: analysing non-standard data &study designs

C67 Comparing subgroups Mandatory If subgroup analyses are to be

compared, and there arejudged to be sufficient studiesto do this meaningfully, use aformal statistical test tocompare them.

Concluding that there is adifference in effect in differentsubgroups on the basis ofdifferences in the level ofstatistical significance withinsubgroups can be verymisleading.

See Handbook Section10.11.3.1

Cochrane Trainingresources: exploringheterogeneity and commoninterpretation errors

C68 Interpreting subgroup analyses Mandatory If subgroup analyses are

conducted, follow the subgroupanalysis plan specified in theprotocol without undueemphasis on particular findings.

Selective reporting, or over-interpretation, of particularsubgroups or particularsubgroup analyses should beavoided. This is a problemespecially when multiplesubgroup analyses areperformed. This does notpreclude the use of sensibleand honest post hoc subgroupanalyses.


Cochrane Training resources: exploring heterogeneity and common interpretation errors

C69 Considering statisticalheterogeneity when interpretingthe results

Mandatory

Take into account anystatistical heterogeneity wheninterpreting the results,particularly when there isvariation in the direction ofeffect.

The presence of heterogeneityaffects the extent to whichgeneralizable conclusions canbe formed. If a fixed-effectanalysis is used, the confidenceintervals ignore the extent ofheterogeneity. If a random-effects analysis is used, theresult pertains to the meaneffect across studies. In bothcases, the implications ofnotable heterogeneity should be



Page 23/26

addressed. It may be possibleto understand the reasons forthe heterogeneity if there aresufficient studies.

C70 Addressing non-standarddesigns

Mandatory

Consider the impact on theanalysis of clustering, matchingor other non-standard designfeatures of the included studies

Cluster-randomized trials, cross-over trials, studies involvingmeasurements on multiple bodyparts, and other designs needto be addressed specifically,since a naive analysis mightunderestimate or overestimatethe precision of the study.Failure to account for clusteringis likely to overestimate theprecision of the study,that is, togive it confidence intervals thatare too narrow and a weightthat is too large. Failure toaccount for correlation is likelyto underestimate the precisionof the study, that is, to give itconfidence intervals that are toowide and a weight that is toosmall.


Cochrane Training resource: non-standard study designs

C71 Sensitivity analysis Highly desirable Use sensitivity analyses to

assess the robustness ofresults, such as the impact ofnotable assumptions, imputeddata, borderline decisions andstudies at high risk of bias.

It is important to be aware whenresults are robust, since thestrength of the conclusion maybe strengthened or weakened.



C72 Interpreting results Mandatory Focus interpretation of results

on estimates of effect and theirconfidence intervals, avoidinguse of a distinction between“statistically significant” and“statistically non-significant".

Authors commonly mistake alack of evidence of effect asevidence of a lack of effect.


Cochrane Training resource: common interpretation errors


C73 Investigating risk of bias due tomissing results

Highly desirable

Consider the potential impact ofnon-reporting biases on theresults of the review or the meta-analyses it contains.

There is overwhelmingevidence of non-reportingbiases of various types. Thesecan be addressed at variouspoints in the review. A thoroughsearch, and attempts to obtainunpublished results, mightminimize the risk. Analyses ofthe results of included studies,for example using funnel plots,can sometimes help determinethe possible extent of theproblem, as can attempts toidentify study protocols, whichshould be a routine feature ofCochrane Reviews.


Cochrane Training resources: small study effects & reportingbiases


Page 24/26

Assessing the quality of evidence and summarizing the findings (C74-C75)

Assessing the quality of evidence and summarizing the findings

Cochrane Training resource: GRADE approach to evaluating evidence quality

Cochrane Interactive Learning: module 7 - interpreting the findings

Standard Rationale and elaboration ResourcesC74 Assessing the certainty of the

body of evidenceMandatory

Use the five GRADEconsiderations (risk of bias,consistency of effect,imprecision, indirectness andpublication bias) to assess thecertainty of the body ofevidence for each outcome,and to draw conclusions aboutthe certainty of evidence withinthe text of the review.

GRADE is the most widely usedapproach for summarizingconfidence in effects ofinterventions by outcomeacross studies. It is preferableto use the online GRADEprotool, and to use it as describedin the help system of thesoftware. This should help toensure that author teams areaccessing the same informationto inform their judgments.Ideally, two people workingindependently should assessthe certainty of the body ofevidence and reach aconsensus view on anydowngrading decisions. Thefive GRADE considerationsshould be addressedirrespective of whether thereview includes a ‘Summary offindings’ table. It is helpful todraw on this information in theDiscussion, in the Authors’conclusions and to convey thecertainty in the evidence in theAbstract and Plain languagesummary.


Common issues in Summary ofFindings tables.

Planning GRADE andSummary of Findings tables.

Incorporating GRADE inCochrane Reviews.

C75 Justifying assessments of thecertainty of the body ofevidence

Mandatory

Justify and document allassessments of the certainty ofthe body of evidence (forexample downgrading orupgrading if using GRADE).

The adoption of a structuredapproach ensures transparencyin formulating an interpretationof the evidence, and the resultis more informative to the user.


Reference

Reference

Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. (2014) Better reporting of interventions: template forintervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687. doi: 10.1136/bmj.g1687

Page 25/26

Citation

Citation of the Standards for the conduct of new Cochrane Intervention Reviews

Please cite this section as: Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R. Standards for theconduct of new Cochrane Intervention Reviews. In: Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E,Churchill R. Methodological Expectations of Cochrane Intervention Reviews. Cochrane: London, February 2021.

Powered by TCPDF (www.tcpdf.org)

Page 26/26

http://www.tcpdf.org

Documents

Standards for the CONDUCT of new Cochrane Intervention