Guide for PH Protocol_Nov 2011_final for Website

The Cochrane Public Health Group

Guide for developing a Cochrane protocol

This Guide has been produced by the Cochrane Public Health Group (CPHG) to make the process of preparing a protocol for a public health review in Cochrane format as clear as possible. The editorial base has developed guidelines that detail each section of the protocol and what should be included. These guidelines are a distillation of the Cochrane Handbook for Systematic Reviews of Interventions, review group policies, recommendations from the CPHG Editorial Team and also incorporate the Methodological Standards for the Conduct of Cochrane Intervention Reviews, released in November 2011. Please refer to The Cochrane Collaboration Training website at http://training.cochrane.org/ to access online training modules for Cochrane review authors.

http://training.cochrane.org/

Last updated: 24 November 2011 - 2 -

Table of contents

TABLE OF CONTENTS ............................................................................................................................................. - 2 -

GETTING STARTED................................................................................................................................................. - 3 -

FILLING OUT THE PROTOCOL TEMPLATE ............................................................................................................... - 4 -

BACKGROUND ....................................................................................................................................................... - 4 - Description of the condition .......................................................................................................................... - 4 - Description of the intervention ...................................................................................................................... - 4 - How the intervention might work .................................................................................................................. - 4 - Why it is important to do this review ............................................................................................................. - 5 -

OBJECTIVES .......................................................................................................................................................... - 6 - Resources ..................................................................................................................................................... - 6 -

METHODS ............................................................................................................................................................ - 6 - Criteria for considering studies for this review ............................................................................................... - 6 - Types of studies ............................................................................................................................................ - 6 - Types of participants..................................................................................................................................... - 7 - Types of interventions ................................................................................................................................... - 7 - Types of outcome measures .......................................................................................................................... - 7 -

SEARCH METHODS ................................................................................................................................................. - 9 - DATA COLLECTION AND ANALYSIS ............................................................................................................................. - 10 -

Selection of studies ..................................................................................................................................... - 10 - Data extraction and management .............................................................................................................. - 10 - Assessment of risk of bias in included studies .............................................................................................. - 12 - Measures of treatment effect ..................................................................................................................... - 13 - Unit of analysis issues ................................................................................................................................. - 14 - Dealing with missing data ........................................................................................................................... - 14 - Assessment of heterogeneity ...................................................................................................................... - 15 - Assessment of reporting biases ................................................................................................................... - 15 - Data synthesis ............................................................................................................................................ - 16 - Subgroup analysis and investigation of heterogeneity ................................................................................. - 17 - Sensitivity analysis ...................................................................................................................................... - 17 -

ACKNOWLEDGEMENTS .......................................................................................................................................... - 17 - CONTRIBUTIONS OF AUTHORS ................................................................................................................................. - 18 - POTENTIAL CONFLICT OF INTEREST ............................................................................................................................ - 18 - REFERENCES - OTHER REFERENCES ........................................................................................................................... - 18 -

Additional references .................................................................................................................................. - 18 - Other published versions of the review ........................................................................................................ - 19 -

REFERENCES ....................................................................................................................................................... - 20 -

APPENDIX 1 – GUIDANCE FOR ESTABLISHING A REVIEW ADVISORY GROUP ....................................................... - 21 -

APPENDIX 2 - PUBLIC HEALTH GROUP EDITORIAL PROCESS ................................................................................ - 24 -

APPENDIX 3 – RISK OF BIAS ASSESSMENT ........................................................................................................... - 26 -

APPENDIX 4 – DATA EXTRACTION AND ASSESSMENT FORM ............................................................................... - 29 -

APPENDIX 5 – CHECKLIST FOR SUBMISSION ........................................................................................................ - 48 -


Getting started

Now that your title has been registered there are several resources you will need to help you to complete your protocol. These are listed below and we have provided relevant links throughout these documents. All are available at www.cochrane.org and www.ims.cochrane.org. If you have problems downloading these files please contact the Review Group Coordinator. You should also have received an email notifying you that a user account has now been set up for you in Archie. Once you have activated this account you will be able to check out your review.

1) Cochrane Handbook for Systematic Reviews of Interventions The Cochrane Handbook explains the Review process (parts have been summarised in this document but you will need to refer to The Cochrane Handbook for more complete guidance) (Higgins and Green 2008). The handbook is incorporated in the RevMan 5.1 software and also available at http://www.cochrane-handbook.org. This document includes a chapter specific to public health reviews (Chapter 21).

2) RevMan 5.1 This is the program which you must use to write your protocol and later your review. It is free software that you can download easily to your computer. A tutorial is accessible from the help menu in RevMan. ALL AUTHORS ARE ASKED TO WORK THROUGH THE REVMAN TUTORIAL BEFORE WORKING ON YOUR PROTOCOL.

3) RevMan user guide This guide can be accessed as a PDF from the help menu in RevMan and provides a detailed description about how to use RevMan. You should continue to refer to this user guide throughout the review production process.

4) Cochrane style guide This guide explains the style conventions for a Cochrane review. www.cochrane.org/training/authors-mes/cochrane-style-guide

5) Other resources There are a number of other resources available to support you. Many are listed on the Cochrane Public Health Group website (http://ph.cochrane.org/resources-and-guidance)

6) Advisory group The CPHG recommends that all review teams establish a Review Advisory Group (RAG) to advise them on the scope of their review and its relevance to end-users. One author, usually the lead author, should take responsibility for establishing the RAG prior to commencing work on the protocol. The CPHG can provide you with assistance in identifying relevant RAG members but ultimately it is the author team’s responsibility. Appendix 1 outlines guidance for establishing and working with a RAG

http://www.cochrane.de/

http://www.cochrane-handbook.org/

http://www.cochrane.org/style/home.htm

http://ph.cochrane.org/resources-and-guidance


Filling out the protocol template

This document provides advice on what to include in each of the sections presented to review authors in a RevMan protocol template. The headings described below will align with the set headings of the protocol template. The advice reflects CPHG editorial policy and following these guiding principles should ensure fewer modifications are requested at editorial revision stage. See Appendix 2 for an overview of the stages each review goes through prior to publication. You should also refer to Chapter 4 of the Cochrane Handbook as you complete the protocol. This document does not repeat information in the Cochrane Handbook but rather provides supplementary material specific to CPHG requirements.

Background

Description of the condition

This section should describe the condition or issue that the intervention/s under review are aiming to address, including information on the historical, (and perhaps political), social, economic, geographical and biological perspectives of the problem or issue so as to set a context for the review. This will help to establish the rationale for the review and explain the importance of the questions being asked. It may be appropriate to change this heading to ‘Description of the Issue’.

Description of the intervention

Define all terms and interventions clearly and try to set a tone that does not pre-judge the value of the intervention (i.e. the likely effectiveness of the intervention/s). Provision of examples of interventions and their components here will help the reader gain a better understanding of the interventions the authors refer to thereafter in the “Types of Interventions” of the Methods section. The complexity of the intervention in seeking to address the problem or issue (especially if it is delivered in many different contexts, using different methods and tools or includes many interventions to meet the desired outcome/s) needs to be acknowledged, even if the review is only looking at a component of the problem or issue. Areas of uncertainty about the intervention and issues that may be controversial or the subject of public concern should be highlighted. While this section may require you to cover technicalities of the intervention, it is important to write this clearly and in plain language to aid reader comprehension. We recommend including definitions of key concepts in this section.

How the intervention might work

In this section identify the theoretical underpinnings and refer to literature that identifies a potential pathway of effect between intervention and outcomes. You might like to include a logic model here which shows the connections between determinants of health, interventions and outcomes. Logic models can also help to identify the interventions of interest and important outcomes, or provide a logical rationale for why only a component of an intervention is being reviewed (and point to where other reviews may need to be carried out to complete the evidence picture). The following diagram from the published protocol for Community-based


interventions for enhancing access to and/or consumption of fruits and vegetable access among 5-18 year olds (Ganann et al 2010) is an example of how a logic model might be constructed.

Why it is important to do this review

Clearly describe the justification for doing this review. Cochrane reviews should include a focus on exploring uncertainty about effects and/or uncertainty about effects within particular subgroups. You should include supporting references to existing and ongoing primary research and reviews (including non-Cochrane reviews) of the research topic, to highlight what has been learned from past efforts as well as to point out any inconsistencies, methodological strengths and weaknesses, and evidence ‘gaps’ that still remain. The contribution of your planned review should be emphasised by clearly stating the unresolved questions and controversies that will be addressed. To instruct the end-user on the potential application of review findings, include a brief statement on how this could inform practice or policy decisions. This section should be comprehensive but concise (1-2 paragraphs per subsection). Resources

Cochrane Handbook - Chapters 4 and 5

The Cochrane Public Health Group Training Handbook (http://ph.cochrane.org/resources-and-guidance) – see How to Ask an Answerable Question




Objectives

The objective/s of the review should be clear and specific with a precise statement that covers both intervention/s or exposure/s to be reviewed, the population of interest (e.g. whole population or particular sub-group/s) and the types of outcome measures of interest. One sentence may be enough; however, for some review questions it may be more appropriate to break it down into primary and secondary objectives. The intention to identify and report on adverse consequences or effects of the intervention/s in the review also should be included in this section. The objectives should be consistent with the review title.

Resources

Cochrane Handbook – Chapter 4

Methods

The purpose of the methods sections of a protocol is to describe how the review will be conducted and to theoretically allow for the review to be replicated by others. Please refer to the Cochrane-Campbell Methods Group Equity Checklist

(http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf) throughout the development of your protocol. This highlights methodological aspects to consider at both the protocol and the review stage to help you identify the impact of interventions on equity in your review.

Criteria for considering studies for this review

This section should make it clear how studies will be selected for inclusion in the analysis. All inclusion/exclusion criteria must be explicitly stated within this section, even if mentioned again elsewhere throughout your protocol. The protocol needs to reflect your intention to include information that will allow the review author, and hence the reader, to judge what factors may increase or decrease effectiveness, and under what circumstances/contexts. You do not need to provide any text under this heading. Rather complete each of the sections below. Review authors should note that if non-randomised studies are to be included detailed descriptions of your approach will be needed throughout your protocol. Suggestions are provided throughout this guidance document. Note that any changes to eligibility criteria or outcomes studies will need to be justified in the review.

Types of studies

In this section you need to list the study designs you will include in your review. Whilst you do not need to document your study selection in detail, study design/s chosen should be determined by, and appropriate for, the intervention and the research question. You should consider the risk of bias of each study design and value added to the review. Clear justification for choice of eligible study designs must be included. Randomised controlled trials should be included if they are feasible for the interventions and outcomes of interest. If an RCT is not able to be used as a study

http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf


design due to ethical or other reasons the review needs to be explicit about which study designs are appropriate to include and why. It may be appropriate to include more than one study design, even if it is likely that RCTs are available, to increase external validity of the review findings. Studies should be included irrespective of their publication status, unless explicitly justified in your protocol. The CPHG accepts the following study designs as appropriate to the review question: RCTs, cluster RCTs, non-randomised controlled studies (including controlled before and after studies and interrupted time series studies (with three time points before and after the intervention). Whilst there are challenges associated with using study labels, authors are encouraged to refer to Box 13.1.a of the Cochrane Handbook to ensure they are clear about the types of study design descriptors they might find in the literature. You could consider including a list of definitions of your chosen study designs. The focus for eligibility must be on the features of a study’s design, rather than the design labels used. You should state in this section if you will be collecting and extracting information from qualitative studies emanating from included intervention studies. Qualitative research studies can be used in the review to help contextualize the major findings, rather than provide causative understandings. The extent to which you search for and include qualitative studies will depend on the questions you wish to answer in your review.

Types of participants

Clearly describe the population for which the intervention under review is targeted at or likely to have an effect on, and clearly specify inclusion and exclusion criteria for your review. These may be based on individuals or communities. Examples of defining characteristics may be geographic (e.g. where they live or work), demographic (e.g. age, sex) and/or social factors (e.g. education level, ‘at-risk’ groups (as specified by the authors). Any restrictions with respect to specific population characteristics, settings or factors needs to be justified and reflect information presented in the Background section. Many reviews published within the remit of the CPHG will have community as their participants. Reviews should provide a definition for community that is appropriate for the intervention. It is important to distinguish between ‘community-wide’ and ‘community-based’ interventions, if these terms are used in your review.

Types of interventions

List the intervention/s that will be included in the review, and specify clear inclusion and exclusion criteria (e.g. any essential components, minimum duration of the intervention, etc.). This section should not describe or define interventions - this detail should be included in the Background section (Description of the intervention). This section should therefore be brief and outline the types or groups of included interventions, as referred to in the Background section. Authors should also include a list of the interventions that will be excluded from the review. Include specific examples of relevant interventions/programs of both included and excluded studies when possible. If relevant to the inclusion criteria, identify any specific intervention/s your intervention/s of interest should be compared against (i.e. what the control group will be).

Types of outcome measures

The outcomes you plan to report for your review should be pre-specified in your review to avoid bias in reporting your findings. If possible, you should also define in advance the details of what will be considered acceptable outcome measures for your review and also how outcome


measures will be selected when there are several possible measures (e.g. multiple definitions, assessors or scales). You should include all important outcomes in the protocol, even if you believe there will be insufficient data present in the existing research to include in the synthesis. As well as potential benefits, you must consider any potential adverse consequences of the intervention and ensure that these are addressed in your review. Public health questions often have a large number of outcomes of interest – we encourage authors to think carefully about what the main outcomes of interest are likely to be for end-users. Once you have a complete list of the outcomes of interest, you should identify a short list of main outcomes. The Cochrane Handbook suggests no more than seven main outcomes should be chosen, however this should be examined on an individual review basis. These outcomes will be used to summarise the key findings of your completed review, for example in the abstract and Summary of Findings tables (if you include this in your review). For some review questions, it may be appropriate to summarise findings for groups of similar outcomes. From among the main outcomes, select a small number of primary outcomes (usually 3 or less). A primary outcome is of core interest to your review – i.e. the outcome that best answers your effectiveness question. You should include at least one undesirable outcome (adverse effect), known or hypothesised, with a rationale for why you think these may arise and why they are important to include in the review. All other main outcomes become secondary outcomes, along with any other additional outcomes the review intends to address. These may include unintended outcomes that occur as a result of the intervention. These are considered helpful in explaining intervention effects or the integrity of the intervention. If you have a large number of secondary outcomes you should consider the most appropriate ways of categorising these. If you are planning to include a Summary of Findings table in your review (not compulsory but strongly recommended in The Cochrane Handbook), you should state the outcomes you will be prioritising for this here. For public health reviews, it may often be appropriate to include measures of changes in social and environmental determinants of population health, as opposed to or in addition to direct health outcomes. For example, changes in children’s purchasing habits (as a result of new canteen policies in a school), and reduction in violent behaviour (as a result of adoption of responsible drinking policies at a sports club). You must clarify in advance whether any outcomes will be used as criteria for including studies (rather than as a list of the outcomes of interest within included studies). Outcome measure often do not necessarily form part of the criteria for including studies in a review, however some reviews do legitimately restrict eligibility to specific outcomes. Furthermore, if relevant to your topic, you may wish to specify requirements for the timing of outcome measurement, such as a minimum period after the intervention before which an effect would not be detectable (e.g. the effect of an intervention in pregnant women on the rate of babies born with low birth weight could not be observable for several months after the intervention). Resources

Cochrane Handbook – Chapters 4, 5, 13 and 21


Chapter 21 in Cochrane Handbook on Reviews in Health Promotion and Public Health (www.cochrane-handbook.org)

The Cochrane Non-Randomised Studies Methods Group has provided information on how non-randomised studies should be dealt with in Cochrane reviews (see Chapter 13 in Cochrane Handbook (www.cochrane-handbook.org).

Cochrane Consumers and Communication Group have useful information on study designs at :

www.latrobe.edu.au/cochrane/assets/downloads/StudyQualityGuide050307.pdf )

Refer to Cochrane Health Equity website for information on assessing issues relevant to equity, along with a checklist for authors that can be used and submitted along with the draft protocol (http://equity.cochrane.org)

Search methods

Include a description of the search strategy to be used to retrieve studies. You should list the sources to be searched (e.g. reference databases, personal contacts, hand searches of journals). At a minimum you should search Medline, Embase and CENTRAL. Include a rationale for the choice of literature sources (e.g. ERIC for reviews of educational interventions) if possible and the years to be covered. It may be useful to refer to your logic model to justify your approach. Consider including databases relevant for health equity. Relevant trial/study registers and repositories of results must also be included in your search to reduce the risk of publication bias and identify ongoing studies. Reference lists of included studies and any relevant systematic reviews identified must also be searched and this should be noted in your protocol. You should also describe the mechanisms you intend to use or resources that you have available to enable you to retrieve potentially relevant documents, especially ones that are unpublished and/or written in languages other than English. For example, you may plan to contact relevant individuals and organisations for information about unpublished or ongoing studies. You should not apply inclusion restrictions based on publication status or language. The CPHG can assist authors in identifying potential options for translations if they are required. You should consider mechanisms for searching grey literature, for example using databases such as OpenSIGLE. If searching restrictions are necessary, a rationale must be provided. You should at least make a first attempt at a search strategy (ideally utilising an information specialist available to the review author team). The CPHG Trials Search Coordinator, Ruth Turley ([email protected]) will provide feedback once you have submitted your protocol for editorial review. Your protocol should include (as an appendix) a complete electronic search strategy for one database (usually Medline). Consider including terms or concepts in your search strategy that are relevant for health equity. Note that generally CPHG review authors will need to conduct their own searches. We therefore recommend that an information specialist be recruited to the author team. You should outline in your protocol who will conduct the searches. Resources

Cochrane Handbook – Chapters 6 and 21




http://www.latrobe.edu.au/cochrane/assets/downloads/StudyQualityGuide050307.pdf

mailto:[email protected]



Data collection and analysis

Selection of studies

This section should outline how results of the various searches will be assessed for inclusion, by whom and what you will do if more information is required to determine eligibility (e.g. contacting the authors for additional information or finding a process paper associated with the study which outlines further information). Specify how many review authors will independently review all abstracts and titles and how disputes regarding inclusion will be resolved (there must be at least two, and we recommend a third being available to resolve disputes). Specify that all irrelevant titles should be excluded and that full-text papers will be obtained where titles are deemed to be relevant or where eligibility is unclear. Again, state how many review authors will assess these full text papers and how disagreement will be resolved. You should note that you will keep a record of reasons for excluding studies. Documentation regarding inclusion decisions must be sufficient to complete a PRISMA flow chart and a table of ‘Characteristics of excluded studies’. If any statistics will be used to determine inter-rater agreement (e.g. using Cohen's kappa) include these details. Please indicate if articles in a language other than English will be translated. The reference management software you will be using to manage the records retrieved from searches of electronic databases should be named. The most commonly used software programs are Endnote and Reference Manager.

It is common for included studies to refer to a process evaluation or other methodological detail that is published elsewhere in a separate paper. These additional papers must be obtained (when referred to in the primary paper) and considered as part of the included study. They are likely to contain important information needed to understand the implementation of the intervention and adequately assess the risk of bias of the study. Multiple reports of the same study must be collated, so that each study (rather than each paper) is the unit of interest in the review.

Data extraction and management

This section should provide an adequate description of methods used to collect data from included studies. You should mention how many people will conduct data extraction and how any disagreements will be resolved. It is highly desirable that that study characteristics are extracted in duplicate (i.e two review author’s extract data and a third review author be consulted where there is disagreement), but not essential. It is essential that outcome data is extracted in duplicate. You also need to identify the data extraction form/methods you will be using. Appendix 4 provides a sample data extraction form that you can modify to meet the needs of your review. It is recommended that you pilot your data extraction form, to ensure that all participating authors are retrieving comparable results, and this should be noted in the protocol. Inclusion of the data extraction form that your author team has developed would be useful to include as additional information in a figure or additional table.

In the text of the protocol you should include a brief description of the categories of data you intend to collect. In order to provide essential detail for decision makers, information about context, implementation factors, equity, cost and sustainability must be collected from included studies where available (in addition to data on effectiveness). As mentioned in the above section, included studies may refer to a process evaluation or other methodological detail that is


published elsewhere. These additional papers must be obtained and relevant information extracted, as a minimum requirement. In this section of the protocol you should state that you will collect this information (i.e. context, implementation, cost, sustainability etc) and what you will do with it thereafter. i.e. at a minimum, this information should be reported in the Characteristics of included studies table. If you plan to use this information to help synthesise your findings (for example, grouping studies by measure of intensity or level of implementation), you should state this in your protocol (with the caveat that this would only be done if adequate data was available). You may also choose to conduct additional searches for contextual information, such as implementation factors, cost and sustainability, beyond that linked with included studies. Any plans to do this should be noted in your protocol.

Within your data extraction form, you should identify studies that report on socio-demographic characteristics known to be important from an equity perspective. For this process, use the PROGRESS (Place, Race, Occupation, Gender, Religion, Education, Socioeconomic status, Social status) framework and, at a minimum, report for each included study which of the eight PROGRESS factors were reported for participants at baseline and which were reported at endpoint. In addition to the PROGRESS framework, also collect whether or not interventions included particular strategies to address diversity or disadvantage. We strongly recommend that you incorporate the Cochrane-Campbell Methods Group Equity Checklist in your data extraction form and you should note this in your protocol (http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf). This may help you to identify the impact of interventions on equity later on in your analysis, if this is an important consideration for your review. In this section you should state that all potential moderators/confounders of study outcomes will be included in the extraction form (even if some of these characteristics are not expected to be formally tested or discussed in the final review). Consider potential confounders and adjustment processes. When extracting data, include this information. It can be helpful to refer to The Quality Assessment Tool for Quantitative Studies (www.myhamilton.ca/nr/rdonlyres/6b3670ac-8134-4f76-a64c-9c39dbc0f768/0/qatool.pdf), developed by The Effective Public Health Practice Project (EPHPP), to be used in conjunction with their Quality Assessment Tool for Quantitative Studies Dictionary (www.myhamilton.ca/nr/rdonlyres/f5944f3b-15a9-46e7-8afd-1cd67628e33d/0/qadictionary.pdf) to check for items that you may wish to consider including as part of your data extraction form. If you use items from this tool, you need to state this in your protocol. Document how you intend to handle instances in which a single study of effectiveness provides data on multiple measures of the same or similar outcomes and you may need to choose what to report (e.g. when several variations on an outcome are measured (e.g. weight and BMI) or when the same outcome is measured at multiple points in time). An explanation of the criteria used to determine whether multiple outcomes from the same or related studies are independent data points should be included. Authors of primary studies should be contacted where information is missing or clarification is needed. You should note this in the later section on Dealing with missing data. You should outline

http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf

http://www.myhamilton.ca/nr/rdonlyres/f5944f3b-15a9-46e7-8afd-1cd67628e33d/0/qadictionary.pdf


how you will attempt to avoid duplicate publication bias and state how multiple reports and if publications of the same study will be assembled and compared for duplication, completeness and possible contradictions. Please note in this section if you will use RevMan to manage data storage and analysis. If you intend to use alternative software you must discuss this with the CPHG, and note your intention in the protocol. Resources



Refer to Cochrane Health Equity website for information on assessing issues relevant to equity, along with a checklist for authors that can be used and submitted along with the draft protocol (http://equity.cochrane.org)

Assessment of risk of bias in included studies

In this section, you should provide an adequate description of the tool(s) you will use to assess the risk of bias of included studies. You should also describe how the tool(s) will be implemented and the criteria used to assign studies, for example, to judgements of low risk, high risk and unclear risk of bias. The risk of bias assessment must be conducted in duplicate with a clear process for resolving disagreements. Supporting information to justify all risk of bias judgements must be included in the risk of bias tables. You can consider including the source of the information, for example, direct quotes from the study paper. If you are only including randomised controlled trials, we recommend that you use the Cochrane Collaboration’s Risk of Bias (RoB) tool. This includes selection bias, performance bias, attrition bias, detection bias and reporting bias. With regard to the assessment of blinding, we recommend that you consider separately the risk of bias due to lack of blinding for (i) participants and study personnel (performance bias) and (ii) outcome assessment (detection bias). It is also often appropriate to consider the risk of bias due to lack of blinding separately for different types of outcomes. When assessing attrition bias, it is recommended to consider the impact of missing data separately for different outcomes. If you are including non-randomised studies we recommend you use the Effective Practice and Organisation of Care (EPOC) RoB Tool for studies with a separate control group instead. This can be used for randomised controlled trials as well as controlled before and after studies and other nonrandomised designs that include a control group (with the exception of interrupted time series studies). This tool includes the standard Cochrane RoB tool items as well as an additional item to consider the likelihood of contamination. Importantly for nonrandomised studies, it also includes additional items to assess the risk of selection bias and subsequent confounding (“were baseline outcome measurements similar?” and “were baseline characteristics similar?”). We recommend supplementing this with another additional item, ”did the study authors appropriately adjust for important confounders in their analysis?”.



If you are including ITS studies we recommend that you assess the risk of bias of these studies using the EPOC RoB tool for ITS study designs which includes four items from the Cochrane 'Risk of bias' tool to assess performance, attrition, detection and reporting bias as well as the following additional items relevant for ITS studies: “was the intervention independent of other changes?”, “was the shape of the intervention effect pre-specified?” and “was the intervention unlikely to affect data collection?”. Both of the EPOC RoB tools mentioned above can be found on the EPOC website: http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/Suggested%20risk%20of%20bias%20criteria%20for%20EPOC%20reviews.doc as well as guidance for how to prepare a RoB table: http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/How%20to%20prepare%20a%20risk%20of%20bias%20table%20for%20reviews%20that%20include%20more%20than%20one%20study%20design.doc. Note that for some items in the RoB tool, for example blinding, it may make sense to provide a different assessment for different outcomes (for example outcome assessors may be blinded for some outcome measures and not others, or some measures may be more objective/subjective than others). Where appropriate, add additional items into the RoB table to allow for this. Alternative approaches should be discussed and agreed with your contact editor. See Appendix 3 for general information on Risk of bias tables (referenced from the Cochrane Handbook). This is an area of methodological development and any new tools will be available on the CPHG website. In this section you also need to describe the method by which you will summarise the risk of bias assessments. Rather than at the study level, it is recommended that you do this at the outcome level. This is due to the fact that the risk of bias may be different for different outcomes within the same study. To do this you can provide an overall risk of bias assessment for the main outcomes within each study, then provide an overall risk of bias assessment for relevant outcomes across studies, so that outcomes will be judged overall as ‘Low’, ‘Medium’ or ‘High’ risk of bias given overall considerations of the study designs, and the potential impact of the identified risks noted in the table for each study that contributed results for that outcome. Also, consider how you will address risk of bias in the synthesis of your results, considering how potential study biases might affect your conclusions.

Measures of treatment effect

In this section you should state how outcomes will be reported (e.g. dichotomous data) and how you will analyse and compare them (e.g. using Risk Ratios). It is likely that a number of quantitative outcome measures may be identified. You need to firstly identity the types of data you will obtain from your included study designs. For controlled studies, with continuous data, we recommend reporting means or changes in mean scores. Weighted mean difference can also be reported for continuous outcomes. Standardised mean differences should be reported when different studies use different scales to report the same outcome (e.g. quality of life scales). Where a number of outcome measures are identified, authors should use the ratio of means method (Friedrich 2008). Dichotomous (or binary) outcomes can expressed as relative risks (RR), odds ratio (OR) or risk difference (RD), however the CPHG recommends using RR (Deeks 2002). We also recommend using RR for categorical data (e.g. outcomes reported on a short Likert

http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/Suggested%20risk%20of%20bias%20criteria%20for%20EPOC%20reviews.doc


http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/How%20to%20prepare%20a%20risk%20of%20bias%20table%20for%20reviews%20that%20include%20more%20than%20one%20study%20design.doc




scale).

Unit of analysis issues

It can be difficult to identify the unit of analysis issues that may emerge from your included studies. However, it is important to consider and document in this section what these might be, based on your included study designs. Special issues in the analysis of studies with non-standard designs, such as cross-over trials and cluster-randomised trials, should be described here. Where these studies are not analysed appropriately this may result in unit of analysis error, for example where cluster-randomised trials are analysed at the individual participant level without taking into account the effect of clustering. Effects can be recalculated using the approximately correct analysis or by inflating standard errors which is equivalent to calculating new sample sizes. If you plan to include these studies you should consult a statistician to identify how unit of analysis errors will be treated, and note this in the protocol. If you find studies with multiple intervention groups, you will need to consider and document how you will deal with multiple groups in your analysis. Report outcomes for all groups of interest in your review, however only include groups that meet the eligibility criteria. If you identify more than one intervention group of interest, you may need to divide your analysis into pair-wise comparisons (e.g. Group A vs control, Group B vs control, Group A vs Group B) and conduct a meta-analysis for each comparison, if appropriate to do so. Be careful not to include a group of participants twice in the same meta-analysis. One way to include two pair-wise comparisons against the same control group in one meta-analysis is to simply halve the number of participants in the control group. Document your intentions regarding these issues in this section. Resources

Cochrane Handbook - Chapter 7 and 9 (data extraction and methods of analysis by type of outcome)


Non-standard designs are discussed in detail in Chapters 9 and 16 of the Cochrane Handbook for Systematic Reviews of Interventions, including cluster-randomised trials (Section 16.3), cross-over trials (Section 16.4), and studies with multiple intervention groups (Section 16.5)

Dealing with missing data

Strategies for dealing with missing data should be described in this section. This will principally include missing information on the methods used in included studies, missing participants due to drop-out (and whether an intention-to-treat analysis will be conducted), and missing statistics (such as standard deviations or correlation coefficients or where data is reported at baseline but is not reported at outcome). It may be necessary for you to contact authors where data are missing. Your strategy for contacting authors (e.g. using email addresses on the study’s publication or accessing phone directories from author’s documented affiliated organisation) should be outlined in the protocol. Section 16.1.2 of the Cochrane Handbook outlines the options available to review authors if missing data is not found. Refer to Section 16.1.2 of The Cochrane



Handbook which outlines the options available to review authors if missing data is not found and state in your protocol which option you will use. The option chosen will depend on what is practical and whether or not it can be assumed that the data is missing at random. Resources

Cochrane Handbook - Chapter 16 (Sections 16.1 and 16.2)

Assessment of heterogeneity

This section should outline how you plan to assess the heterogeneity (or differences) between your included studies. It is helpful to consider and refer in this section to methodological heterogeneity (e.g. how similar or different your included studies are in terms of study design, types of participants, interventions and outcomes), as well as statistical heterogeneity (i.e variability in the intervention effects being evaluated in the different studies) which is a consequence of methodological heterogeneity. If you plan to conduct a meta-analysis you should consider and document how you will assess statistical heterogeneity. You should state that this analysis will be used to determine whether it is suitable to conduct a meta-analysis or to analyse your studies qualitatively. The CPHG recommend that you should use the I2 statistic to quantify the level of heterogeneity present. Review authors may also consider using the Chi square test for heterogeneity (p<0.10). RevMan will calculate these statistics for you. If there is a high level of heterogeneity between studies, it may be inappropriate to conduct a meta-analysis. Meta-analysis should only be considered when a group of studies is sufficiently similar in terms of participants, interventions and outcomes to provide a meaningful summary. If the studies are comparable in these respects, but there remains a very high level of heterogeneity, you may decide not to conduct a meta-analysis, and to present your results in a qualitative analysis. If you plan to use a heterogeneity threshold above which you will not conduct a meta-analysis, you should note this in the protocol. A rough guide to interpretation of the I2 statistic is given in the Cochrane Handbook, however note that thresholds for the interpretation of I2 can be misleading, due to the range of factors that may contribute to heterogeneity. For this reason, statistical assessment of heterogeneity is not a substitute method for exploring the causes of variation between your studies. You should outline the statistical methods you will use for conducting subgroup analyses and investigation of heterogeneity in the section below. Resources


Assessment of reporting biases

Assessment of reporting bias is often difficult in public health reviews where the numbers of studies can be small. If you have a small number of studies (<10) you should follow methods described in the Cochrane Handbook. If, however, a larger number of studies (>10) are available for review, it may be useful to investigate reporting bias using a funnel plot. You should identify both approaches in your protocol as you will not know how many included studies you will find at protocol stage. Resources



Data synthesis

The protocol should outline the procedures you intend to use to analyse and summarise the study results, including whether or not you intend to carry out meta-analyses. This section should contain a clear rationale for any choices, considering the potential impact of each choice on the outcomes of the review. Meta-analyses should only be undertaken if participants, interventions and comparisons are judged to be sufficiently similar to ensure an answer that is meaningful. It may be useful to organise this section by study type and/or by type of outcome data (e.g. continuous or binary). More specifically, if the intention is to carry out a quantitative analysis of results, you should outline in the protocol:

the software package that will be used to conduct the analyses (this should be RevMan and alternatives should be discussed with the CPHG);

how statistics describing the overall literature will be presented; why a particular effect size metric is to be used; if adjustments to effect sizes will be made for any reason; techniques to be used to combine results of separate tests; techniques to be used to assess and then analyse variability in findings

across tests. As a default option, the CPHG recommend using the random-effects model to incorporate heterogeneity into your meta-analyses (as opposed to the fixed-effect model). The random-effects model allows for a greater level of natural heterogeneity between studies, assuming that each study is estimating a unique ‘true’ effect applicable to the time, population and context in which the study was conducted. Fixed effects meta-analysis may be included in subsequent sensitivity analysis or when significant justification is provided in the protocol. For qualitative syntheses (with or without a meta analysis), you need to provide a rationale for how you intend to organise studies/findings to arrive at reasonable conclusions and present information in a useful way for end-users of the review. For example, you may like to synthesise studies grouped by the type of intervention, the length of the intervention, the type of outcome, or by study design. It may be difficult to identify what is most appropriate at protocol stage but we encourage you to report your anticipated approach or to identify the options for analysis and how the decision on synthesis structure will be reached. You must include information about how you will summarise the findings of the review. Use the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness and publication bias) to assess the quality of the body of evidence for each outcome, and to draw conclusions about the quality of the body of evidence within the text of the review. Note that this does not mean you have to provide a GRADE rating for the quality of the body of evidence (high, moderate, low, very low), however you must consider the five factors mentioned above in your review. Be sure to justify and document all assessments so that they can be included in your review. These factors must be addressed irrespective of whether your review will include a ‘Summary of Findings Table’ (SoFT). If you plan to include a formal SoFT in your review, you should include this information in this section of your protocol. It is good practice to list the


outcomes (up to seven) that you plan to include within the SoFT, as well as which comparisons and subgroups will be covered, if appropriate. A SoFT should include the number of participants and studies for each outcome, summarise the intervention effect and include a measure of the quality of the body of evidence addressing the considerations listed above.

Subgroup analysis and investigation of heterogeneity

In this section you should list your proposed sub-group analyses, selecting key factors that may differ between or within your included studies, and that you anticipate will be associated with differences in the effect observed, such as differences in the population, intervention or context of the studies. These should be restricted in number with a rationale provided for each. Issues of equity may be a key consideration. Include information about any formal statistical tests that will be used to compare subgroups, in the event that there are sufficient studies for this to be meaningful. It may be possible to undertake a meta-regression. Please note if this is an option for your review. Resources


Campbell and Cochrane Equity Methods Group Equity Checklist: http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist.pdf

Sensitivity analysis

You should document in this section under what circumstances a sensitivity analysis will be undertaken. Sensitivity analysis is often confused with subgroup analysis – although they are different. Sensitivity analysis is undertaken to identify whether review findings are dependent on the decisions made during the review process, such as about study inclusion/exclusion, the selection of data to analyse, the analysis methods used, imputed data, and the impact of studies at high risk of bias. Some examples of decisions that may indicate a need for sensitivity analysis are listed in The Cochrane Handbook (Chapter 9, section 9.7). Where it is known in advance that decisions have been made that may affect the results of the review, sensitivity analysis should be planned and noted in the protocol. During the course of the review, other study characteristics may also be identified for inclusion in the sensitivity analysis. This is acceptable and should be stated as a possibility in the protocol. Resources Cochrane Handbook – Chapter 9

Acknowledgements

Acknowledge any individuals or organisations who may have made a sufficient contribution to developing the protocol, (e.g. secretarial support, protocol referees, review advisory group members) but are not included in the Contributions of Authors section. Please ensure you have obtained permission to name these people in the protocol, otherwise a general acknowledgment based on roles (‘our advisory group members’, ‘external peer referees of the protocol’). It is not necessary to acknowledge the support of specific people within the CPHG team who have helped

http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist.pdf


you in developing the protocol. A broad acknowledgement of the CPHG team is quite adequate if you wish to acknowledge the group.

Contributions of authors

Identify who has or will perform each of the following tasks (authors’ initials only): Draft the protocol Study selection Extract data from studies Enter data into RevMan Carry out the analysis Interpret the analysis Draft the final review Disagreement resolution Update the review

Potential conflict of interest

You should report any conflict of interest of any of the authors, which may influence their judgments in conducting the review, at protocol stage (refer to the summary of the Collaboration’s policy on conflicts of interest in Chapter 2 (Section 2.6) of the Cochrane Handbook for Systematic Reviews of Interventions for more information). This will include any present or past affiliations or other involvement in any organisation or entity with an interest in the review that might lead to a real or perceived conflict of interest. Areas of uncertainty should always be discussed with the CPHG. Situations that might be perceived by others as being capable of influencing a review author’s judgements include personal, political, academic and other possible conflicts, as well as financial conflicts. Authors must state if they have been involved in a study that may be included in the review. This section should reflect the information contained in authors’ ‘Declarations of interest statement’ within the Disclosure of Potential Conflicts of Interest Forms. You can access these forms from within your Revman file by clicking File -> Reports -> Declaration of Interest (you will receive a reminder from the editorial office to fill out electronically during the course of completing your protocol if you have not done so already). If there are no known conflicts of interest, this should be stated explicitly, for example, by writing ‘None known’.

References - Other references

Additional references

References cited in the text should be listed here. Other reference categories (such as Included Studies) are generally not used in a protocol, but will be used in the completed review. References are referred to throughout the review using a Reference ID, usually comprising the first author’s surname and year of publication (e.g. Smith 2001), and all references must be linked


from the text of the protocol using this ID. You can enter the references individually, or import them from reference management software. Instructions on entering, importing and linked references are available from the Help menu in RevMan. Please run the Citation wizard in Revman, to help identify other Cochrane reviews of potential relevance to your review, to cite where appropriate in the protocol (and review thereafter).

Other published versions of the review

If the review has previously been published (e.g. in a journal or textbook) it should be listed here. Completing our Checklist Appendix 5 is a checklist we would like the author team to complete and send back when submitting their protocol for editorial review. This will help ensure you have followed our guidance and ensure speedier progress through this editorial stage.


References

Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat.Med. 21 (11):1575-1600, 2002.

Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Inj Control Saf Promot. 2003 Mar-Jun;10(1-2):11-2.

Friedrich JO, Adhikari NKJ, Beyene J. The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: A simulation study. BMC Medical Research Methodology 2008;8(32).

Ganann R, Fitzpatrick-Lewis D, Ciliska D, Dobbins M, Krishnaratne S, Beyers J, Fieldhouse P, Delgado Noguera MF, Gauvin FP, Tort S, Hams SP, Martinez-Zapata MJ, Wolfenden L, Bonfill Cosp X, Clay F. Community-based interventions for enhancing access to or consumption of fruit and vegetables (or both) among five to 18-year olds (Protocol). Cochrane Database of Systematic Reviews 2010, Issue 8. Art. No.: CD008644. DOI: 10.1002/14651858.CD008644.

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003; 327(7414):557-60.

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. 2008. Available from www.cochrane-handbook.org.

Kavanagh J, Oliver S, Caird J, Tucker H, Greaves A, Harden A, et al. Inequalities and the mental health of young people: a systematic review of secondary school-based cognitive behavioural interventions. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.; 2009.

National Health Service: Centre for Reviews and Dissemination. Undertaking systematic reviews of research on effectiveness. CRD's guidance for those carrying out or commissioning reviews. CRD Report Number 4 (2nd Edition). 2001 [cited 2007 19 September]; Available from: http://www.york.ac.uk/inst/crd/report4.htm. Rees R, Oliver S, Harden A, Shepherd J, Kavanagh J, Burchett H, et al. Use of an advisory group to ensure relevance: reflections on participation of stakeholders in a review of sexual health promotion for men who have sex with men (MSM), in XII Cochrane Colloquium. 2004: Ottawa, Canada. Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews Evid Based Nurs, 2004. 1(3): p. 176-84. Thompson SG and Pocock SJ. Can meta-analysis be trusted. Lancet 1991 Nov 2;338(8775):1127-30.

http://www.york.ac.uk/inst/crd/report4.htm


Appendix 1 – Guidance for establishing a Review Advisory Group

The Cochrane Handbook suggests the establishment of Review Advisory Groups to help reviewers outline the parameters of their review. The establishment and management of an advisory group can be difficult and potentially time-consuming but it is an important component in the initial stages of review development and it can help establish appropriate review parameters so that the end product reflects the needs of end users. Systematic reviews are likely to be more relevant to the end user and of higher quality if they are informed by advice from people with a range of experiences, in terms of both the topic and the methodology (Rees 2004, Thomas 2004, NHS CRD 2001). This guidance has been developed by the Cochrane Health Promotion and Public Health Field. What is the role of a review advisory group? Review Advisory Groups are established to help reviewers outline the parameters of their proposed review to ensure that the end product reflects the needs of its potential readers and users. Systematic reviews are likely to be more relevant to the end user and of higher quality if they are informed by advice from people with a range of experiences. Examples of opinions sought from the review advisory group:

Does the review question seem to capture the essence of the topic under review (will it sound interesting and useful to its target audience)?

What interventions should be included in the review?

Which populations should be included in the review and which should be excluded?

What types of outcomes should the review include?

How should equity issues be highlighted in the review?

Are the needs of developing countries considered in the review? How do I establish an advisory group? The Cochrane entities with whom you are involved, especially Fields (www.cochrane.org/contact/entities.htm#FIELDLIST), may be a useful source of advisory group members. Many entities keep a database of members tagged with skills, special interests and expertise. You should email relevant entity contacts and request for them to contact relevant members. Alternately, it may be appropriate for you to make contact directly with potential members with known interest or expertise in the topic under review. It is important that you have set up tasks or terms of reference for your advisory group prior to making contact with potential members. This will ensure that roles and responsibilities are known from the outset. Who should the members of the advisory group be? Advisory groups are not intended to be another layer of peer-review. Nor are they intended to assist with technical review support (e.g. statistical expertise). If you require this level of assistance you should contact your review group or consider recruiting an additional reviewer


with these skills to assist you. Advisory Group members should only be used to provide content-related support, highlighting what end users of the review will want to have included in the review. They may be able to direct you to additional studies and/or to provide background information on the topic, particularly within the context of their local situation. This latter point is a good reason why membership of the advisory group should be inclusive of people from different parts of the world, to ensure the end review has relevance globally. . Experience by Effective Public Health Practice Project in Canada suggests 6 members is an appropriate size for the advisory group. The members of your advisory group will vary depending on your review question. However, it may be useful to consider members in the following categories:

Consumers (those with whom the intervention/s under review are targeted)

Content experts

Policy-makers

Practitioners (those implementing the intervention/s under review) Who is responsible for coordinating the advisory group? The lead author should take primary responsibility for coordinating the advisory group and establishing a communication strategy that is acceptable to all (and reflective of the resources available to the review team) . Contact should be made via the lead author in the first instance. Lead authors should cc all authors into correspondence with advisory group members. It may also be appropriate to cc others in (e.g. any Cochrane entities with whom you are involved). What information does the lead author need to provide prospective advisory group members with? Potential members should be provided with adequate details about the review (title, authors etc), preferably before the title is registered with a CRG. They also may need information about the Cochrane Collaboration as all members may not be familiar with Cochrane and the review process. Call for advisory group members should include a clearly defined role, remits and boundaries (potentially a terms of reference document) and timeline of tasks. What processes need to be established for the advisory group to work effectively? To ensure that your advisory group works effectively it is important that you establish roles and responsibilities (You may want to formalise this in a terms of reference document). This will ensure that authors and advisory group members are clear about the role of the advisory group. Again, processes may differ although you should consider the following:

What is the role of each advisory group member (for example, will each answer concerns about their area of expertise or experience only or the whole review)?

What tasks do you want them to complete?

What method of communication will be used and how frequently will the advisory group members be consulted?

What workload is involved?

Are there timelines that need to be considered?

When does the work of the advisory group end (once the parameters for the protocol have been accepted by the registering CRG?)


How will advice be managed and what will happen if conflicting advice (or that quite contrary to the reviewer’s beliefs) is offered?


Appendix 2 - Public Health Group editorial process

The overall process of review development can be seen below: On completion of the draft protocol it should be submitted to the Managing Editor who then organises for an internal review of the protocol by your assigned Contact editor, methods advisor, statistical editor, the trial search coordinator and, if appropriate, the CPHG’s developing countries editorial consultant. All the comments are compiled and sent to the author. Prompt reply and amendment will ensure timely progression to the next stage – external review. Generally 3-4 external referees are recruited (composing of content experts,potential end users and if appropriate and possible, a consumer). Once these referees have agreed to referee the protocol the relevant documents are sent and there is an understanding that the comments should be returned to the CPHG within two weeks. All the comments are compiled by the Contact editor (with guiding comments added) and sent to the author. Again, prompt reply and amendment will ensure quicker inclusion into The Cochrane Library. On resubmission, the Contact editor and the Coordinating Editor will then review the changes. Once the changes have been approved, the protocol will be sent to Wiley Publishes for copy editing and thereafter the protocol marked by the Managing Editor for publication. Thereafter all authors will be sent a Permission to Publish Form, along with the final version of the protocol for approval, to sign. The protocol can NOT be published until this form is signed (electronically) by ALL authors (therefore all authors need to be available to do so before publication deadline).

Registration of title

Submission of

Protocol

(editorial

base) Integration into

Public Health Group module

Publication in

Cochrane Library

Submission of

completed

review

(editorial base)

Review andcomments

6-12

months

3-6

months

4 weeks

6-8 weeks Review and comments

Note: Protocol/review may undergo 2-3 drafts/reviews before acceptance for publication


The same editorial processes apply for review development and publication.


Appendix 3 – Risk of bias assessment

See Table 8.5.c (Criteria for judging risk of bias in the ‘Risk of bias’ assessment tool) in Cochrane Handbook for Systematic Reviews of Interventions.

Extract from Cochrane EPOC Data Collection Checklist for assessing risk of bias in controlled studies and ITS Source:


6.4.1 Risk of bias for studies with a separate control group (RCTs, CCTs, CBAs) Nine standard criteria are used for all RCTs, CCTs and CBAs. Further information can be obtained from the Cochrane handbook section on Risk of Bias and from the draft methods paper on risk of bias under the EPOC specific resources section of the EPOC website. Was the allocation sequence adequately generated? Score “Yes” if a random component in the sequence generation process is described (eg Referring to a random number table). Score ”No” when a nonrandom method is used (eg performed by date of admission). CCTs and CBAs should be scored “No”. Score “unclear” if not specified in the paper. Was the allocation adequately concealed? Score “Yes” if the unit of allocation was by institution, team or professional and allocation was performed on all units at the start of the study; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on-site computer system or sealed opaque envelopes were used. CBAs should be scored “No”. Score “unclear” if not specified in the paper. Were baseline outcome measurements similar?* Score “Yes” if performance or patient outcomes were measured prior to the intervention, and no important differences were present across study groups. In RCTs, score “Yes” if imbalanced but appropriate adjusted analysis was performed (e.g. Analysis of covariance). Score “No” if important differences were present and not adjusted for in analysis.** If RCTs have no baseline measure of outcome, score “Unclear”.** Were baseline characteristics similar? Score “Yes” if baseline characteristics of the study and control providers are reported and similar. Score “Unclear” if it is not clear in the paper (e.g. characteristics are mentioned in text but no data were presented). Score “No” if there is no report of characteristics in text or tables or if there are differences between control and intervention providers. Note that in some cases imbalance in patient characteristics may be due to recruitment bias whereby the provider was responsible for recruiting patients into the trial.

Were incomplete outcome data adequately addressed?*

Score “Yes” if missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the intervention and control groups or the proportion of missing data was less than the effect size i.e. unlikely to overturn the study result). Score “No” if missing outcome data was likely to bias the results. Score “Unclear” if not specified in the paper (Do not assume 100% follow up unless stated explicitly).

Was knowledge of the allocated interventions adequately prevented during the study? *

Score “Yes” if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or question as defined by the authors. Score “No” if the outcomes were not assessed blindly. Score “unclear” if not specified in the paper.




Was the study adequately protected against contamination? Score “Yes” if allocation was by community, institution or practice and it is unlikely that the control group received the intervention. Score “No” if it is likely that the control group received the intervention (e.g. if patients rather than professionals were randomised). Score “unclear” if professionals were allocated within a clinic or practice and it is possible that communication between intervention and control professionals could have occurred (e.g. physicians within practices were allocated to intervention or control) Was the study free from selective outcome reporting? Score “Yes” if there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section). Score “No” if some important outcomes are subsequently omitted from the results. Score “unclear” if not specified in the paper. Was the study free from other risks of bias? Score “Yes” if there is no evidence of other risk of biases

* If some primary outcomes were imbalanced at baseline, assessed blindly or affected by missing data and others were not, each primary outcome can be scored separately. **If “UNCLEAR” or “No”, but there is sufficient data in the paper to do an adjusted analysis (e.g. Baseline adjustment analys is or Intention to treat analysis) the criteria should be re scored to “Yes”.

6.4.2 Risk of bias for interrupted time series studies

Seven standard criteria are used for all ITS studies. Further information can be obtained from the Cochrane handbook section on Risk of Bias and from the draft methods paper on risk of bias under the EPOC specific resources section of the EPOC website. Note: If the ITS study has ignored secular (trend) changes and performed a simple t-test of the pre versus post intervention periods without further justification, the study should not be included in the review unless reanalysis is possible.

Was the intervention independent of other changes? Score “Yes” if there are compelling arguments that the intervention occurred independently of other changes over time and the outcome was not influenced by other confounding variables/historic events during study period. If Events/variables identified, note what they are. Score “NO” if reported that intervention was not independent of other changes in time.

Was the shape of the intervention effect pre-specified? Score ”Yes” if point of analysis is the point of intervention OR a rational explanation for the shape of intervention effect was given by the author(s). Where appropriate, this should include an explanation if the point of analysis is NOT the point of intervention;Score “No” if it is clear that the condition above is not met

Was the intervention unlikely to affect data collection? Score “Yes” if reported that intervention itself was unlikely to affect data collection (for example, sources and methods of data collection were the same before and after the intervention); Score “No” if the intervention itself was likely to affect data collection (for example, any change in source or method of data collection reported).

Was knowledge of the allocated interventions adequately prevented during the study?***

Score “Yes” if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or question as defined by the authors. Score “No” if the outcomes were not assessed blindly. Score “unclear” if not specified in the paper.

Were incomplete outcome data adequately addressed?***


Score “Yes” if missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the pre- and post-intervention periods or the proportion of missing data was less than the effect size i.e. unlikely to overturn the study result). Score “No” if missing outcome data was likely to bias the results. Score “Unclear” if not specified in the paper (Do not assume 100% follow up unless stated explicitly).

Was the study free from selective outcome reporting? Score “Yes” if there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section). Score “No” if some important outcomes are subsequently omitted from the results. Score “unclear” if not specified in the paper. Was the study free from other risks of bias? Score “Yes” if there is no evidence of other risk of biases. e.g. should consider if seasonality is an issue (i.e. if January to June comprises the pre-intervention period and july to December the post, could the “seasons’ have caused a spurious effect). *** If some primary outcomes were assessed blindly or affected by missing data and others were not, each primary outcome can be scored separately.


Appendix 4 – Data Extraction and Assessment Form

This form is to be modified in keeping with the following instructions. Some questions may be changed from open-ended questions to specific data items where appropriate. Refer to the Cochrane Handbook when undertaking modifications to this form.

Sections can be expanded and irrelevant sections can be removed. It is difficult to design a single form that meets the needs of all reviews. It is therefore important that you consider your needs carefully prior to data extraction and pilot your process. Elements within the template are not intended for use as a scoring system, but rather suggests elements which should be addressed in the Table of Included Studies and Comparisons and Data in RevMan. The components of the Risk of Bias Table have been incorporated into this form. If you are using an additional quality assessment tool you will need to add appropriate questions to reflect the additional components. Notes on using a data extraction form:

Pilot the Data Extraction Form you plan on using (and note in your protocol that it will, or has, been piloted)

Be consistent in the order and style you use to describe the information. This will make it easier to complete the Table of Included Studies, prevent you from overlooking information and make reading of the review easier.

Highlight any missing information as unclear or not described, to make it clear to the reader of your review that the information was not included in the description of the study, not that you forgot to extract it.

You should include instructions and decision rules on the data collection form. It is crucial that you practice using the form and receive, or give, training if the form was designed by someone other than the person using it.

Assessment of Risk of Bias has been adapted from Cochrane Handbook Table 8.5.a: The Cochrane Collaboration’s tool for assessing risk of bias. http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/How%20to%20prepare%20a%20risk%20of%20bias%20table%20for%20reviews%20that%20include%20more%20than%20one%20study%20design.doc. Criteria for judging risk of bias as well as examples of appropriate methods of addressing each form of bias are provided in Chapter 8 of the Cochrane Handbook, particularly Table 8.5.c. For tips on how to enter data into RevMan 5, see “Risk of Bias” tables in the RevMan User Guide.





Data Extraction and Assessment Form – template (cut and paste and modify to suit your review)

Study ID: Report ID : Date form completed:

First author: Year of study: Data extractor:

Citation:

1. General Information

Publication type Journal Article Abstract Other (specify e.g. book chapter)___________________

Country of study:

Funding source of study: Potential conflict of interest from funding? Y / N / unclear

2. Study Eligibility

Study Characteristics Page/ Para/ Figure #

Type of study

(Review authors to add/remove designs based on criteria specified in protocol)

Randomised Controlled Trial (RCT)

Cluster Randomised Controlled Trial (cluster RCT)

Controlled Before and After (CBA) study

Contemporaneous data collection

Comparable control site

At least 2 x intervention and 2 x control clusters

Interrupted Time Series (ITS)

At least 1 time point before and 1 after the intervention

Clearly defined intervention point

Other design (specify):

A process evaluation of an included study design

Does the study design meet the criteria for inclusion?

Yes No Exclude Unclear

Description in text:


Participants

(Review authors insert inclusion criteria as defined in Protocol)

Describe the participants included:

Are participants defined as a group having specific social or cultural characteristics?

Yes No Unclear

Details:

How is the geographic boundary defined?

Details:

Specific location (e.g. state / country):

Do the participants meet the criteria for inclusion?


Types of intervention


Strategies included in the intervention

Focus of the intervention

Does the intervention meet the criteria for inclusion?


Duration of intervention

Start date: Stop date: Intervention duration:

Is the duration of intervention adequate for inclusion?




List outcomes:

Outcome measured at a population level or individual level?

Details:

Do the outcome measures meet the criteria for inclusion?



Summary of Assessment for Inclusion

Include in review Exclude from review

Independently assessed, and then compared? Yes No

Differences resolved Yes No

Request further details? Yes No Contact details of authors:

Notes:

DO NOT PROCEED IF PAPER EXCLUDED FROM REVIEW

3. Study details

Study intention Descriptions as stated in the report/paper Page/ Para/ Figure #

Aim of intervention

What was the problem that this intervention was designed to address?

Aim of study

What was the study designed to assess? Are these clearly stated?

Equity pointer: Social context of the study

e.g. was study conducted in a particular setting that might target/exclude specific population s? See also Inclusion/exclusion criteria under Methods, below.

Start and end date of the study

Identify which elements of planning of the intervention should be included

Total study duration

Methods

Descriptions as stated in the report/paper

Page/ Para/ Figure #

Method/s of recruitment of participants (How were potential participants approached and invited to participate? Where were participants recruited from? Does this differ from the intervention setting?)

Inclusion/exclusion criteria for participation in study

Representativeness of sample: Are participants in the study likely to be representative of the target population?

Total number of intervention groups

Assumed risk estimate (e. .baseline or population risk noted in Background)

References:

Sample size calculation: What assumptions were made? Were these assumptions appropriate?

(Yes/No/Unclear)


What was the unit of randomisation? Allocation by individuals or cluster/groups

What was the unit of analysis? Is this the same as the unit of randomisation?

(Yes/No/Unclear)

Statistical methods used and appropriateness of these methods

(Check with your statistician if unsure about appropriateness)

Results Participants Include if relevant

Include information for each group (i.e. intervention and controls) under study

Page/ Para/ Figure #

What percentage of selected individuals agreed to participate?

Total number randomised (or total pop. at start of study for NRCTs)

Number allocated to each intervention group (no. of individuals)

For cluster trials, number of clusters, number of people per cluster

Where there any significant baseline imbalances?

Yes No Unclear

Details:

Number and reason for (and sociodemographic differences of) withdrawals and exclusions for each intervention group

Were patients who entered the study adequately accounted for?

What percentage of patients completed the study?

What percentage of participants received the allocated intervention or exposure of interest?

Is the analysis performed by intervention allocation status (intention to treat) rather than the actual intervention received? Have any attempts been made to impute missing data?

Age (median, mean and range if possible)

Sex


Race/Ethnicity

Principal health problem (incl. stage of illness)

Diagnostic criteria

Co-morbidity

Other sociodemographics (eg. Educational level, literacy level, soci-economic status, first language. Also consider possible proxies for these e.g. low baseline nutritional status )

PROGRESS categories reported at baseline (indicate letters of those reported: Place of residence, race, occupation, gender, religion, education, SES, social capital)

Subgroups Enter a description of any participant subgroups from this paper to be analysed in the review.

Intervention Group 1 (copy and paste table for each Intervention group) Group name: (State brief name for this intervention group.) Page/

Para/ Figure #

Details of intervention or control condition (Include if relevant in sufficient detail for replication)

Setting eg multicentre, university teaching hospitals, rural, metropolitan, school, workplace, community, GP clinic, etc.

Theoretical basis (include key references)

Content (list the strategies intended and delivered)

Did the intervention include strategies to address diversity/disadvantage?

Enter a description of any relevant strategies

Delivery (eg. Stages (sequential or simultaneous), timing, frequency, duration, intensity,


fidelity – process indicators)

Providers (who, number, education/training in intervention delivery, ethnicity etc. if potentially relevant to acceptance and uptake by participants

Co-interventions

Duration of intervention

Duration of follow-up

Was sustainability discussed by the authors? Was is a consideration in study development?

Economic variables ie costs of the intervention, and changes in other (eg health care)

costs as result of intervention

Yes List in Outcome section if appropriate

No Unclear

Details:

Other economic information (from a societal, non-healthcare view – e.g. lost wages, time)

Yes

No

Details:

Resource requirements to replicate intervention (e.g. staff numbers, hours of implementation, equipment?)

Subgroups Enter a description of any intervention subgroups from this report to be analysed in the review.

What are the moderators/mediators of changes stated in the study?

Do the authors describe any political or organisational context?

List relevant dot points

Were any partnerships referred to? List these as dot points

Was a process evaluation conducted?

What components were included in the process evaluation? (eg. dose, frequency, consistency, implemented as intended etc)

Control/comparison (what information is provided about what the control or comparison group received?)

Enter a description of what was provided for the control group, if applicable

Outcomes (This table is set up for 2 outcome measure to save spaces, copy and paste table as often as required)

Costs associated with the intervention can be linked with provider or participant outcomes in an

economic evaluation (depends on the type of economic evaluation)


Question Outcome 1 Page/ Para/

Figure #

Outcome 2 Page/ Para/

Figure #

Is there an analytic framework applied (e.g. logic model, conceptual framework)?

Outcome definition (with diagnostic criteria if relevant)

Type of outcome: Is this a modifiable variable (Community level, neighbourhood level, individual level) or desired health outcome

Time points measured

Time points reported

Is there adequate latency for the outcome to be observed?

Is the measure repeated on the same individuals or redrawn from the population / community for each time point?

Unit of measurement (if relevant)

For scales – upper and lower limits and indicate whether high or low score is good

How is the measure applied? Telephone survey, mail survey, in person by trained assessor, routinely collected data, other

How is the outcome reported? Self or study assessor

Is this outcome/tool validated?

…And has it been used as validated?

Is it a reliable outcome measure?

Is there adequate power for this outcome?


Were PROGRESS categories analysed by outcome? Indicate the letters of those that outcomes were analysed by (place of residence, race, occupation, gender, religion, education, SES, social capital)


Results Copy and paste the appropriate table for each outcome and subgroup at each timepoint, including baseline For RCT/CCT Dichotomous outcome page/para/fig

Comparison Outcome Subgroup Timepoint Results Intervention Comparison Events No. participants Events No. participants

No. of missing participants and reasons

Any other results reported

Reanalysis required? (specify - (e.g. correlation adjustment)

Reanalysis possible?

yes/no/unclear

Reanalysed results

For RCT/CCT Continuous outcome page/para/fig

Comparison Outcome Subgroup Timepoint Post-intervention or change from baseline?

Results Intervention Comparison Mean SD (or

other variance)

No. participants

Mean SD (or other variance)

No. participants

No. missing participants and reasons

Any other results


reported

Reanalysis required? (specify)


yes/no/unclear

Reanalysed results

For RCT/CCT Generic inverse variance method Page/para/figure

Comparison Outcome Subgroup Timepoint Results Effect estimate SE (or other variance) Intervention no. Control no.





yes/no/unclear

Reanalysed results

For CBA Page/para/fig

Comparison Assignment How were control and treatment groups selected?? Is there likely to be an

effect if these were the opposite way?

Contemporaneous data collection?

Outcome Subgroup Timepoint Post-intervention or change from baseline?

Intervention Comparison No. participants measured



Baseline result (with variance measure)

Post-intervention results (with variance measure)

Change (Post – baseline) (with variance measure)

Difference in change (intervention – control) (with variance measure)




yes/no/unclear

Reanalysed results

For ITS Generic inverse variance method Page/para/fig

Comparison Outcome Subgroup Length of timepoints measured

Snapshot or interval measured

No. participants measured


Pre-intervention Post-intervention No. of timepoints measured


Mean value (with variance measure)

Difference in means (post – pre)

Percent relative change

Result reported by authors (with variance measure)



yes/no/unclear

Individual time point results

Read from figure?

yes/no

Reanalysed results

Change in level SE Change in slope SE


Other relevant information Were outcomes relating to harms/unintended effects of the intervention described? Include any data for these in the outcomes tables above

Potential for author conflict ie. evidence that author or data collectors would benefit if results favoured the intervention under study or the control

Key conclusions of the study authors

Could the inclusion of this study potentially bias the generalisability of the review? Equity pointer: Remember to consider whether disadvantaged populations may have been excluded from the study.

Is there potential for differences in relative effects between advantaged and disadvantaged populations? (e.g. are children from lower income families less likely to wear bicycle helmets)

Are interventions likely to be aimed at the disadvantaged? (e.g. school meals aimed at poor children).

Issues affecting directness (Note any aspects of population, intervention, etc. that affect this study’s direct applicability to the review question)

References to other relevant studies

Additional notes by review authors

Correspondence required for further study information (from whom, what and when)


Risk of bias assessment

Please refer to Chapter 8 - Table 8.5.c: Criteria for judging risk of bias in the ‘Risk of bias’ assessment tool and to the Cochrane EPOC Group’s guidance for assessing Risk of bias for studies with a separate control group (RCTs, CCTs, CBAs) and Risk of bias for interrupted time series studies (Appendix 3) for additional guidance for scoring Yes/No/Unclear. Note that the table below includes items from both EPOC tools. The ITS tool has been incorporated into the bottom of the table and all items for ITS studies are denoted by ITS preceding the risk of bias question.

Domain Review authors’ judgement*

Description Page/ Para/ Figure #

Was the allocation sequence adequately generated?

Yes / No / Unclear

Describe the method used to generate the allocation sequence in sufficient detail to allow an assessment of whether it should produce comparable groups.

Was allocation adequately concealed?

Yes / No / Unclear

Describe the method used to conceal the allocation sequence in sufficient detail to determine whether intervention allocations could have been foreseen in advance of, or during, enrolment.

Were baseline outcome measurements similar?

Yes/No/Unclear

Note whether baseline outcome measurements were reported and whether there were any important differences between groups. If there were important differences between groups, note whether appropriate adjusted analysis was performed to account for this.

Were baseline characteristics similar?

Yes/No/Unclear

Note whether baseline characteristics were reported and whether there were any important differences between groups.

Were incomplete outcome data adequately addressed?

Assessments should be made for each main outcome (or class of outcomes).

Yes / No / Unclear

Describe the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. State whether attrition and exclusions were reported, the numbers in each intervention group (compared with total randomized participants), reasons for attrition/exclusions where reported, and any re-inclusions in analyses performed by the review authors.

Was knowledge of the allocated intervention adequately prevented during the study?

Separate assessments should be made for relevant groups of

Yes / No / Unclear

Describe all measures used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. Provide any information relating to whether the intended blinding was effective, or whether blinding was appropriate.

Participants – yes, no, unclear [record supporting statement from study].

Investigators – yes, no, unclear [record supporting statement from study].

Outcomes assessors – yes, no, unclear [record supporting


people involved in the study i.e participants, outcome assessors, investigators, data assessors etc

statement from study].

Data assessors – yes, no, unclear [record supporting statement from study].

Was the study adequately protected against contamination?

Yes/No/Unclear

State whether and how the possibility of contamination was minimised by the study design/implementation.

Are reports of the study free of suggestion of selective outcome reporting?


Yes / No / Unclear

State how the possibility of selective outcome reporting was examined by the review authors, and what was found.

Other sources of bias

Yes / No / Unclear

State any important concerns about bias not addressed in the other domains in the tool.

ITS: Was the intervention independent of other changes?

Yes/No/Unclear

Describe whether or not the intervention occurred independently of other changes over time and whether or not the outcomes may have been influenced by other confounding variables/historic events during the study period.


ITS: Was the shape of the intervention effect pre-specified?

Yes/No/Unclear

State whether or not the point of analysis was the point of intervention. If not, describe whether a rationale for the shape of the intervention effect was given by the study authors.

ITS: Was the intervention unlikely to affect data collection?

Yes/No/Unclear

Describe whether or not the intervention was likely to affect data collection and what the potential impact might have been.

ITS: Was knowledge of the allocated interventions adequately prevented during the study?

Separate assessments should be made for relevant groups of people involved in the study i.e participants, outcome assessors, investigators, data assessors etc

Yes/No/Unclear

Describe all measures used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. Provide any information relating to whether the intended blinding was effective, or whether blinding was appropriate.

Participants – yes, no, unclear [record supporting statement from study].

Investigators – yes, no, unclear [record supporting statement from study].

Outcomes assessors – yes, no, unclear [record supporting statement from study].

Data assessors – yes, no, unclear [record supporting statement from study].

ITS: Was incomplete outcome data adequately addressed?


Yes/No/Unclear

Describe the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. State whether attrition and exclusions were reported, the numbers in each intervention group (compared with total randomized participants), reasons for attrition/exclusions where reported, and any re-inclusions in analyses performed by the review authors.

ITS: Was the study free from selective reporting?

Yes/No/Unclear

State how the possibility of selective outcome reporting was examined by the review authors, and what was found.


ITS: Was the study free from other risks of bias?

Yes/No/Unclear

State any important concerns about bias not addressed in the other domains in the tool.

* Note: For each section above ‘Yes’ indicates a ‘low risk of bias’; ‘No’ indicates a ‘high risk of bias’; ‘Unclear’ indicates an ‘uncertain risk of bias’. When entering the data into RevMan, the options to choose from will be ‘Low’, ‘High’ and ‘Unclear’


Results Comparison: Outcome: Subcategory: Treatment group: Control group:

Observed (n) total (N) observed (n) total (N)

Treatment group: Control group:

Total randomised

excluded*

Observed

lost to follow up*

*Reasons for loss/exclusion: Subcategory: Treatment group: Control group:

Observed (n) total (N) observed (n) total (N)

Treatment group: Control group:

Total randomised

excluded*

Observed

lost to follow up*

*Reasons for loss/exclusion


Appendix 5 – Checklist for submission

This checklist should be cut and pasted, completed and sent with your protocol to the referees. Please ensure all sections have been addressed by your protocol. This will speed up the process and ensure your protocol is quickly included on the Cochrane Library. Please remove this section and use to check your protocol before submitting to the Coordinator.

Title

Is the title consistent with the one originally approved by the CPHG?

Background

Does the background support the need for a systematic review by providing sufficient information on the frequency and severity of the problem/public health issue and the uncertainties in its management?

Objective/s

Is the main objective of the review specified in terms of intervention(s), issue being addressed, population and outcomes (both beneficial and harmful)? To evaluate the benefits and harms of various initiatives that aim to ensure adequate access for all to food in communities within developed countries.

Selection criteria

Types of studies

Have you adequately identified which study designs will be included?

Is there a rationale provided for why the study designs have been included/excluded?

Types of participants

Are the characteristics of the issue being addressed and the population with whom the intervention is targeted described adequately?

Have the population groups to be excluded been specified?

Have the appropriate population groups been excluded?

Types of interventions

Have the study interventions been described?

Have the control interventions been described?

Have all relevant interventions for the issue being addressed and question asked been identified?

Have the interventions to be excluded been described?


Are the interventions to be excluded appropriate?


Are the outcome measures for benefits and harms of the intervention(s) clearly defined in nature and in timing?

Are the outcome measures used important to the population with whom the intervention is targeted?

Have all relevant outcomes (both beneficial and harmful) been included?

If specific outcomes have not been included, does this conform to the question asked?

Search methods for identification of studies

Has the search strategy been included and are the search terms appropriate?

Are the dates that each source will be searched indicated?

Does the search strategy include contacting experts in the field?

Have the appropriate subject headings, key words and text words for the clinical problem and population been used?

Has the Cochrane Collaboration search strategy to identify RCTs been used?

Has the Trials Search Coordinator been contacted?

Are studies in languages other than English to be included?

Has the team considered how duplicate publications of the same study will be identified and dealt with?

Will the following data sources be searched?

The Cochrane Public Health Group specialised register

Cochrane Central Register of Controlled Trials (CENTRAL) (most recent)

MEDLINE (from 1950 - )

EMBASE (from 1980 - )

Other databases relevant to the review topic

Reference lists of textbooks, reviews (including previous systematic reviews), and previous trials

Conference proceedings

Assessment of risk of bias

Have the criteria to be used to assess the individual studies risk of bias been reported?

Does the criteria to be used to assess study bias include:-

Sequence generation


Allocation concealment

Blinding o Participants o Investigators o Outcome assessment o Data assessors

Incomplete outcome data

Selective outcome reporting

Other potential biases

Methods of the Review

Will at least two authors of the review:-

Perform the literature search?

Determine study eligibility?

Assess study quality?

Extract data?

Enter data in RevMan?

Will authors work independently?

Will consensus and/or liaison with a third author be used to resolve disagreement between the primary authors?

Will authors of primary studies be contacted for clarification of unclear data or to obtain missing information?

Will you attempt to analyse for possible publication bias using funnel plots or other methods?

Will plausible explanations for variations in treatment effect be explored using subgroup analysis based on study quality, population and interventions?

Statistical analysis

Will the results of primary studies be reported with 95% confidence intervals using relative risk (RR) for dichotomous outcomes and mean difference (MD) for continuous outcomes?

Have the methods used to pool the results of the primary studies been reported?

Are these methods relevant?

Will RR and MD summary statistics be calculated using a random effects model? Example of text: Statistical analysis will be performed using RevMan. For dichotomous outcomes (relapse or no relapse) results will be expressed as relative risks with 95% confidence intervals. Data will be pooled using the random effects model. Where continuous scales of measurement are used to assess the effects of treatment (e.g. time to relapse), the weighted mean difference will be used, or the standardised mean difference if different scales have been used

Have you stated how you will test for heterogeneity? Example of text: Heterogeneity will be analysed using the Cochran Q test on N-1 degrees of freedom, with an α of 0.1 used for statistical significance.


Have you specified how you will determine the applicability of the results to individuals? Example of text: Calculation of absolute risk reductions with intervention in relation to different baseline risk of the event with no intervention or a different intervention.

Acknowledgements

Have you acknowledged all the relevant people and/or organizations?

Declarations of interest

Have you and your coauthors declared any potential conflicts of interest?

References

Have you checked your references?

Sources of support

Have you listed your internal sources of support (e.g. hospital, university)?

Have you listed your external sources of support (e.g. scholarship, bursaries)?

OTHER

Have you completed and are including with your protocol submission, the Health Equity Checklist?

Documents

Guide for PH Protocol_Nov 2011_final for Website