Slide set for editors training day edited for blog

COHG Editors’ & Clinical Advisors’

Meeting

23 October 2013

The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.

Agenda

COHG Editors & Clinical Advisors Meeting 23 October 2013

1. Welcome2. Role of Editor 3. Prioritisation of topics 4. Title registration – discussion of proposed titles 5. Introduction to MECIR 6. Screening of reviews/ Tips for good practice 7. Worked example – flossing review:

a. Abstract b. Risk of Biasc. Outcomes d. Summary of findings e. Consistency across review

The role of a Cochrane OHG

Editor


Co-ordinating Editors

Managing Editor/Deputy Managing Editor

Trials Search Co-ordinator (Feedback Editor)

Consumer Co-ordinator

Editors

Clinical Advisors

Referees

Research/Editorial Support Co-ordinator

Administrative Assistant

COHG’s Editorial team

To support review teams:

805 registered-COHG authors

COHG’s membership

To explore the role of Editors

Minimum requirements

Input into editorial process

Role of Contact Editor

Open discussion

Aim of session

Minimum requirements:

Completed a review prior becoming an editor

Attend at least two Cochrane/COHG events a year (e.g. editorial meetings, review proposal meetings, regional meetings, colloquia, training days…)

Input into editorial process

Act as Contact Editor

Editors

Selecting and shaping the Cochrane Reviews prepared by COHG

new review proposals

priorities / emerging areas of research

Providing comments on review drafts

Editorial process input

Allocated reviews within specialty;

To provide ad-hoc methodological and/or content advice to review teams throughout entire editorial process;

Judge whether the review is ready for peer review and help identify appropriate referees;

Look over submitted comments; outline expected revisions; advise authors on response to referee comments;

Examine revised reviews and confirm whether review should go forward for signing off/publication.

Act as Contact Editor

What aspects of the Editor’s role could be made easier?

List of assigned reviews?

Improved checklist form?

More/less time for comments?

Sufficient support from Editorial base?

Unmet training needs?

Ways to keep up-to-date with developments in Cochrane or COHG?

Discussion

Prioritisation of Cochrane OHG Reviews


Base-line salaries (NIHR)

Global Alliance (donations from dental bodies) British Orthodontic Society, UK Royal College of Surgeons of Edinburgh, UK New York University College of Dentistry, USA British Society of Paediatric Dentistry, UK National Center for Dental Hygiene Research &

Practice, USA Canadian Dental Hygienists Association, Canada British Society of Periodontology, UK British Society of Oral Surgeons, UK Mayo Clinic, USA

Funding providers

Importance of our reviews asking relevant questions

Important reviews are kept up-to-date

Two international prioritisation exercises so far:

Orthodontics and paediatric dentistry

Prioritisation of reviews

1. Invitation to all COHG paediatric authors;

2. Invitation to international expert panel;

3. Two teleconferences;

4. Final list of reviews

Paediatric prioritisation exercise process

Paul Ashley, UK

Joel Berg, USA

Michael Casas, Canada

Urshla Chaudhry (BSPD), UK

Ben Cole, UK

Michael Cranfield, UK

Ivana Čukovid-Bagid, Croatia

Peter Day, UK

Chris Deery, UK

Monty Duggal, UK

Mike Harrison, UK

Marie-Therese Hosey, UK

Milton Houpt, USA

Norbert Krämer, Germany

Jessica Lee, USA

Nick Lygidakis, Greece

David Manton, Australia

Jeanette Mooney, UK

Linda Rosenberg, USA

Svante Twetman, Denmark

Jaap Veerkamp, Netherlands

Richard Welbury, Scotland

Expert panel members: paediatric dentistry

Prevention

Fluoride varnishes, gels, mouthrinses, toothpastes (8)

Sealants (2)

Results of paediatric prioritisation exercise

Treatment

Different methods for treating and managing decay in primary and permanent dentition (6)

Pulp treatment (1)

Preoperative analgesics (1)


Additional topics (with reviews)

Primary school-based behavioural interventions for preventing caries

Hypnosis for children undergoing dental treatment

Sedation of children undergoing dental treatment


Additional topics (no reviews)

Complex interventions for preventing caries

Diagnosis of caries

Management of early caries lesions in children


Paediatric experts wanted to collaborate over guidelines

Not what we are here for, but we thought of an idea to help:

Guideline repository

Expert panel discussion: paediatric guidelines

The purpose of the repository is to:

help identify priority review topics that could inform guideline development;

identify areas of duplication/overlap, where evidence tables could be shared between Guideline Development Groups (GDGs);

increase stakeholder involvement in guideline development by widening dissemination.

Guideline repository

We already liaise with NICE and SIGN to ensure we are prioritising the reviews they need

Fluoride varnish review

Initial review published in 2002 (9 trials)

Update published July 2013 (22 trials)

Guideline implementation

Prioritisation exercises need to:

include all stakeholders

cover whole of oral health

Prioritisation of reviews

Introduction to MECIR


Methodological Expectations of CochraneIntervention Reviews

Methodological standards for the conduct of new Cochrane Intervention Reviews December 2012

Standards for the reporting of new Cochrane Intervention Reviews December 2012

Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS) February 2013

MECIR

Standards for conduct

Standards for conduct

Standards for reporting

Standards for reporting

Reporting of PLS (PLEACS)

Reporting of PLS (PLEACS)

Aim

To highlight some of the key features of a Cochrane Review and apply MECIR standards where appropriate:

Abstract Risk of Bias Outcomes Summary of Findings Consistency across the review

Sambunjak D, Nickerson JW, Poklepović T, Johnson TM, Imai P, Tugwell P, Worthington HV. Flossing for the management of periodontal diseases and dental caries in adults. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD008829. DOI: 10.1002/14651858.CD008829.pub2.

Worksheet

You’ve been provided with: A set of MECIR standards for the reporting of

abstracts

An abstract for the flossing review

Read through the abstract and considerwhether the reporting of the results and conclusion in the abstract complies with the MECIR standards.

Abstract

Do the conclusions follow on from the results?

What changes, if any, would you make?

Abstract

Risk of Bias

Chapter 8 of the Cochrane Handbook

It’s about validity of findings

It’s not about ‘quality’

A main component of the Summary of Findings

Bias Authors’

judgement

Support for judgement

Random sequence generation

(selection bias)

Allocation concealment

(selection bias)

Blinding of participants and

personnel (performance bias)

Blinding of outcome assessment

(detection bias)

Incomplete outcome data

(attrition bias)

Selective reporting

(reporting bias)

Other bias

Example RoB table

Children were sequentially allocated. The first child who came for examination to the first group, second child to the second group

Random sequence generation:HIGH - systematic method, not true randomisation

Further examples

The study was conducted in a single blind fashion

One of the blinding domains, but we don’t know which one: UNCLEAR – who is blinded?

Further examples

An envelope was opened, in which it indicated whether the patient should receive the bio-adhesive gel or not

Allocation concealment:UNCLEAR – is it opaque, sealed and sequentially numbered, and were they centrally/remotely assigned???

Further examples

22 participants had amalgam restorations whilst 23 had composite restorations. 8/47 teeth had restoration failures in the amalgam group compared to 6/52 in the composite group.

Selective reporting:HIGH - reported at tooth level rather than at patient level, ignoring the paired nature of the data and clustering of teeth in the mouth

Further examples

In a trial of drug X versus placebo to treat oral mucositis, 13/35 were withdrawn from the drug X group, 7 of these due to lack of efficacy. 25/34 were withdrawn from the placebo group, 17 due to lack of efficacy. An intention-to-treat (ITT) analysis was carried out including those remaining.

Incomplete outcome data:HIGH – only 9 of the 34 in placebo group analysed

Further examples

Summarising RoB

Include ‘all outcomes that are likely to be meaningful to clinicians, patients (consumers), the general public, administrators and policy makers….’

may not have considered/evaluated all in trial

unlikely to be reported in same publication

useful to acknowledge even if not recorded/ reported

Which outcomes?

What are the main outcomes of interest for decision making?

Importance of outcomes from clinicians, consumer advisory groups, guidelines, scoping search COMET initiative

≤3 primary outcomes (beneficial and adverse)

Info for SoF table – key information ‘There should be no more than seven main outcomes, which

should generally not include surrogate or interim outcomes.’

Directly relevant to review question, not ‘what the review found’

Prioritisation

be well-defined measuring a single concept or where an outcome is complex, this should be clearly specified

specify the timing of measurement clinically important period of measurement multiple observations introduce unit of analysis issues

specify the nature of measurement subjective or objective outcomes, validated scale of

measurement, consistent with other reviews in same clinical area?

be directly measured wherever possible, avoiding surrogate outcome measures

have relevance for future updates of the review

The outcomes of a review should evaluate both the beneficial andnegative effects of the interventions.

Outcomes should…

"Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“

The main outcome measures for children and carers were long-term freedom from the main symptom of dental decay - pain.

Primary outcome measures

freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling

time until filling or crown needs to be replaced or requires further intervention

proportion of filled or crowned teeth retained until appropriate age of shedding

Problem illustration

Single or multiple outcomes

"Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“

Other measures of success recorded and analysed

absence of clinical or radiographic evidence of secondary caries

other clinical signs of pathology (fracture of tooth or filling, wear of crown, inflammation of gingival (gum) tissue)

patient satisfaction

costs to patient and provider

adverse events

Problem illustration

Different outcomes (surrogate?)

Single or multiple outcomes

Any adverse events or those likely to be related to the intervention?

Primary outcome measures :

Pain from dental decay (after the procedure) at time points x, y and z

Treatment success at x year (defined as freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling)

Time until filling or crown needs to be replaced or requires further intervention

Proportion of filled or crowned teeth retained until appropriate age of shedding

Other measures of success recorded and analysed:

Absence of clinical evidence of secondary caries (add definition how clinical evidence of secondary caries are to be measured). Where studies do not report clinical outcomes for secondary caries, radiographic evidence will be reported if available.*

Inflammation of gingival (gum) tissue

Peri-operative pain - Pain (as rated by children) during the procedure (adverse event)

Bleeding ( adverse events)

Patient satisfaction

Costs to patient and provider*This means that only one of these will be reported if both have data are available in the studies. If only radiographic is available, then this will be included as a surrogate for the real outcome of interest.

Possible solutions

Primary outcomesThe main outcome of interest was success or failure of the replacement or repair restoration and associated tooth as assessed by clinical examination.

The primary outcome measures were therefore the clinical acceptability or unacceptability of each restoration, defined by the USPHS criteria, Ryge criteria or modifications of these scales, and assessed by clinical examination.

It was anticipated that this would be recorded as success or failure of the restoration, and/or that further repair or replacement of the restoration was necessary.

Other outcome measures indicating the failure of a replacement or repaired amalgam restoration included the following (occurring in relation to the repaired/replaced amalgam): placement of an additional restoration e.g. crown or inlay; root filling; clinical symptoms e.g. pain, swelling, diagnosis of pulpitis, abscess

formation, and extraction of the tooth.

Studies should have determined success or failure according to the same criteria used in the decision to replace or repair the restoration.

Comparable review

Secondary outcomesIn addition to the main outcome measure, any outcomes reported perioperatively (e.g. pain/discomfort) and postoperatively (i.e. within 48 hours; e.g. pain/discomfort) were to be recorded. Where any other outcomes were presented e.g. related to patient experience or aesthetics these were also to be recorded.

Timing of outcome assessmentThe decision on which outcome period to use for the review was to be based on the most commonly reported period(s) of assessment amongst studies meeting the inclusion criteria.

Outcome data from all periods of follow-up were to be included, but where the period of follow-up differed between studies, this was to be categorised as medium-term (less than 5 years) or long-term (5 years and above).

Time-to-event (survival data) was to be collected and analysed where available.

Comparable review

Remember!

Reporting of outcomes should rarely determine the eligibility of studies for a review.

Studies shouldn’t be excluded because they provide no 'usable' data. Even in absence of any information for a review outcome there will still be an entry into the SoFtable.

Look for ‘expected’ key outcomes in the clinical area are routinely measured

Study eligibility

Summary of Findings

Based on work presented by Nancy Santesso, Quebec 2013


Developed PICO

Located studies

Abstracted data from studies

Assessed the risk of bias in the studies

Synthesised the data for each outcome

Assessed heterogeneity and publication bias

Assess the evidence and draw conclusions

Summary of Findings

How do we:

interpret results and draw conclusions?

GRADE

present results to reader/users?

Summary of Findings tables

Summary of Findings

Two main concepts when interpreting results:

magnitude of effect

confidence in that effect

Summary of Findings

Summary of Findings

Summary of Findings

Summary of Findings

Determinants of quality of evidence/confidence in effect

Consider 5 factors to evaluate the quality of the evidence:

Risk of bias

Inconsistency (or heterogeneity)

Indirectness (PICO and applicability)

Imprecision(number of events and confidence intervals)

Publication biasSee also Cochrane Handbook, Chapter 12

Quality of Evidence varies from:

HIGH

MODERATE

LOW

VERY LOW

Confidence

Pulling it together

Quality Interpretation

HighWe are very confident that the true effect lies close to that of the

estimate of the effect.

ModerateWe are moderately confident in the effect estimate:

The true effect is likely to be close to the estimate of the effect,

but there is a possibility that it is substantially different.

LowOur confidence in the effect estimate is limited:

The true effect may be substantially different from the estimate

of the effect.

Very lowWe have very little confidence in the effect estimate:

The true effect is likely to be substantially different from the

estimate of effect.

Pulling it together

Systematic review and clinical question information: Participants, interventions and comparisons

Outcomes

Present important outcomes:

Select from primary outcomes identified a priori in protocol or those important to making recommendations

Choose patient-important outcomes

Include outcomes, whether you have results or not

Describe the outcomes (e.g. Provide scale range and indicate a high score)

Outcomes

What happens to people with, and without, intervention?

Results: Absolute effects

Results: Relative effects

Only some studies contributed information about an outcome

Results: Number of participants/studies

Results: Quality of the evidence

Results: Comments

Clarification

Judgements

Transparency

Results: Footnotes

Considerations when making conclusions Do I believe the results from these studies?

Risk of bias

Are the results consistent across studies? Inconsistency

Are these all of the studies? Publication bias

Is this effect size precise? Imprecision

How do these results apply? Applicability, directness

Conclusions

You are provided with a SoF table for the flossing review.

Two forest plots showing the results for gingivitis and plaque at 1 month.

Examine the SoF and the Forest Plots for discussion

Flossing review: SoF

You are provided with:

MECIR standards that relate to the conclusions from the main text of the review

the conclusions from the flossing review

First consider the information presented inthe conclusion alone, and then compare it to that in the abstract and summary of findings

Internal consistency

• What changes, if any, do you think are necessary in the conclusion?

• How does the information in the conclusion compare to that in the abstract and summary of findings?

• Are there any additional changes you would make and why?

Internal consistency

What aspects of the Editor’s role could be made easier?

List of assigned reviews?

Improved checklist form?

More/less time for comments?

Sufficient support from Editorial base?

Unmet training needs?

Ways to keep up-to-date with developments in Cochrane or COHG?

Discussion

Health & Medicine

Slide set for editors training day edited for blog