Chocrane Col

7/31/2019 Chocrane Col

1/28

Chocrane Col.

systematicreview and its

appraisal


2/28

Systematic reviews of randomised controlledtrials are considered the best level ofevidence for answering questions about theeffectiveness of healthcare interventions.

In addition to the reduction in bias, one of themany advantages of systematic reviews isthat they enable us to reduce the ever-increasing torrent of both published andunpublished research literature intomanageable portions


3/28

Advantage of

meta-analysisPooling of data from individual studies leads

to an increase in sample size, and an

increase in powerwhich is particularly important when the size

of effect is small or there is a relatively low

event rate.

The increase in sample size not only means

an increase in power, but also an increase in

the precision in the estimate of effect


4/28


5/28

1. What are the reviews objectives?

To focus on well-defined questions, stating thepopulations, intervention/control groups, and

outcomes to be included.2. How comprehensive was the search strategy?To search for all the literature relevant to the question.Published and unpublished literature should besought, any restrictions regarding language of

publication should be stated and justified, as shouldthe time period covered by the search. Ideally asystematic review needs to be up to date,incorporating all the recent literature.


6/28

Appraisal of SW

3. What were the inclusion/exclusion criteria?The criteria for selecting or rejecting studies should beclearly stated and appropriate. The process* by

which articles are assessed for relevance should alsobe recorded.

4. How was the validity of the primary studiesassessed?The process* by which validity assessment was

undertaken and the criteria used to assess the qualityof the primary studies should be clear. It should alsobe apparent how the results of the validityassessment are used within the reviews datasynthesis.


7/28

Appraisal of SW

5. How were data extracted from theprimary studies?The process* by which data was extracted

from the primary studies should betransparent.

6. Are the characteristics of the includedstudies clearly displayed?

A table illustrating the study characteristics ofeach included primary study should bepresented.


8/28

Appraisal of SW

7. Does the review examinedifferences/similarities between

the included studies and their results?

Heterogeneity between studies should be exploredand the reasons for any variations discussed.Heterogeneity can be explored statistically,graphically or through a narrative.8. Was the synthesis of the data carried out

appropriately?Was data pooled qualitatively or statistically? Ifstatistical pooling (meta-analysis) was used, was itused appropriately?

9. Were the results interpreted appropriately?

Any conclusions, implications for research or practiceshould follow on logically from the results.


9/28

How to producea Systematic

Review?


10/28

First step: to specify a tight question.

population (group to whom the intervention willapply),

intervention (the therapy, treatment or preventivepolicy to be carried out),

comparison (what will the intervention becompared against it could be a common

alternative intervention, a placebo or nointervention) and

outcomes (what do we wish to measure at theend, what is important to us and to consumers?).


11/28

A clear

Describing the background to the work,hypothesis to be tested and methodology tobe used

Allowing peer (and often consumer) review ofthe question to be asked, and methods to beused, so that these can be improved.

It also limits vague inclusion criteria that may

preferentially allow in studies withgood

results, and data dredging where lots of

analyses are tried out, but only those withsignificant results reported.


12/28

Clear

Besides the population, intervention,comparison and outcomes that should berepresented in the inclusion and exclusion

criteria, it is important to specify the type ofstudies that will offer the least biasedevidence for the review-- RCT

Ideally the process of deciding on inclusion of

studies is performed independently by atleast two people, on a form specificallydesigned for the review, sometimes blindedto authors and results.


13/28

Transparent inclusive

To include all the published and preferably also unpublisheddata that exist.

Ideally several types of searching are adopted, so that if onestrategy misses a relevant study it may be picked up through

another searching method.Search strategies generally include several of the following:structured searches of several electronic databases (includingthe Cochrane Library)

Checking through the reference lists of included studies andrelevant reviews

Letters to relevant pharmaceutical companies and experts in thefield asking about unpublished or ongoing work

Handsearching of relevant journals or conference abstracts,

Translation of foreign language articles.


14/28

Quality assessment of

the included studiesAssessment of study validity (preferably independentlyduplicated) and some statement on how those biases may affectoutcomes is essential in understanding the believability of the

results of a systematic reviewSelection bias

Attrition bias (where more participants drop out of oneexperimental arm for some reason),

Performance bias (where those receiving the intervention and/orthose caring for them are aware of the experimental allocation

and may alter concurrent treatments accordingly)Detection bias (where those assessing outcomes are aware ofthe experimental allocation and may be open to biased outcomemeasurement).8


15/28

Ideally the process is independently

duplicated, based on prior decisions (in

the protocol), comprehensive (on a formdesigned for the review, and may

involve contacting authors to fill in any

gaps in published reports)Clearly tabulated to allow transparency

and possibly corrections at a later date.


16/28

of data

Narrative or statistical pooling or meta-

analysis?

Narrative or meta-analytic comparisonsand sub-groupings should be pre-

specified in the protocol (to avoid

multiple analyses being carried out withonly the statistically significant ones

being published).


17/28

To pool extracted numerical data, weighted

so that larger studies, or those with less

variability, contribute more to the outcome.This pooling provides an answer with greater

precision that each included study on its own

The pictorial representation of a meta-

analysis is called a forest plot (see example).


18/28

Forest plot of continuous data


19/28

Statistical analysis

Fixed effects (where it is assumed that the trueoutcomes of the various studies are the same)

Random effects methodologies (where the true

outcomes are assumed to vary a little with differingstudy inclusion, dose, duration etc).

Where fixed effects meta-analysis produces a resultthat is statistically heterogeneous it is usual to switchto random effects meta-analysis.

Statistical heterogeneity of studies (large differencesin their results, suggesting differing true outcomes) isideally explored through subgrouping or meta-regression.


20/28

A meta-analysis is a very good way of summarisingdata from a group of studies. However, this is onlyuseful where the set of studies is representative ofthe whole body of literature, so should generally berestricted to use within systematic reviews.Meta-analyses can be performed by hand or with acalculator, but are usually completed with the help ofspecialised computer software (that may also createa forest plot).

There are many very good types of softwareavailable, but for those embarking on a Cochranereview the free Review Manager software(downloadable from the main Cochrane website) isexcellent, creating forest plots


21/28

Cochrane Collaboration

Cochrane Library andOral Health Group


22/28

In 1972, the British epidemiologist ArchieCochrane published an influential bookEffectiveness and Efficiency. RandomReflections on Health Services.

It is surely a great criticism of our professionthat we have not organised a criticalsummary, by speciality or subspecialty,updated periodically, of all relevantrandomised controlled trials (ArchieCochrane)


23/28

Cochrane Collaboration was formed in October 1993.

The Cochrane Collaboration aims to help peoplemake well-informed decisions about health care by

preparing, maintaining, and promoting theaccessibility of systematic reviews of the effects ofhealth care interventions.

Over the last ten years it has grown into aninternational organisation, currently over 6,000people contributing from over 60 countries.

What is remarkable about the CochraneCollaboration is that the majority of these contributorsundertake their Cochrane work in their own time.


24/28

The main product of the CochraneCollaboration is The Cochrane Database of

Systematic Reviewsthat forms part ofTheCochrane Library, a quarterly electronicpublication.

It contains the full text of more than 1350

regularly updated systematic reviews andmore than 1,000 protocols for reviews inprogress. Several hundred reviews andprotocols are added annually.


25/28

CL policy

It is Cochrane policy that reviewers revisit theirreview and update it within two years of it beingpublished on The Cochrane Library.

There are several stages to the Cochrane peerreview process, including the assessment ofprotocols, evaluation of the reviews methodologyand content by editors, peer reviewers and potentialend users / consumers.

It has been suggested that the existence of such athorough refereeing process ultimately leads toCochrane reviews being less prone to bias thansystematic reviews and meta-analysis published inpaper-based journals.


26/28

Responsible for preparing and maintaining systematic reviewswithin the scope of oral health.

Oral health is broadly conceived to include the prevention,

treatment and rehabilitation or oral, dental and craniofacialdiseases and disorders.

Alexia Antczak-Bouckoms initially set up the OHG in NewEngland (USA) in 1994. The group moved to Manchester (UK)in 1996 and secured National Health Service (NHS) funding forthe editorial base in 1997.

The editorial base is situated in the Manchester DentalEducation Centre, University Dental Hospital of Manchesterunder the Co-ordinating Editorship of Professor William Shawand Dr Helen Worthington.


27/28

OHGs Specialised

Register of TrialsIt currently holds over 13,400 reports of oral health related trials(RCTs, CCTs) and related references from a wide range ofbibliographical sources including MEDLINE, EMBASE, CINAHL,CANCERLIT, PSYCLIT, and the Cochrane Controlled TrialsRegister in addition to conference proceedings.

The register is continually growing as a result of on-goingelectronic searching and the OHGs organised programme ofhandsearching the oral health literature.

This handsearching programme also contributes to theCochrane Collaborations worldwide handsearching programmeco-ordinated by the New England Cochane Centre, USA.

This collection of references from various sources makes theSpecialised Register a unique and valuable resource and thebest starting point for anyone considering a systematic reviewwith the oral health field.


28/28

The Oral Health Group welcomes all those interested incontributing to the work of the group.

There are several options for participation, either as a lead

reviewer, assisting as a co-reviewer, handsearching a journal toidentify RCTs, or by becoming a member of the panel of peerreviewers or consumers.

For further details or an information pack please refer to thegroups website: www.cochrane-oral.man.ac.uk

Contact: Emma Tavender, Co-ordinator, Cochrane Oral Health

Group, MANDEC, University Dental Hospital of Manchester,Higher Cambridge Street, Manchester M15 6FH. Tel: +44 161275 7818, Fax: +44 161 275 7815,

Email: [email protected]
http://www.cochrane-oral.man.ac.uk/http://www.cochrane-oral.man.ac.uk/http://www.cochrane-oral.man.ac.uk/http://www.cochrane-oral.man.ac.uk/

Documents

Chocrane Col