Literature searching &critical appraisal

Chihaya Koriyama

August 15, 2011 (Lecture 2)

Why is literature search important?Literature search helps you to:

Know what has been already done Get up-to-date information Develop your research hypothesis Design your study

Study designHow to collect the informationData analysis

Previous studies are your best teachers

Spend ENOUGH TIME

on literature search

BEFORE you start

designing your study

On-line information sources

• PubMed

http://www.ncbi.nlm.nih.gov/sites/entrez• embase

http://www.embase.com/• Free Medical Journals

http://www.freemedicaljournals.com/• Popline

http://www.popline.org/• HINARI

http://www.who.int/hinari/en/

How to do a literature search

• Web site of Robert Gordon University, UKhttp://www4.rgu.ac.uk/library/howto/page.cfm?pge=25989

• Stages of the search• Record keeping and referencing• Sources of information• Search techniques• Evaluating the results of your search• Useful online tutorials and web pages

6

Key words

Number of hits

See all related articles

Start with Review

When not sure of the keyword

Critical appraisal of scientific articles

What is critical appraisal?

Critical appraisal is a systematic approach to:

- reading,

- understanding,

- interpreting,

- identifying the limitations of and

- deciding upon the usefulness of results of

scientific papers.

What is Journal Club?Presentation of an article

related to the presenter's research,

followed by discussion with other members.

* It can be weekly, biweekly, monthly, etc.

Discussion should focus on what can be learned from the article,

not on the criticism of the article.

Can you find this information in the paper?

Check the publication year, Journal name, and author(s)

1. What is the research question?

2. What is the study type (design)?

3. Selection issues

4. What are the outcome factors and how are they measured?

5. What are the study factors and how are they measured?

Can you find this information (continued)?

6. What important potential confounders are considered?

7. What is the statistical method in the study?

8. Statistical results

9. What conclusions did the authors reach about the research question?

10.Study limitations?

11.How do you apply the findings into your daily clinical practice or research?

An ideal abstract contains the underlined information at least.

3. Selection issues (1)

• Are there selection biases?• In a case-control study, suppose you

want to compare exercise habits.

cases controlsAre they reasonablycomparable?

3. Selection issues (2)

• In an intervention study, how were subjects recruited and assigned to groups?

• In a cohort study, how many reached final follow-up?

Randomly?

4. What are the outcome factors and how are they measured?

Are all relevant outcomes assessed?

Is there measurement error?

5. What are the study factors andHow are they measured?

Is there measurement error?

Consider the sensitivity & specificity of each detection method.

6. Confounder(s) Suppose, you want to compare the

proportion of cigarette smokers between stomach cases and controls.

You found more smokers in the GC cases.

I found a significant association between smoking and gastric cancer risk!

6. Confounder(s) However, you found that there were more

males in GC cases than controls. Since a proportion of smokers in males

usually higher than that in females, your finding might be distorted because of the different sex distribution.

After adjusting the effect of sex distribution, there was no difference in the association between smoking and GC risk.

7. What is the statistical method in the study?

• Are the results dealt with adequately?

– The effects of confounder(s) should be controlled.

8. Statistical results

Please check Is a P value given?

Are confidence intervals (CI) given?

If results negative and CI not given, is the power of the test given?

Interpretation of P value

• When P value is less than significance level, which is usually 0.05, we often “reject the null hypothesis”

Statistically significant!

Conversely, when P value is greater than 0.05, we conclude that the result is not statistically significant.

What does “no significance” mean?

No sufficient evidence to doubt the validity of the null hypothesis

We cannot reject the null hypothesis (we concede that there is no difference)

– The statistical test does not prove the null hypothesis.

Interpretation of confidence intervals All estimates have their corresponding

confidence intervals. Suppose, you want to know the mean

cholesterol level in a population but you

cannot examine all subjects. You randomly select a part of

the subjects and calculate the

mean value. You can also calculate its confidence

interval.

Set of 95% confidence intervals of 100 experiments

True mean value

These five out of 100 results do not include the true value.

Your result may be one of these results but nobody can tell you “which one”.

Confidence intervals of odds ratio

• Odds ratio : a measure of the effect, the strength of the association between a study factor and outcome.

Odds in the exposed group

=

Odds in the un-exposed group

Disease risk

Odds ratio =1 : there is no difference in the disease risk between the exposed and un-exposed groups.

Odds ratio > 1: disease risk in the exposed group is higher than that in the un-exposed group.

Odds ratio < 1: disease risk in the exposed group is lower than that in the un-exposed group.

Campos et al. (J Cancer Res Clin Oncol 2008)

Since “odds ratio =1” means that there is no difference in the risk between this group (Q4) and referent group (Q1)

If 95%CI contains “1”: Not statistically differentIf 95%CI does NOT contain “1”: Statistically different

Crude OR=(38x72)/(57x71) = 0.68When it is adjusted byconfounding factors ・・・・