Statistical Analysis

MBS-01

Nominal Data

• Dichotomous data• Categorizes variables• Assigns names, letters or descriptors• Gender, race, yes or no• No rank or mathematical

relationship to each other• Have equivalent weight

or value

Ordinal Data

• Reflects an order to variables or data

• Does not imply magnitude

• Zero point is arbitrary

• Pain Scale 0-10

Continuous Data

• Interval or Ratio Data

• Can compare absolute magnitude

• Implies a numeric relationship

• Can undergo arithmetic operations

• Data can be averaged and manipulated

Measures of Disease Frequency

Incidence Rate

• Proportion of group initially healthy that will develop a disease within a specified period of time

• Expressed in person-days, person-months, etc.

• Measures only new cases

Prevalence• Proportion of people in

the population with disease at a given time

• Measures all existing cases

• Underestimates acute or rapidly occurring illnesses

Relative Risk and Odds Ratio

• Measures of association• Measures of disease frequency• Expressed as a single value• Describes the strength of association

between the exposure and outcome• Does not imply any extent of variation• Often accompanied by a confidence

interval

Relative Risk

• How many times more likely an outcome is for one group compared with another

• Ranges from 0 to infinity – RR of 0 is no association– RR of 1 is risk of acquiring disease same for subjects

with and without risk factor– Association is stronger as RR increases (>10 felt to be strong association)– RR = 0.5 then initial risk is cut in half– RR = 2 then initial risk is doubled

• Used in cohort (follow-up) design

Odds Ratio

• Estimator of relative risk• Compares the prevalence of a disease

when a specific factor is present or absent• Assumes cases & control gp representative

of general population with respect to occurrence of risk factors

• Assumes the frequency of disease in exposed or unexposed is small

Odds Ratio

• Cross-sectional & Case-control design• Uses single value to describe strength

of the association between exposure and outcome

• OR <1 then risk as decreased• OR = 1 no association between risk

factor and disease• OR >1 then risk has increased

Number Needed to Treat(NNT)

• How many patients must be treated to get one good event (or prevent one bad event)

• Applicable to groups of patients with similar underlying risk

• Calculated from follow-up and experimental design studies

Disease

Present Absent

Fac

tor

Exposed A B

Not Exposed

C D

Relative Risk = A /(A+B)

C/(C+D)

Odds Ratio = A * D

B * CNumber Needed to Treat_________1___________[A/(A+B) ] - [C/(C+D)]

Research vs Null Hypothesis

Research Hypothesis:• The hypothesis tested by the study• Can be one tailed:a difference in only 1 direction• Can be two tailed:a difference in two directions

Null Hypothesis:• Opposite of the research hypothesis• Hypothesis of no difference• Statistics are applied to the null hypothesis

Interval and Ratio DataNormal Distribution

Nominal and Ordinal DataNonnormal Distribution

Independent Measurements

• Seen in parallel design trials

• Data do not depend on each other

• Data do not reflect serial measurements

Sample

Treatment

Treatment

MeasureOutcomeVariables

MeasureOutcomeVariables

Population

X

Dependent Measurements

• Seen in cross-over design studies

• Seen in studies using matched groups

• Data depend on or reflect each other

Sample X TreatmentMeasureOutcome

MeasureOutcome

MeasureOutcome

Statistical Analysis

Data Type

Samples are:

Measure of Correlation

2 Indep 2 Related 3 or > Indep 3 or > Related

Nominal Chi-square McNemar Chi square Cochran Q Contingency coefficient

Ordinal Mann-Whitney U or

Wilcoxon Rank Sum

Sign test or

Wilcoxon Signed Rank Test

Kruskal-Wallis one-way analysis of variance (ANOVA)

Friedman 2 way analysis of variance (ANOVA)

Spearman rank correl coefficient

Kendel rank correlation coefficient

Interval Student’s t test

Mann-Whitney U test

Paired t test One-way analysis of variance (ANOVA)

2 way (repeated measures) analysis of variance (ANOVA)

Pearson Correlation Coefficient

Errors Related to Hypothesis

Research

Conclusion

True Situation: Null Hypothesis is:

True False

Do Not

Reject H0

(No Difference)

Correct Decision

Confidence Level

Probability =1-

Error: Type II

Probability=

Reject H0

(Difference)

Error: Type I

Significance Level

Probability =

Correct Decision

Power of test

Probability =1-

Alpha

• The probability of making a Type I error

• Predetermined by the investigator

• Usual values 0.05 or 0.1 (1 in 10 or 1 in 20 chance of Type I error)

• P value: the numeric representation of

Beta and Power

• Power is the probability of avoiding a Type II error.

• Or the chance of finding a difference if it truly exists

• Power is 1-• Increase power by increasing n,

increasing , or increasing the size of difference accepted

Summary

• Statistics tell us about the role that sampling variability plays in results

• Statistics make no claim about the validity of a study

• Consider the impact of Type I and II Errors • Results May be Statistically Significant but

Clinically irrelevant.

Error, Validity and Precision

Random Error

• Not constant error

• Due to chance

• Unknown sources of variation equally likely to affect findings in either direction

• Seen as inconsistency in repeated or equivalent measurements when made on the same object or person

• Increase sample size to reduce random error

Systematic Error

• Constant error• Due to bias• Sources of variation that affect findings in

one direction• Improve study design to reduce• Investigator should include explanation of

systematic error in publication• Change of sample size will not affect

systematic error

Reliability vs Validity of Data

• Reliability: reproducibility of measurement

• Validity:extent to which differences in scores reflect the true differences among individuals on the characteristic we are seeking to measure

Validity

Study Results

Truth in Study Results

Truth in the Universe

Internal Validity

External Validity

Errors of chance and bias

Threats to Internal Validity

• History: naturally occurring event external to study but occurring simultaneously

• Maturation: change in the study subject occurring as a function of the passage of time

• Instrumentation: changes or errors in the measuring instrument or observer

Threats to Internal Validity

• Selection: the way in which the subjects were selected and assigned to treatment groups

• Experimental Mortality: dropout, nonresponse, or death

• Main testing effect: being tested may bring about a change in behavior on a second observation

• Statistical regression: tendency of extremes to move toward the mean during an experiment

Threats to Internal Validity

Hx Mat Ins Sel Mor

One-Shot Case Study - - - - -

One-Gp Pre & Post Tx - - - +/- +

Static Group Comp + - + - -

Pre & Post Tx Control Gp + + + + +

Post Tx only Control Group Design

+ + + + -

Nonequivalent Pre-post Tx Control Group Design

+ + + +/- +

Threats to External Validity

• Interaction of Subject Variables and Tx: tx has varied effects on subgroups

• Interactive Effect of Testing:pretesting may sensitize subjects to variable

• Reactive Effects of Experimental Arrangements: study setting may be atypical

• Multiple Treatment Interferences:same subject given several treatments; effects of earlier treatments not completely erasable

• Hawthorne Effect:volunteers try to give “right” answer

Measures of Central Tendency

• Mean: average• Median: midpoint where ½ of observations

fall above and ½ fall below the value• Mode: most frequently encountered number• In a normal, or Gaussian, distribution the

mean, median, and mode are identical

Precision

• How closely the estimates will tend to cluster about the true value

• Larger the standard deviation or standard error of the mean the less precise the data

Normality

0

1

2

3

4

5

6

0 1 2 3 4 5 6 7 8 9 10

Pain Score

Pat

ien

ts

Mean, median, & mode

Skewed Distribution

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10

Pain Score

Pat

ien

ts

ModeMedian

Mean

Measures of Variability

Standard Deviation• Measures how close

the values cluster to the sample mean

• Interval Data• Square root of

variance• Reported as +/-• If normal distribution1 S.D. = 68% data2 S.D. = 95% data3 S.D. = 99% data

Standard Error of the Mean

• Estimates mean of population from sample meanˉх

• Equals S.D./ square root of n

• Smaller number than SD therefore often reported as measure of dispersion

Confidence Intervals

• Further defines the p value by giving a range of values to describe the data

• An interval that will, with the probability of a confidence level, contain the true difference being investigated

• A confidence interval which includes “0” does not permit rejection of the null

Sample Size, Meta-Analysis & Evidence

Based Medicine

MBS-01

Sample Size

• Determined before initiation of study• Re-evaluated at conclusion of study due tp

dropouts or deaths• Often not included in publication• Nomograms, formulas, tables are available

to assist reader with sample size calculation• Studies of inadequate sample size: pilot

studies

Sample Size Determinants

• The outcome being evaluated

• Tolerable risk of error (α and β)

• Clinically important difference ()

• Variability of measurement (s and s2)

• Ratio of experimental to control subjects

Sample Size Determinants

• The outcome being evaluated– Dichotomous outcome– Continuous outcome

• Tolerable risk of error (α and β)• Clinically important difference ()• Variability of measurement (s and s2)• Ratio of experimental to control

subjects

Sample Size Determinants

• The outcome being evaluated• Tolerable risk of error (α and β)

– Type I (α)• concludes there is a difference when in fact there is not • convention sets risk at 1-in-20 chance or an α of 0.05

– Type II (β)• concludes there is no difference when in fact one exists• Convention sets risk at 2-in-10 chance or a β of 0.2

• Clinically important difference ()• Variability of measurement (s and s2)• Ratio of experimental to control subjects

Sample Size Determinants

• The outcome being evaluated• Tolerable risk of error (α and β)• Difference between experimental and

control group ()– What is clinically important– Detecting a small difference between groups requires larger sample size

• Variability of measurement (s and s2)• Ratio of experimental to control subjects

Sample Size Determinants

• The outcome being evaluated• Tolerable risk of error (α and β)• Clinically important difference ()• Variability of measurement (s and s2)

– Expressed as s for continuous variables– Not specified for dichotomous variables– Smaller spread around mean requires smaller sample size

• Ratio of experimental to control subjects

Sample Size Determinants

• The outcome being evaluated• Tolerable risk of error (α and β)• Clinically important difference ()• Variability of measurement (s and s2)• Ratio of experimental to control subjects

– One-to-one ratio minimizes sample size– Assumed to be one-to-one in Young Nomogram

Sample Size

• Equations & nomograms differ for type of outcome measured; nominal vs continuous

• Z (Standard Normal) distribution is used for alpha and beta – Mean of 0– S.D. of 1

• Can solve for any portion of equations if other factors are known

• Plan for drop-out

Sample Size Equation for Dichotomous Variables

P: proportion of responders in both groups

P1: proportion of responders in experimental group

P0: proportion of responders in control groups

Use 2 sided alpha of 0.05 Zα =1.96

Use one-sided beta of 0.20 Zβ = 0.84

2

βαn

01

0011/2

PP

)P(1P)P(1PzP)2P(1z

Sample Size Equation for Continuous Variables

2

2/2 )z(z2 2

δn

βασ

Use 2 sided alpha of 0.05 Zα =1.96

Use one-sided beta of 0.20 Zβ = 0.84

: difference in means for control & experimental group

σ: weighted average standard deviation in the control group

Nomograms for Calculation of Sample Size

• Assumes parallel study (no crossover) with two groups

• Assumes 2 tailed alpha; if not, overestimates needed sample size

• Assumes alpha 0.05 and beta 0.2• Approximation used retrospectively to

critique another author’s conclusions

Number of Patients Nomogram

MBS-01

Sample Size and Study Outcome

• Difference Observed– n is large enough to identify difference– Difference may be overestimated if n is smaller

than would have been determined using sample size calculations

– Extremely large n may find difference but clinical significance of the difference lacking

• No difference Observed– n may be too small– Truly may be no difference

Meta-Analysis

A statistical analysis which combines or integrates the results of several independent clinical trials considered by the analyst to be combinable.

Goal is to discern a more objective and generalized answer to a particular question or clinical dilemma by combining results from different studies that have similar research hypotheses.

The Meta-Analysis

• Tertiary literature• Applied to experimental and observational

designs• Useful when previous studies inconclusive,

contradicting, or sample size is too small• Can be used to increase power of a study• Exercise caution and skepticism in evaluating the conclusion!

MBS-01

Quality Areas of Meta-Analysis

• Study design

• Ability of data to be combined

• Control of bias

• Statistical analysis

• Sensitivity analysis

• Application of results

Meta-Analysis: Study Design

• Must clearly define focused clinical question

• Provide details of literature searches• Searches should be comprehensive

– Published and unpublished data– References within publications– Foreign language literature

• Blind selection of articles

by >1 reviewer

Meta-Analysis: Selection

• Identify how articles selected for inclusion prior to project initiation

• Identify variables pooled to answer question prior to project initiation

• Identify quality of articles to be selected prior to project initiation

• Weighting of articles?

Meta-Analysis: Combinability

• Determine range of variables to be included– Identify data at identical timepoints– Identify baseline and therapeutic

intervention data to be included

• Evaluate homogeneity of the outcome variables

Meta-Analysis: Bias

• Publication bias• Language bias• Reference bias• Selection bias• Duplicate publication bias• Data extraction bias• Support bias• Bias within clinical trial

Meta-Analysis:Application of Results to Practice

• Relative Risk Ratio

• Absolute Risk Reduction

• Number Needed to Treat

• Consideration to trials not included

Evidence Based Medicine

Explicit and judicious use of current best evidence in making decisions about the care of individual patients

Evidence Based Medicine

• Convert information needs into a clearly defined answerable clinical question

• Conduct a systematic search for the best available evidence for the problem

• Evaluate validity & applicability of evidence• Prepare a synthesis or summary of the evidence

for decision making and implement the decision in practice• Evaluate performance & follow-up on any areas for improvement