Introductory StatisticsIntroductory Statistics

Brian Wells, MSM, MPHBrian Wells, MSM, MPH

With contributions from I. Alan Fein, MD, MPHWith contributions from I. Alan Fein, MD, MPH

““It’s all about observing It’s all about observing what you see.”what you see.”

Yogi Berra, Great American Yogi Berra, Great American philosopherphilosopher

Intro to StatisticsIntro to Statistics

Statistics – What’s It All Statistics – What’s It All About?About?

1.1. The hunt for the truth.The hunt for the truth.2.2. Finding relationships and causalityFinding relationships and causality3.3. Separating the wheat from the chaff Separating the wheat from the chaff

[lots of chaff out there!][lots of chaff out there!]4.4. A process of critical thinking and an A process of critical thinking and an

analytic approach to the literature analytic approach to the literature and researchand research

5.5. It can help you avoid contracting the It can help you avoid contracting the dreaded “data rich but information dreaded “data rich but information poor” syndrome!poor” syndrome!

Statistics and the “Truth”Statistics and the “Truth”

Can we ever know the truth?Can we ever know the truth?

Statistics is a way of telling us Statistics is a way of telling us the likelihood that we have the likelihood that we have arrived at the truth of a matter arrived at the truth of a matter [or not!].[or not!].

Statistics – Based on Statistics – Based on Probability and Multiple Probability and Multiple

AssumptionsAssumptions An approach to searching for the truth An approach to searching for the truth

which recognizes that there is little, if which recognizes that there is little, if anything, which is concrete or anything, which is concrete or dichotomousdichotomous

Employs quantitative concepts like Employs quantitative concepts like “confidence”, “reliability”, and “confidence”, “reliability”, and “significance” to get at the “truth”“significance” to get at the “truth”

Statistics & the “Truth”Statistics & the “Truth”

Two kinds of statistics:Two kinds of statistics: Descriptive – describes “what is”Descriptive – describes “what is” Experimental – makes a point – tries to Experimental – makes a point – tries to

“prove” something“prove” something Problem: Almost impossible to prove Problem: Almost impossible to prove

something, but much easier to disprovesomething, but much easier to disprove Thus: Null hypothesis HThus: Null hypothesis Hoo i.e., there is no i.e., there is no

differencedifference

Types of Statistics Types of Statistics

Descriptive: describe / communicate Descriptive: describe / communicate what you see without any attempts at what you see without any attempts at generalizing beyond the sample at handgeneralizing beyond the sample at hand

Inferential: determine the likelihood that Inferential: determine the likelihood that observed results:observed results: Can be generalized to larger Can be generalized to larger

populationspopulations Occurred by chance rather than as a Occurred by chance rather than as a

result of known, specified result of known, specified interventions [interventions [HHoo]]

The Null HypothesisThe Null Hypothesis

A hypothesis which is tested for possible A hypothesis which is tested for possible rejection under the assumption that it is rejection under the assumption that it is true (usually that observations are the true (usually that observations are the result of chance). result of chance).

Experimental stats works to disprove the Experimental stats works to disprove the null hypothesis, to show that the null null hypothesis, to show that the null hypothesis is wrong, that a difference hypothesis is wrong, that a difference exists. [e.g., glucose levels and diabetics] exists. [e.g., glucose levels and diabetics] In other words, that you have found or In other words, that you have found or discovered something new.discovered something new.

The null hypothesis usually represents the The null hypothesis usually represents the opposite of what the researcher may opposite of what the researcher may believe to be true.believe to be true.

Normality Normality

When variability in data points is due to the sum of When variability in data points is due to the sum of numerous independent sources, with no one source numerous independent sources, with no one source dominating, the result should be a normal, or Gaussian dominating, the result should be a normal, or Gaussian distribution (named for Karl Friedrich Gauss).distribution (named for Karl Friedrich Gauss).

Note: technically, true Gaussian distributions do not Note: technically, true Gaussian distributions do not occur in nature: Gaussian distributions extend infinitely occur in nature: Gaussian distributions extend infinitely in both directions. Bell shaped curves are the norm for in both directions. Bell shaped curves are the norm for biological data, with end points to the right and left of biological data, with end points to the right and left of the mean.the mean.

Bell curve vs. normal distribution vs. Gaussian Bell curve vs. normal distribution vs. Gaussian distributiondistribution

Normal distribution is unfortunately named because it Normal distribution is unfortunately named because it encourages the fallacy that many or all probability encourages the fallacy that many or all probability distributions are "normal". distributions are "normal".

Why Is the Normal Distribution Why Is the Normal Distribution Important?Important?

Provides the fundamental mathematical Provides the fundamental mathematical substrate permitting the use and calculation of substrate permitting the use and calculation of most statistical analyses.most statistical analyses.

The ND represents one of the empirically The ND represents one of the empirically verified elementary “truths about the general verified elementary “truths about the general nature of reality” and its status can be nature of reality” and its status can be compared with one of the fundamental laws of compared with one of the fundamental laws of natural sciences. The exact shape of the ND natural sciences. The exact shape of the ND is defined by a function which has only two is defined by a function which has only two parameters: mean and standard deviation.parameters: mean and standard deviation.

Gaussian DistributionGaussian Distribution

Why Examine the Data?Why Examine the Data?

Means are the most commonly used Means are the most commonly used measures of populationsmeasures of populations Amenable to mathematical manipulationAmenable to mathematical manipulation Handy measure of central tendencyHandy measure of central tendency

BUT – can be misleadingBUT – can be misleading Examine the curve, median, mode, Examine the curve, median, mode,

range, and outliersrange, and outliers Look for bi- or multi-modal Look for bi- or multi-modal

distributionsdistributions

Beware small samples!Beware small samples!

Fancy statistical tests can bury Fancy statistical tests can bury the truth here!the truth here!

Conversely, not finding any Conversely, not finding any difference means nothingdifference means nothing

Normality testing is pointless Normality testing is pointless when there are less than 20-30 when there are less than 20-30 data points – can be misleadingdata points – can be misleading

Variability of MeasurementsVariability of Measurements

Unbiasedness: tendency to arrive at the true Unbiasedness: tendency to arrive at the true or correct valueor correct value

Precision: degree of spread of series of Precision: degree of spread of series of observations [repeatability] [also referred to observations [repeatability] [also referred to as reliability] Can also refer to # of decimal as reliability] Can also refer to # of decimal points – can be misleadingpoints – can be misleading

Accuracy: encompasses both unbiasedness Accuracy: encompasses both unbiasedness and precision. Accurate measurements are and precision. Accurate measurements are both unbiased and precise. Inaccurate both unbiased and precise. Inaccurate measurements may be either biased or measurements may be either biased or imprecise or both.imprecise or both.

The MeanThe Mean

vs X vs X Potentially dangerous and Potentially dangerous and

misleading valuemisleading value = = true population meantrue population mean Xbar = mean of sampleXbar = mean of sample

Standard DeviationStandard Deviation

SD is a measure of scatter or dispersion of SD is a measure of scatter or dispersion of data around the meandata around the mean

~68% of values fall within 1 SD of the ~68% of values fall within 1 SD of the mean [+/- one Z]mean [+/- one Z]

~95% of values fall within 2 SD of the ~95% of values fall within 2 SD of the mean, with ~2.5% in each tail.mean, with ~2.5% in each tail.

Single Sample MeansSingle Sample Means

The mean value you calculate from The mean value you calculate from your sample of data points depends your sample of data points depends on which values you happened to on which values you happened to sample and is unlikely to equal the sample and is unlikely to equal the true population mean exactly.true population mean exactly.

Confidence IntervalsConfidence Intervals

A confidence interval gives an estimated range of A confidence interval gives an estimated range of values which is likely to include an unknown values which is likely to include an unknown population parameter, the estimated range being population parameter, the estimated range being calculated from a given set of sample data.calculated from a given set of sample data.

If independent samples are taken repeatedly from If independent samples are taken repeatedly from the same population, and a confidence interval the same population, and a confidence interval calculated for each sample, then a certain calculated for each sample, then a certain percentage (confidence level) of the intervals will percentage (confidence level) of the intervals will include the unknown population parameter. include the unknown population parameter. Confidence intervals are usually calculated so that Confidence intervals are usually calculated so that this percentage is 95%, but we can produce 90%, this percentage is 95%, but we can produce 90%, 99%, 99.9% (or whatever) confidence intervals for 99%, 99.9% (or whatever) confidence intervals for the unknown parameter.the unknown parameter.

Confidence IntervalsConfidence Intervals

The width of the confidence interval gives us The width of the confidence interval gives us some idea about how uncertain we are about some idea about how uncertain we are about the unknown parameter (see precision). A the unknown parameter (see precision). A very wide interval may indicate that more very wide interval may indicate that more data should be collected before anything very data should be collected before anything very definite can be said about the parameter.definite can be said about the parameter.

Confidence intervals are more informative Confidence intervals are more informative than the simple results of hypothesis tests than the simple results of hypothesis tests (where we decide "reject Ho" or "don't reject (where we decide "reject Ho" or "don't reject Ho") since they provide a range of plausible Ho") since they provide a range of plausible values for the unknown parameter.values for the unknown parameter.

Errors of Analysis/DetectionErrors of Analysis/Detection

Type 1 [alpha] error – finding a Type 1 [alpha] error – finding a significant difference when none really significant difference when none really exists - usually due to random errorexists - usually due to random error

Type 2 [beta] error – not finding a Type 2 [beta] error – not finding a difference when there is in fact a difference when there is in fact a difference – more likely to occur with difference – more likely to occur with smaller sample sizes.smaller sample sizes.

Chosen significance levels will impact Chosen significance levels will impact both of theseboth of these

Errors of AnalysisErrors of Analysis

TRUE STATEDECISION

Null Hypothesis is Correct

Null Hypothesis is wrong

 Reject Null Hypothesis as Incorrect – i.e., find a difference

 Type I ERROR probability =   

 CORRECT probability = 1-  

"power"

Accept Null Hypothesis – decide that there are no effects or differences

CORRECTprobability = 1 -   

Type IIERROR probability =  

Statistical SignificanceStatistical Significance

The probability that the findings are The probability that the findings are due to chance alonedue to chance alone

p<.05 – less than a 5% likelihood of p<.05 – less than a 5% likelihood of findings being due to chancefindings being due to chance

P<.05 is commonly used but P<.05 is commonly used but arbitraryarbitrary

Two-Sample Two-Sample tt-Test for -Test for Means (unequal Means (unequal

variance)variance) Used to determine if two Used to determine if two

population means are equalpopulation means are equal HHoo : : 11 = = 2 2

HHaa: : 11 = = 22

Test Statistic: Test Statistic: DF:DF:

Reject Reject HHo o if:if:

Critical ValuesCritical Values

Assuming that alpha = 0.05 (this Assuming that alpha = 0.05 (this is a standard measure), we can is a standard measure), we can say the following:say the following:

limlimυ => ∞ υ => ∞ tt(α/2, υ)(α/2, υ) = 1.96 = 1.96 So, for So, for υυ > 100, T-Critical = 1.96 > 100, T-Critical = 1.96 For all other values, a For all other values, a t-t-Value Value

table is neededtable is needed

Chart of Critical ValuesChart of Critical Valuest-Test Critical values

1.5

1.7

1.9

2.1

2.3

2.5

2.7

0 20 40 60 80 100 120

Degrees of Freedom

Cri

tic

al

Va

lue

Critical Value Limit t-Test Critical Values

What does it mean?What does it mean?

Rejecting HRejecting Hoo with a positive T-value with a positive T-value means that it is likely that the means that it is likely that the provider has a higher mean billing provider has a higher mean billing level at the 95% confidence levellevel at the 95% confidence level

Again, it’s probability – this does Again, it’s probability – this does not mean we know for sure that not mean we know for sure that the provider’s mean billing level is the provider’s mean billing level is higher than his/her peershigher than his/her peers

Result can be skewed if underlying Result can be skewed if underlying data is sufficiently non-Gaussiandata is sufficiently non-Gaussian

Analytics Analytics RecommendationsRecommendations

Test for equivariance using F-distributionTest for equivariance using F-distribution Add test for normality to database (e.g. Add test for normality to database (e.g.

Kolmogorov-Smirnov, Shapiro-Wilk)Kolmogorov-Smirnov, Shapiro-Wilk) Use non-parametric test or Use non-parametric test or

transformation when data is non-transformation when data is non-GaussianGaussian

Can not discriminate between a Can not discriminate between a physician billing higher levels because of physician billing higher levels because of upcoding and a physician simply seeing upcoding and a physician simply seeing sicker patients than his/her peers – need sicker patients than his/her peers – need clinical outcomes measureclinical outcomes measure

Wells 2006

Thank you!Thank you!