t-tests

Dr Bryan Mills

Why?

When

• 2 sets of data with normal distributions

• A difference is considered significant if the probability of getting that difference by random chance is very small.

• P value:– The probability of making an error by chance

• Historically we use p < 0.05

The magnitude of the effect

How Different?

The Spread of Data

n

Hypothesis Tests

• Hypothesis:– A statement which can be proven false

• Null hypothesis HO:– “There is no difference”

• Alternative hypothesis (HA):– “There is a difference…”

• Try to “reject the null hypothesis”– If the null hypothesis is false, it is likely that our

alternative hypothesis is true– “False” – there is only a small probability that the

results we observed could have occurred by chance

AlphaLevel

Reject Null Hypothesis

P > 0.05Not

significantNo

P < 0.05 1 in 20 Significant Yes

Types of Error

Accept NullReject Null

(assume difference)

Null is TrueCorrect

DecisionType I Error

Alpha

Null is False(true difference)

Type II ErrorBeta

Correct Decision

Paired Two-Sample For Means

• I have two sets of data, one before an experiment (change effect) one after. Are the means significantly different (2-tailed), is the first greater (1-tailed) or is there no difference (null hypothesis).

• i.e. is there greater variation between the two samples than within the samples

• For example - have students tests scores improved after a revision session, have average wages improved after a government initiative has been put in place?

Two-Sample Assuming Equal Variances analysis tool

• homoscedastic t-test - has same variance• I have two sets of data from two different

settings (Grades for women v grades for men, mean profit Cornish firms v mean profit Devon firms). Do they share a common parent population (are all three means the same, population, men, women).

• Are the means significantly different (2-tailed), is the first greater (1-tailed) or is there no difference (null hypothesis).

Two-Sample Assuming Unequal Variances

• heteroscedastic t-test - has different variances

Tails

• NOTE: It is always more difficult to demonstrate 2-tails as the part of distribution you are looking for is reduced (0.05/2).

1-Tail 0.05 2 -Tail 0.025 * two

Excel Output

t-Test: Two-Sample Assuming Unequal Variances

sample 1 sample 2

Mean 23.241 17.597

Variance 83.381 156.742

Observations 30.000 30.000

Hypothesised Mean Diff 0.000

df 53.000

t Stat 1.995

P(T<=t) one-tail 0.026 You need this !

t Critical one-tail 1.674

P(T<=t) two-tail 0.051

t Critical two-tail 2.006

Old Way

• T-stat is above One tailed critical - retain (reject Ho)

• Two tailed is below - reject (retain Ho)

• t Stat 1.995

• t Critical one-tail 1.674

• t Critical two-tail 2.006

  Women Men   Men 2 Women 2

Mean 62.27 60.12 Mean 60.38 64.53

Variance 4.77 3.84 Variance 63.78 89.34

Observations 20 20 Observations 20 20

Hypothesized Mean Difference 0

Hypothesized Mean Difference 0

df 38 df 37

t Stat 3.28 t Stat -1.50

P(T<=t) one-tail 0.001 p<0.05 P(T<=t) one-tail 0.071 p>0.05

t Critical one-tail 1.69 significant difference t Critical one-tail 1.69 not significant

P(T<=t) two-tail 0.00 P(T<=t) two-tail 0.14

t Critical two-tail 2.02   t Critical two-tail 2.027  

60 6560 62

t-Test: Paired Two Sample for Means

sample 1 sample 2

Mean 23.241 17.597

Variance 83.381 156.742

Observations 30.000 30.000

Pearson Correlation -0.036

Hypothesised Mean Difference 0.000

df 29.000

t Stat 1.962

P(T<=t) one-tail 0.030

t Critical one-tail 1.699

P(T<=t) two-tail 0.059

t Critical two-tail 2.045

The difference in [whatever the data represents] between sample 1 (M = 23.241, VAR = 83.381) and sample 2 (M = 17.597, VAR = 156.742) was statistically significant, t (29) = 1.962, p < .05, one-tailed.

Non-parametric alternatives

• Mann-Whitney U test

Heights of males (cm)

Heights of females (cm)

Ranks of male heights

Ranks of female heights

193 175 1 7

188 173 2 8

185 168 3 10

183 165 4 11

180 163 5 12

178 6

170 9

n1 = 7 n2 = 5 R1 = 30 R2 = 48

U = nU = n11nn22 + + nn11(n(n11+1)+1) – R – R11

22

U=(7)(5) + U=(7)(5) + (7)(8)(7)(8) – 30 – 30 22

U = 35 + 28 – 30U = 35 + 28 – 30

U = 33U = 33

Which is then compared to a Which is then compared to a table of critical valuestable of critical values

http://www.umes.edu/sciences/MEESProgram/ExperimentalDesign/Parametric%20versus%20Nonparametric%20Statistics.ppt