What is a paired samples t test

Paired Samples t-tests

Another important test of differences is the t-test for paired samples. This is also known as a t-test for repeated measures or a t-test for matched samples.

Another important test of differences is the t-test for paired samples. This is also known as a t-test for repeated measures or a t-test for matched samples.

Whenever two distributions of a dependent variable are highly correlated, either because they are distributions of pre and post tests from the same people,



one month later


one month later

Whenever two distributions of a dependent variable are highly correlated, either because they are distributions of pre and post tests from the same people, or from two samples that are matched such that there is a one-to-one correspondence between each subject in group one and its matched pair in group two,


one month later

BrownHaired

MatchedSubjects


one month later

one month later

BrownHaired

MatchedSubjects

RedHaired

MatchedSubjects

Whenever two distributions of a dependent variable are highly correlated, either because they are distributions of pre and post tests from the same people, or from two samples that are matched such that there is a one-to-one correspondence between each subject in group one and its matched pair in group two, the appropriate test of differences is the paired samples t-test.

If we have a single dependent variable that exists on an interval or ratio scale such as scores on a test,



and is reasonably normally distributed;







and a single independent variable (e.g., when you took the test)



and a single independent variable (e.g., when you took the test)



and a single independent variable (e.g., when you took the test) which has two levels



and a single independent variable (e.g., when you took the test) which has two levels



and a single independent variable (e.g., when you took the test) which has two levels which are repeated or matched,



and a single independent variable (e.g., when you took the test) which has two levels which are repeated or matched,



and a single independent variable (e.g., when you took the test) which has two levels which are repeated or matched, then we use the pair wise t-test to test for differences between the two samples of the dependent variable.




Single-sample t-test Independent Sample t-test

Paired-sample t-test




Single-sample t-test Independent Sample t-test

Paired-sample t-test

First we begin with the null hypothesis:


There are no significant difference in pre and post scores (dependent variable).



Or: There are no significant difference between group one and its matched group in terms of the dependent variable.










no difference

Another way to state the null-hypothesis for a paired-sample t-test is by hypothesizing that the difference between the pre-post test or matched pairs is ZERO.


Name _________

1. The word of the day is ______2. The State bird of Oklahoma is ______3. The favorite dessert of Queen Elizabeth is

________4. The capital of Siam is ___________5. The best movie of 1972 was ___________6. The worst spelling bee in the world is _______7. What is the capital of Mars?8. How many people fit in a subway car in Tokyo? 9. The best episode of the Doctor Who reboot is

___________________________10. The worst thing you can do with silly putty is

________________11. How heavy is a 15 pound bowling ball?12. What is the best flavor of Mamba fruit chews?13. The weather in Alaska is __________________14. Bugs Bunny is funniest dressed up as a

_____________15. How much does a taxi cost in London?16. What is the best flavor Jello Pudding pop?17. How far is Wichita from Baghdad?18. What is the most important thing to do on Arbor

Day?19. What goes best with vanilla custard?

Name _________







Name _________


________4. The capital of Siam is ___________5. The best movie of 1972 was ___________6. The worst spelling bee in the world is _______7. What is the capital of Mars?8. How many people fit in a subway car in Tokyo?

9. The best episode of the Doctor Who reboot is ___________________________

10. The worst thing you can do with silly putty is ________________11. How heavy is a 15 pound bowling ball?

12. What is the best flavor of Mamba fruit chews?13. The weather in Alaska is __________________14. Bugs Bunny is funniest dressed up as a _____________15. How much does a taxi cost in London?

16. What is the best flavor Jello Pudding pop?17. How far is Wichita from Baghdad?18. What is the most important thing to do on Arbor


Name _________

1. The word of the day is ______

2. The State bird of Oklahoma is ______

3. The favorite dessert of Queen Elizabeth is

________

4. The capital of Siam is ___________

5. The best movie of 1972 was ___________

6. The worst spelling bee in the world is _______

7. What is the capital of Mars?

8. How many people fit in a subway car in Tokyo?

9. The best episode of the Doctor Who reboot is

___________________________

10. The worst thing you can do with silly putty is

________________

11. How heavy is a 15 pound bowling ball?

12. What is the best flavor of Mamba fruit chews?

13. The weather in Alaska is __________________

14. Bugs Bunny is funniest dressed up as a

_____________

15. How much does a taxi cost in London?

16. What is the best flavor Jello Pudding pop?

17. How far is Wichita from Baghdad?

18. What is the most important thing to do on Arbor


Name _________







Name _________




________







___________________________


________________





_____________





Day?

19. What goes best with vanilla custard?

Name _________




________







___________________________


________________





_____________





Day?


Name _________

1. The word of the day is ______2. The State bird of Oklahoma is ______3. The favorite dessert of Queen Elizabeth is ________4. The capital of Siam is ___________5. The best movie of 1972 was ___________6. The worst spelling bee in the world is _______7. What is the capital of Mars?8. How many people fit in a subway car in Tokyo?

9. The best episode of the Doctor Who reboot is ___________________________10. The worst thing you can do with silly putty is ________________11. How heavy is a 15 pound bowling ball?12. What is the best flavor of Mamba fruit chews?13. The weather in Alaska is __________________14. Bugs Bunny is funniest dressed up as a _____________15. How much does a taxi cost in London?16. What is the best flavor Jello Pudding pop?17. How far is Wichita from Baghdad?18. What is the most important thing to do on Arbor Day?19. What goes best with vanilla custard?


Name _________







Name _________







Name _________


________4. The capital of Siam is ___________5. The best movie of 1972 was ___________6. The worst spelling bee in the world is _______7. What is the capital of Mars?8. How many people fit in a subway car in Tokyo?

9. The best episode of the Doctor Who reboot is ___________________________

10. The worst thing you can do with silly putty is ________________11. How heavy is a 15 pound bowling ball?

12. What is the best flavor of Mamba fruit chews?13. The weather in Alaska is __________________14. Bugs Bunny is funniest dressed up as a _____________15. How much does a taxi cost in London?

16. What is the best flavor Jello Pudding pop?17. How far is Wichita from Baghdad?18. What is the most important thing to do on Arbor


Name _________




________







___________________________


________________





_____________






Name _________







Name _________




________







___________________________


________________





_____________





Day?


Name _________




________







___________________________


________________





_____________





Day?


Name _________

1. The word of the day is ______2. The State bird of Oklahoma is ______3. The favorite dessert of Queen Elizabeth is ________4. The capital of Siam is ___________5. The best movie of 1972 was ___________6. The worst spelling bee in the world is _______7. What is the capital of Mars?8. How many people fit in a subway car in Tokyo?

9. The best episode of the Doctor Who reboot is ___________________________10. The worst thing you can do with silly putty is ________________11. How heavy is a 15 pound bowling ball?12. What is the best flavor of Mamba fruit chews?13. The weather in Alaska is __________________14. Bugs Bunny is funniest dressed up as a _____________15. How much does a taxi cost in London?16. What is the best flavor Jello Pudding pop?17. How far is Wichita from Baghdad?18. What is the most important thing to do on Arbor Day?19. What goes best with vanilla custard?

= 0−

Conceptually, aggregating the exact differences between each pair makes most sense and leads to the correct degrees of freedom, sampling distribution and critical value.

Conceptually, aggregating the exact differences between each pair makes most sense and leads to the correct degrees of freedom, sampling distribution and critical value.

Correct Degrees of Freedom– 50 people take a pre-test – Same 50 take a post test – 50 – 1 = 49 degrees of freedom

Correct Sampling Distribution

Correct Sampling Distribution

Sample Mean Distribution of difference between the pre and

post samples test scores

Sample Mean Distribution of pre-test scores.

Sample Mean Distribution of post-test scores.

=−

and Correct Critical Value

and Correct Critical Value

The formula for the paired samples t-test is:


Σ(X1-X2)SEdifft =


Σ(X1-X2)SEdifft =Paired t-test value or

the number of standard error

values that separate these two means

The formula for the paired samples t-test is:Pre Post

1 7

2 6

1 8

Σ(Xpre-Xpost)SEdifft =


1 7

2 6

1 8


Subtract each pretest score from each

posttest score and the sum them up


1 7

2 6

1 8


Difference

6

4

7


1 7

2 6

1 8


Difference

6

4

7

Sum of all Differences

6 + 4 + 7 = 17


1 7

2 6

1 8


Difference

6

4

7


6 + 4 + 7 = 17

17SEdifft =

The formula for the paired samples t-test is:Σ(Xpre-Xpost)SEdifft =


Now let’s calculate the Standard Error of the

difference


Now let’s calculate the Standard Error of the

difference

It is this statistic (Standard Error of the Difference) that makes it possible to determine if this difference occurred only by chance or if there is a strong probability that they are actually different!

One thing to note: If the Standard Error of the Difference is large (say 170) then the t value will be small.


For example:


For example:Σ(Xpre-Xpost)SEdifft =


For example:Σ(Xpre-Xpost)SEdifft = 17170t =


For example:Σ(Xpre-Xpost)SEdifft = 17170t =

.01t =

However, if the Standard Error of the Difference is small (say 1.7) then the t value will be larger.


For example:


For example:Σ(Xpre-Xpost)SEdifft =


For example:Σ(Xpre-Xpost)SEdifft = 171.7t =


For example:Σ(Xpre-Xpost)SEdifft = 171.7t =

10t =

So what effect will a smaller or larger estimated standard error have on whether a result is statistically significantly different or not?


Well let’s say that our null hypothesis (Ho) is that there is no statistically significant difference between the pre and post test scores below:


Well let’s say that our null hypothesis (Ho) is that there is no statistically significant difference between the pre and post test scores below:

Students Pre Post

1 1 7

2 2 6

3 1 8

mean: 1.3 6.0

Of course, the alternative hypothesis would be that there is no statistically significant difference between the two.


With a t value of .01


With a t value of .01Σ(Xpre-Xpost)SEdifft = 17170t =

.01t =


With a t value of .01, and a sample of 3, and therefore degrees of freedom of 2, we would look up the t critical value at a .05 alpha level (a .05 alpha level means that we are willing to consider an outcome to be significant if it were likely to happen 5 times out of 100 (.05), in other words, if it were a very rare occurrence.)

So we go to our table of t-Distribution Critical Values to find the critical value that needs to be exceeded by our result in order be considered statistically significantly different.



So, the critical value we need to exceed in order to reject the null hypothesis is 2.920.



However, our t-value is



However, our t-value is.01t =



However, our t-value is .01

Our t value of 0.1 does not exceed the t critical of 2.920, therefore we will fail to reject the null-hypothesis (which essentially means to accept the null-hypothesis).

On the other hand, when our t value is 10, under the same conditions,

On the other hand, when our t value is 10, under the same conditions,Σ(Xpre-Xpost)SEdifft = 171.7t =

10t =

On the other hand, when our t value is 10, under the same conditions, our t value of 10 does exceed the t- critical of 2.920, therefore we will reject the null-hypothesis (which essentially means to accept the alternative hypothesis).

So when it comes to inferential statistics (inferring meaning to a larger population from a smaller sample), the size of the standard error determines everything.

So when it comes to inferential statistics (inferring meaning to a larger population from a smaller sample), the size of the standard error determines everything.

Understanding the standard error theoretically may or may not be important for your learning purposes. If it is not, you may want to click quickly through the next 10 slides. If it is important then consider what follows:

If we took 1000 samples of pre- and post-tests and subtracted each pre-test sample from each post-test sample we would have a sampling distribution called the sampling distribution of differences between pre- and post-test samples.






=−






this sample






this sample

−






this sample

− this sample






this sample

− this sample

=






this sample

− this sample

= this sample






this sample






this sample

−






this sample

− this sample






this sample

this sample

=−






this sample

this sample

this sample

=−






this sample

this sample

this sample

=−

etc …

If you calculated the standard deviation of this new subtracted sampling distribution you would have the actual standard error we are looking for for this equation.

If you calculated the standard deviation of this new subtracted sampling distribution you would have the actual standard error we are looking for for this equation.

Since this is almost impossible to do, the standard error will be estimated.

Let’s see how this is done with a very simple data set. Let’s begin with our null-hypothesis: “Post-test scores are not statistically significantly higher than pre-test scores”.

Let’s see how this is done with a very simple data set. Let’s begin with our null-hypothesis: “Post-test scores are not statistically significantly higher than pre-test scores”.

Students Pre Post

1 1 7

2 2 6

3 1 8

mean: 1.3 7.0

We will calculate each element of the equation below:

We will calculate each element of the equation below:

Let’s begin with the sum of the differences between the pre and post tests:






the same



Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17


17

Just as a contrast, when the difference is smaller, then the sum of all differences will be smaller as well:


Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17


Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17


Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17

Pre Post

4 7

5 6

5 8


Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17

Pre Post

4 7

5 6

5 8

Difference

3

1

3


Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17

Pre Post

4 7

5 6

5 8

Difference

3

1

3


3 + 1 + 3 = 7

Back to our original equation. The next step is to estimate the standard


17


We begin with “n” or the size of the sample, which in this case is 3.

17

3


We begin with “n” or the size of the sample, which in this case is 3.

17

3

3


We begin with “n” or the size of the sample, which in this case is 3. Then we compute Σd2.

17

3

3



Pre Post

1 7

2 6

1 8

Difference

6

4

7



Pre Post

1 7

2 6

1 8

Difference

6

4

7

Squared Difference

36

16

49



Pre Post

1 7

2 6

1 8

Difference

6

4

7

Squared Difference

36

16

49


36 + 16 + 49 = 101

Σd2



Let‘s plug in our numbers:



17

3

3

101


We begin with “n” or the size of the sample, which in this case is 3. Then we compute Σd2. Then we compute (Σd)2.

17

3

3

101



Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17



Pre Post

1 7

2 6

1 8

Difference

6

4

7


6 + 4 + 7 = 17

Squared Sum of all Differences

172 = 289



Let‘s plug in our numbers and then do the calculations:



17

3

3

101 289



17

303

3

289



17

14

2



17

7



17

2.646



6.425

So what does a t value of 6.425 mean? Well, that depends on whether it is larger or smaller than the critical t value.


Do you remember how we determine the critical t value?


Do you remember how we determine the critical t value?All we need is


Do you remember how we determine the critical t value?All we need is • the degrees of freedom (sample size (3) minus 1) and


Do you remember how we determine the critical t value?All we need is • the degrees of freedom (sample size (3) minus 1) and • the alpha level we are willing to live with (in this case

.05. This basically means that we are willing to live with being wrong 5 out of 100 times about our decision.)

So with a degrees of freedom of 2 and an alpha level of .05,


So with a degrees of freedom of 2 and an alpha level of .05, our critical t value is: 2.920

So with a degrees of freedom of 2 and an alpha level of .05, our critical t value is: 2.920

Since our t value (6.245) is greater than our critical t value (2.920), we will reject the null hypothesis and accept the alternative hypothesis that students’ post-test scores are higher than their pre-test scores.

To visualize this on a graph we create a t distribution with degrees of freedom of 2. Since the sample size is so small, this will be a much flatter distribution than a normal distribution.


Here is an example of a normal distribution:





And here is an example of a t-distribution with 2 degrees of freedom:



And here is an example of a t-distribution with 2 degrees of freedom:

You may recall that as the degrees of freedom increase the t-distribution begins to approximate the normal distribution.


Degrees of freedom = 5









At about degrees of freedom of 30 the t-distribution looks almost identical to the normal or standard or z-distribution.

At about degrees of freedom of 30 the t-distribution looks almost identical to the normal or standard or z-distribution.

Let’s go back to our example of a t-distribution with 2 degrees of freedom.



Here is the location of the critical t value of 2.920.


Here is the location of the critical t value of 2.920. 2.920



This means that if there really is no statistical difference between the pre- and post-tests, if we were to hypothetically draw 100 samples 95% of the time those sample would be drawn from this part of the distribution.

2.920



This means that if there really is no statistical difference between the pre- and post-tests, if we were to hypothetically draw 100 samples 95% of the time those sample would be drawn from this part of the distribution.

2.920

95% of samples would be on this side of the distribution



This means that if there really is no statistical difference between the pre- and post-tests, if we were to hypothetically draw 100 samples 95% of the time those sample would be drawn from this part of the distribution. And 5% of the samples would be on this side of the critical t value.

2.920

And so we would say to ourselves: If the t value lands to the right of 2.920, then we would hypothesize that they are not part of the same distribution but are actually two separate distributions.


2.920

pre-tests post-tests


Since our calculated t value is 6.245, we will reject the null hypothesis and accept the alternative hypothesis that they are two different distributions.

2.920critical t

value



Since our calculated t value is 6.245, we will reject the null hypothesis and accept the alternative hypothesis that they are two different distributions.

2.920critical t

value


6.245calculated

t value

Let’s see an example where the pre- and post-test scores are closer to one another. Will their difference be statistically significantly different?


Here is the data set:



Students Pre Post

1 6 7

2 6 6

3 7 8

mean: 6.3 7.0



Here is the null hypothesis: “Post-test scores are not statistically significantly higher than pre-test scores”.

Students Pre Post

1 6 7

2 6 6

3 7 8

mean: 6.3 7.0

We will calculate again each element of the equation below:

We will calculate again each element of the equation below:




the same


Pre Post

6 7

6 6

7 8

Difference

1

0

1


1 + 0 + 1 = 2

Plug in the sum of all differences:

2


1 + 0 + 1 = 2

The “n” or the sample size is 3.

2


2

3


2

3

3

Then we compute Σd2

Then we compute Σd2

Pre Post

6 7

6 6

7 8

Difference

1

0

1

Squared Difference

1

0

1


1 + 0 + 1 = 2

Σd2

Let’s plug in our numbers:


2

3

3

2

Then we compute (Σd)2

Then we compute (Σd)2

Pre Post

6 7

6 6

7 8

Difference

1

0

1


1 + 0 + 1 = 2

Squared Sum of all Differences

22 = 4



2

3

3

2 4

Let’s plug in our numbers and then do the calculations:

2

3

3

2 4


2

6

3

4


2

2

2


2

1


2

1


2.0



So with a degrees of freedom of 2 and an alpha level of .05, our critical t value is: 2.920.

So with a degrees of freedom of 2 and an alpha level of .05, our critical t value is: 2.920.

Since our t value (2.0) is less than our critical t value (2.920) we will reject the null hypothesis and accept the alternative hypothesis that students’ post-test scores are higher than their pretest scores.

Here is the location of the critical t value of 2.920:


2.920


This means that if there really is no statistical difference between the pre and posttests, if we were to hypothetically draw 100 samples, 95% of the time those samples would be drawn from this part of the distribution.

2.920

95% of samples would be on this side of the distribution


This means that if there really is no statistical difference between the pre and posttests, if we were to hypothetically draw 100 samples, 95% of the time those samples would be drawn from this part of the distribution. And 5% of the samples would be on this side of the critical t value.

2.920

If the t value lands to the left of 2.920 then we would say to ourselves,

2.920


“that is such a common occurrence that I hypothesize that they are the same distribution and not two separate distributions.”

2.920 Critical t

value

2.0Calculated

t value



2.920 Critical t

value

2.0Calculated

t value

pre-tests



2.920 Critical t

value

2.0Calculated

t value

pre-tests

post-tests



Since our t value is 2.0, we will fail to reject the null hypothesis.

2.920 Critical t

value

2.0Calculated

t value

pre-tests

post-tests

In Summary: The paired sample t-test used in hypothesis testing to determine if to two matched samples (e.g., pre / posttests or matched in some other way) are statistically significantly different from one another.

End of Presentation

Education

What is a paired samples t test