Introduction to Statistical Methods

By Tom Methven

Digital slides and tools available at:www.macs.hw.ac.uk/~mjc/teaching/

ResearchMethods

Moving Bell-curves

Designing the Experiment

• 1. Define exactly what you want to measure

• 2. Pick which statistical test to use, first

• 3. Decide on your experimental design

Worked Example

• Two different piano designs, iPiano and Wii Piano!

• This example will be built up through the notes to show how to go about designing/analysing a simple experiment.

Level Of Measurement (Non-Parametric)

• Nominal – Labelled results or characteristics– Can be used for equivalence and set membership– Can show commonality but leaves it undefined• E.g. List of names, football players’ numbers

• Ordinal – Results in some sort of rank order– Results do not show differences between places!– No arithmetic is valid, you cannot say 3rd place is

the same as 1st and 2nd combined, for example• E.g. Race results, top 10 films

Level Of Measurement (Parametric)

• Interval – Results placed along a regular scale, with an arbitrary zero-point– Multiplication/Division is meaningless but the ratio of

differences is valid as differences are consistent• E.g. Degrees Fahrenheit

• Ratio – Results placed along a regular scale with an absolute zero-point– Allows for statements such as 6 units is twice as much

as 3 units• E.g. Kelvin and most SI units

Statistic Basics

• Mode The most common score• Median The central score• Mean The ‘average’ of all the scores

• E.g. For the results 9,2,5,3,6,9,5,6,4,2,6• Mode: 6

Median: 5• Mean: 5.18

Worked Example Results

• Using time to successfully play a simple tune as the measure.

• Therefore, we’re using a ratio scaling, so we can use parametric tests.

• Results:

Interface 1 Interface 2

Person 1 4.28 4.38

Person 2 2.78 4.99

Person 3 7.63 4.30

Person 4 7.93 4.27

Person 5 7.19 5.50

Person 6 5.73 5.22

Person 7 8.40 4.09

Person 8 5.88 4.46

Person 9 5.60 4.00

Person 10 4.89 4.90

Mean: 6.03 4.61

Randomisation and Ordering Effects

• Performing an experiment gives experience which could improve subjects’ ‘scores’

• For a two-part test, an equal number of people should try each condition first

• When you have many conditions or trials, it is easiest to show then in a random order

Latin Squares

• A way of counter-balancing condition order for small amounts of conditions

• E.g. For three possible conditions:

• You can also get ‘balanced’ Latin squares for experiments with even number of conditions.

Order of conditions or trials

Group 1 A B C

Group 2 B C A

Group 3 C A B

Accuracy of the Mean

• Variance:

• Standard Deviation:

• Standard Error:

𝑠2 = σ(𝑥𝑖 − 𝑥ҧ)2𝑁− 1

𝑠= ඥ𝑠2

𝑠𝑒 = 𝑠ξ𝑛

Degrees of Freedom

• For sample populations, often ‘N – 1’ is used rather than N. This is the simplest calculation of D.O.F., but it can get very complex.

• We assume that the sample mean is the same as the population mean. Therefore, it is related to how many values are free to vary without altering the required mean value. (Rugby example)

Student’s T-Test

• Used for comparing the means of two sample groups using the variance

• Several types, dependant on the samples:– ‘Dependant/Paired’ – Using this one!– ‘Independent Equal Variance’– ‘Independent Unequal Variance’

• Assumes populations are normally distributed

Student’s T-Test

• Create a ‘null hypothesis’:– E.g. There is no difference in learning between

iPiano and Wii Piano

• Create an ‘alternative hypothesis’:– One-tailed: Wii Piano is easier to learn– Two-tailed: Wii Piano is easier or harder to learn

Dependent T-Test

• Used to compare the results of two groups

= Average difference = Expected difference (0 for null hypothesis) = Standard deviation of differences = Sample Size

𝑡 = 𝐷ഥ− 𝜇𝐷 𝑆𝐷 ξ𝑁Τ

𝐷ഥ 𝜇𝐷 𝑆𝐷 𝑁

Worked Example T Result

= 1.420756421

= 1.985348881

= 10

t-value = 2.26

𝐷ഥ

𝑆𝐷

𝑁

Interpreting T-Value

• The T-Test gives a t-value, which can be converted to a p-value

• If the p-value < 0.05 there is less than a 5% chance the null hypothesis is true

• Therefore we conclude that the change of condition caused the observed effect

• Our p-value was 0.025, so we can accept our alternate hypothesis: ‘Wii Piano is easier to learn’

Effect Size

• Tells us how important the result is in practical terms. Gives an ‘r-value’.

– r = 0.10 (small effect) – 1% of total variance– r = 0.30 (medium effect) – 9% of total variance– r = 0.50 (large effect) – 25% of the variance– r DOES NOT work on a linear scale, so these can only be used as good guidelines. It is becoming more

common to include the r-value as well as the p-value as it is a well understood and widely used check.

𝑟= ඨ 𝑡2𝑡2 + 𝑑𝑓

[letter]-values

Recap:• t-value: Result of the t-test

– Normally converted into a ‘p-value’ for easy comparison between statistical tests. Used to calculate ‘r-value’

• p-value: Is it statistically significantly?– If ‘p-value’ is less than 0.05, then you can discard the null

hypothesis and accept your alternate hypothesis.

• r-value: Is the effect substantial in reality?– Does the result mean anything in real terms? HOW MUCH

does our experiment explain the change in means?

Final Results

• p-value = 0.025• r-value = 0.60• Degrees of freedom = 9

• “The results show that Wii Piano allows users to play a set tune successfully significantly faster than iPiano (p = 0.025). In addition, the effect size was large (r = 0.6), showing the result was substantial in real terms.”

Error Bars

Error bars: Plot standard error

Excel Example

• TTEST in Excel will give a ‘p-value’ directly

Summing Up

• Dependant t-test when using a single group– For example, when using 10 friends to test two interfaces

• Avoid ordering effects– Randomise order and get an even number to do each test first

• Use ‘TTEST’ in Excel to get p-value easily– Easier than calculating it yourself, but harder to get r-value!

• Check p < 0.05 and quote the value and result– Always make sure to quote the actual p-value in your results

as well as saying whether the result was statistically significant

Recommended Reading