Describing and Visualizing Experimental Data, &Tukey’s Quick Test for Significance

STAT:5201

Week 3: Lecture 1

1 / 15

Numerical Summaries

You should be familiar with the usual numerical summaries.

* mean* standard deviation* quartiles (e.g. Q1 or lower quartile, Q3 or upper quartile)* IQR or interquartile range

2 / 15

Graphical Descriptions

Graphical Descriptions: Categorical

Plots can be a very powerful tool for conveying information.

You should be familiar with univariate and bivariate (grouped barchart) plots for categorical variables...

1 Univariate Summaries

There were n = 207 respondents in the data set used for analysis (NOTE: 6 observations were removedprior to analysis as they were collected during May of 2016 and assumed to be test input). The summarydistribution (frequencies) of categorical variables is presented graphically below.

1.1 Visitor Status

1.1.1 Are you a Visitor?

Based on the information from Question 4a, about 40% (n=85) of survey respondents were visitors to IowaCity, and 60% were not visitors (n=122).

0%

20%

40%

60%

Yes No

Visitor

Visitor

Figure 1: Visitor to Iowa City?

1.1.2 If a Visitor, is Festival the Primary Reason for Visiting Iowa City?

As stated above, 85 of the 207 respondents were visitors to Iowa City. Visitors were asked if the festival wastheir primary reason for visiting Iowa City, and 44% (n=37) of the visitors said it was their primary reasonfor coming to Iowa City while 56% (n=48) said it was not.

0%

20%

40%

Yes No

Primary.Reason

Primary.Reason

For those respondents who are visitors:

Figure 2: Within visitor population, festival is primary reason for visiting?

2 Bivariate Relationships

Some of the relationships between variables are shown in graphics below.

2.1 Education and Age

Almost all of the ‘Some High School’ respondents appear in the 14-18 age group. The 18-21 age group hasa majority in the ‘Some College/Trade School’ education group. The 22-28 age group has a most commoncategory of ‘College Graduate’. Higher levels of education tend to be seen in the higher age groups.

14-18 18-21 22-28 29-40 41-55 56-65 Over 65 Prefer Not to Disclose

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

Som

e H

igh

Sch

ool

Hig

h S

choo

l Gra

duat

eS

ome

Col

lege

\Tra

de S

choo

lC

olle

ge G

radu

ate

Gra

duat

e or

Pro

fess

iona

l Sch

ool

Pre

fer N

ot to

Dis

clos

e

0

10

20

Education

count

EducationSome High School

High School Graduate

Some College\Trade School

College Graduate

Graduate or Professional School

Prefer Not to Disclose

Figure 13: Education level by Age.

2.2 Visitor Status and Age

Overall, 40% of all respondents were visitors. One age group that tended to have a slightly higher proportionof visitors was the 56-65 age group (⇠56% visitors), while the 22-28 age group tended to have a lowerproportion of visitors (⇠32% visitors).

14-18 18-21 22-28 29-40 41-55 56-65 Over 65 Prefer Not to Disclose

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

Yes No

0

10

20

30

Visitor

count Visitor

Yes

No

Figure 14: Visitor Status by Age.Bar chart Grouped bar chart

3 / 15

Graphical Descriptions

Graphical Descriptions: Categorical

A mosaic plot is a bivariate plot for categorical variables...

FirstGen/Pell Grant combinations

First Generation

Pel

l Gra

nt

No

Yes

Y

N

4 / 15

Graphical Descriptions

Graphical Descriptions: Continuous

You should be familiar with univariate plots and bivariate plots forcontinuous variables...

Post scores of Zero

Post scores of zero: There are a number of post scores with zerovalues.

I Listening Post has 55 zerosI Reading Post has 30 zerosI Writing Post has 110 zeros

From: Shiyang Chen, Yue Zhao, Xun Li To: Reuben Vyn (UIOWA)STAT:6220 Presentation Wednesday 8th March, 2017 10 / 32

Post scores of ZeroPost scores of zero

I Students’ Post vs. Pre scores plots shown above suggest thosepost scores are not truly of zero values (at least for Listening andReading), as many of their pre-scores are much higher than zero(row of dots along the bottom of the x-axis where POST=0).

I These are likely cases where the students were either absent or didnot complete that component of the assessment.

From: Shiyang Chen, Yue Zhao, Xun Li To: Reuben Vyn (UIOWA)STAT:6220 Presentation Wednesday 8th March, 2017 11 / 32

Histogram Scatterplot5 / 15

Graphical Descriptions

Graphical Descriptions: Continuous and Categorical

A categorical and a continuous...

If#you#have#lots#of#observations,#dot$plots#do#not#work#well,#in#that#case,#boxplots#are#a#better#option#for#side8by8side#comparisons.##But#because#boxplots#provide#a#minimal#amount#of#information,#I#prefer#to#overlay#them#with#the#observed#values.#> boxplot(Cholesterol~StatusName)

##(see#next#page#for#R#code#for#the#plot#below)#

#

CPA EuthTrt Survivor

100

200

300

400

100

200

300

400

CPA EuthTrt Survivor

Cholesterol

CPA

EuthTrt

Survivor

4a#

Side-by-side boxplots is OK, but I prefer to see the overlay ofobservations (see R code at the end).

6 / 15

Graphical Descriptions - Barcode Plots

Another option is a barcode plot

T-F you '(01 w/1)a. A fJ h nCVWt &

v~ -Iv

0101(Jkh cu,oI yen, r t~j/~ hoY<2

c7 sJjl7;t:ceurf cJf'rJ;/Y) /'~C/ ~

50»t.fL qr?'cyn I~ ~ h dtTt-U

0n-ejuJ 1'kl~~5 ~J <V!.e ~ ~~ ~ Iv1» s/~c/ on 10;; e/ f<td' cJ~.

,1:5 CL

I I U II I I I

~ 10

Rwill let you add a "barcode plot" to an existing plot.

> library(graphics)

> plot(density.default(x=faithful$eruptions,bw=0.15»

> rug(faithful$eruptions)

density.default(x = faithful$eruptions, bw = 0.15)~~------------------------~o

-eo

'"o

N = 272 Bandwidth = 0.15

R will let you add a barcode plot (or rug plot) to an existing boxplot.

> boxplot(faithful$eruptions)

> rug(faithful$eruptions,side=2,ticksize=.1)

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

7 / 15

Graphical Descriptions - Barcode Plots

Or a density plot.

> plot(density(x=faithful$eruptions))

> rug(faithful$eruptions)

1 2 3 4 5 6

0.0

0.1

0.2

0.3

0.4

0.5

density.default(x = faithful$eruptions)

N = 272 Bandwidth = 0.3348

Density

8 / 15

Graphical Descriptions - Dot Plots

Dot plots

In experiments with a small the number of observations, a dot plotcan work well for displaying data. It’s very similar to a histogram, butwe can visualize every data point.

From Khan Academy ‘Creating dot plots’

You can round your data values to some appropriate level (significantdigits) to allow unequal measurements that are close together to bestacked on top of each other (for visual benefit).

9 / 15

Statistical Significance through Visualization (Tukey)

When comparing two groups, a visualization of the data is probablyone of the first things you will look at.

Example (Pencil lead strength for two brands)

Comparing pencil-lead strength of two brands of pencils. One factor, twotreatments, CRD.

LJL ttJlAfcv./~ -two or~s ...bt-~.' =r-» fVl<eJJ-~ 5/re4trJt%s ~ hvo(D[)D) h/'~5 ~ ~C/~. 2 ~('~5) CRD.

'B~A f *',: •

8\~B , ' \ I, I) • II

'10 'ts 50

faj+ sfr~::c 0-6S-b1Va./r-l5lMV -lAd =uz... &v:v 1J,a.-tjlufcu:J0 c1 ~ 2 fr/lh J'w-; q ,

(1/1;~ I ~ ~ 3 fall 5fl(J~ .; !3f'~ 1/ )

r;iJ 5fr'iJ~;::: klfaJ,"<I>V-V cMd ~ ~~ f/JC<414 slYLaliuc ~ Ik 2- Jr\ Q "t/ rn a .

(/fb0'lA{ ) -I-k thL 3 f'lr :>1f'aJruv~ (j1'~8)

;:fj~ if"M- J\'G>vf 1rJ~ +tu- 0'\CV)( fix.fY1;A ) ~ ~ V\.-O 'S=» d-I e rs .

In 1959, John W. Tukey described a way to find statisticalsignificance by visually comparing dot plots.

The framework here coincides with a two-sample t-test:1) independent groups2) hypothesis test is on the centers of the distributions

10 / 15

Statistical Significance through Visualization (Tukey)

We will make a decision based on stragglers.

Definition (Left stragglers)

Observations that are less than the larger of the 2 minima.

Definition (Right stragglers)

Observations that are greater than the smaller of the 2 maxima.

Example (Pencil lead strength for two brands, CRD)

LJL ttJlAfcv./~ -two or~s ...bt-~.' =r-» fVl<eJJ-~ 5/re4trJt%s ~ hvo(D[)D) h/'~5 ~ ~C/~. 2 ~('~5) CRD.

'B~A f *',: •

8\~B , ' \ I, I) • II

'10 'ts 50

faj+ sfr~::c 0-6S-b1Va./r-l5lMV -lAd =uz... &v:v 1J,a.-tjlufcu:J0 c1 ~ 2 fr/lh J'w-; q ,

(1/1;~ I ~ ~ 3 fall 5fl(J~ .; !3f'~ 1/ )

r;iJ 5fr'iJ~;::: klfaJ,"<I>V-V cMd ~ ~~ f/JC<414 slYLaliuc ~ Ik 2- Jr\ Q "t/ rn a .

(/fb0'lA{ ) -I-k thL 3 f'lr :>1f'aJruv~ (j1'~8)

;:fj~ if"M- J\'G>vf 1rJ~ +tu- 0'\CV)( fix.fY1;A ) ~ ~ V\.-O 'S=» d-I e rs .

In this example, there are 3 lefts stragglers in Brand A, and 3 rightstragglers in Brand B.

If one group has the max & min, then there are no stragglers.11 / 15

Statistical Significance through Visualization (Tukey)

If the two data sets are roughly of equal size, with 4 or moreobservations in each, then there are two simple rules that can be usedto make a decision...

Tukey’s Quick TestIf the total number of stragglers is 8 or more, then the loca-tions can be judged statistically different at the 0.05 level.

Three-straggler Rule (from Prof. Russ Lenth)If there are at least 3 left stragglers and at least 3 right strag-glers, then the locations can be judged statistically differentat the 0.05 level.

In the CRD example on pencil strength (previous slide), the ‘Threestraggler Rule’ says the brands are statistically significantly differentat the 0.05 level.

12 / 15

Statistical Significance through Visualization (Tukey)

When would you use a visualization quick test?

No computer nearby (outdoors, on the floor of a factory, in the field).Quick and easy to remember the ‘critical values’.Can be carried out anywhere.When you have a small sample size.When normality assumption is not met.

If your sample is large enough that you can show approximatenormality, or for the CLT to be applied, then use a classicaltwo-sample t-test as it will be more powerful.

13 / 15

References

From Tukey...

Lowest 6 A's: 14.6, 14.7, 14.8, 14.9, 15.0, 15.1

1 B 13.8

Highest 6 B's: 15.0, 14.7, 14.6, 14.4, 14.3, 4.2

1 A 17.3

B-A differences (L) increasing: -1.3, -1.2, -1.1, -1.0, -0.9, -0.8

(H) decreasing: -2.3, -2.6, -2.7, -2.9, -3.0, -3.1

lesser of 2: -2.3, -2.6, -2.7, -2.9, -3.0, -3.1

highest of these: -2.3.

thus if i is > -(-2.3) = 2.3 we will find "A - X" significantly less than "B"' Consequently the 95% confidence interval for "A - B" obtained by this

procedure runs from +0.3 to +2.3, that is it contains those q's which are not <0.3 or >2.3.

II. EXPRESSIONS AND COMPARISONS 8. General.

All these derivations are founded on the remark which is fundamental to all permutation tests: If the two populations are the same, then (i) the result of taking a sample from each and combining their values into a single ordered list behaves like an ordered list of a single larger sample from either population, and (ii) each order of A's and B's in such a list is equally probable.

We can and will, obtain approximate results for two very large samples easily and, with somewhat more effort, exact results for two "small" samples.

9. The asymptotic case. If the numbers of A's and B's in the two samples are in the ratio of p to q,

where p + q = 1, then the chance that any particular value (in the list of the two samples ordered together) is an A is p, while the chance that it be a B is q. (Note that p = n/(n + N), q = N/(n + N).)

If both samples are very large, the A-ness or B-ness of one chosen value will be very nearly independent of that of any other chosen value. Thus we shall have, approximately

Prob [exactly k highest are A's] = pk(1 - p) = pkq

since the k highest values must be A's and the next highest value a B. Similarly

38 JOHN W. TUKEY

John Tukey (1959)

A quick, compact, two-sample test to Duckworth’s specifications

Technometrics Vol. 1 (1) , 31 – 48.

14 / 15

R code: Side-by-Side BoxPlots

# side-by-side boxplots with data points overlaid

> library(ggplot2)

> ggplot(data=dt, aes(x=StatusName,y=Cholesterol)) +

geom_boxplot(aes(fill=StatusName)) +

## Set colors:

scale_fill_manual(values = c("#CD5C5C",

"#FF7F24","#70944D")) +

## Change to white background and other items

theme_bw() +

theme(panel.grid.major = element_line(colour =

"grey")) +

theme(axis.title.x=element_blank(),

legend.title=element_blank()) +

## Overlay with the jittered points:

geom_point(color="grey10",

position=position_jitter(w=0.1,h=0.1))

15 / 15