1Advanced Statistics

W3: Multivariate ANOVA

Titis Wijayanto

Industrial Engineering, UGM

Quiz

Find the eigenvalues and eigenvectors of this matrix:

2 In this chapter we extend univariate analysis of variance (ANOVA) to multivariate analysis of variance (MANOVA), in which we measure more than one variable on each experimental unit.

Univariate One-Way Analysis of Variance (ANOVA) In the balanced one-way ANOVA, we have a random sample of n

observations from each of k normal populations with equal variances, as in the following layout:

The k samples or the populations from which they arise are sometimes referred to as groups. The groups may correspond to treatments applied by the researcher in an experiment.

3Univariate One-Way Analysis of Variance (ANOVA)

We will have such hypothesis like this:

Univariate One-Way Analysis of Variance (ANOVA)Example

4Univariate One-Way Analysis of Variance (ANOVA)Example

In this case, with 2 and 12 degrees of freedom the critical values are 3.89 (p= .05) and 6.93 (p= .01).

Thus what can we say???

Multivariate Analysis of Variance (MANOVA)

there may be circumstances in which we are interested in several dependent variables (DV) and in these cases the simple ANOVA

model is inadequate. Instead, we can use an extension of this technique known as

multivariate analysis of variance(or MANOVA).

ANOVA can be used only in situations in which there is one dependent variable (or outcome) and so is known as a univariate test (univariate quite obviously

means one variable); MANOVA is designed to look at several dependent variables (outcomes) simultaneously and so is a multivariate test (multivariate means many variables).

5ANOVA vs MANOVA

Anovas run separately cannot take into account the pattern of covariation among the dependent measures

It may be possible that multiple Anovas may show no

differences while the Manova brings them out

So, by measuring multiple DVs you increase your chances for finding a group difference In this sense, in many cases such a test has more power than the univariate

procedure, but this is not necessarily true as some seem to believe

Also conducting multiple ANOVAs increases the chance for type 1 error and MANOVA can in some cases help control for the inflation

ANOVA vs MANOVA

6ANOVA vs MANOVA

F table = 3.354

ANOVA vs MANOVA

F table = 3.354

7MANOVA

MANOVA uses the same sums of squares as ANOVA.

To understand the relationship between (or among) DVs, MANOVA will use the cross-products.

Cross product

Total cross products

Model cross products

Residual cross products

MANOVATotal cross product (CPT)

We have these information

8MANOVATotal cross product (CPT)

=

()

()

=61.47 5.475.47 141.47

For this case we have

MANOVAModel cross product (CPM)

We have these information

9MANOVAModel cross product (CPM)

The model (or hypothesis) sum of squares and cross-product matrix, H, contains the model sums of squares for each dependent variable and the model cross-product between the two dependent variables

=

()

()

=10.47 7.537.53 19.47

For this case we have

MANOVAResidual cross product (CPR)

10

MANOVAResidual cross product (CPR)

=

R()

R()

=51 1313 122

For this case we have

MANOVA

=10.47 7.537.53 19.47

So we have 3 matrices

=61.47 5.475.47 141.47 =

51 1313 122

Can you find something

suspicious here???61.47 5.475.47 141.47

=10.47 7.537.53 19.47

+51 1313 122T = H + E

11

MANOVA

In univariate ANOVA we calculate the ratio of the systematic variance to the unsystematic variance (i.e. we divide SSM by SSR)

The conceptual equivalent would therefore be to divide the matrix H by the matrix E.

=

???X

=61.47 5.475.47 141.47

51 1313 122

1

=0.2273 0.0820.1930 0.1794

MANOVA

=0.2273 0.0820.1930 0.1794

Find the eigenvectors and the eigenvalues

The eigenvalues are conceptually equivalent to the F-ratio in ANOVA and so the final step is to assess how large these values are compared to what we would expect by chance alone.

From calculation we get 0.335 and 0.073 for eigenvalues

12

MANOVA

=0.2273 0.0820.1930 0.1794

Find the eigenvectors and the eigenvalues

These eigenvalues are conceptually equivalent to the F-ratio in ANOVA and so the final step is to assess how large these values are compared to what we would expect by chance alone.

0.335 00 0.073

These are the EIGENVALUES

MANOVA

The PillaiBartlett trace(also known as Pillais trace) is given by left equation in which represents the eigenvalues for each of the discriminant variates and represents the number of variates. It is similar to the ratio of SSM/SST, which is known as R

2

The HotellingLawley trace (also known as HotellingsT2) is simply the sum of the eigenvalues for each variate.

This test statistic is the sum of SSM/SSR for each of the variatesand so it compares directly to the F-ratio in ANOVA

13

MANOVA

Wilkss lambda is the product of the unexplained variance on each of the variates (the symbol is similar to the summation symbol () that we have encountered already except that it means multiplyrather than add up).

MANOVA

Roys largest root in a sense it is the same as the HotellingLawley trace but for the first variate only, that is:

Statistical significance

14

MANOVA

MANOVA

=0.2273 0.0820.1930 0.1794

Find the eigenvectors and the eigenvalues

From calculation we get this eigenvectors

The eigenvectors show the coefficients which can be used to the create the discriminant function

15

MANOVA

Group homework Create data for a one-way MANOVA with 4 dependent variables and 4 levels in the way.

Create the data in such a way that there is no difference between the rest three mean vectors, but there is a difference between the fourth mean vector and the other three. Prove this with a boxplot, the Wilk's Lambda, Pillai trace, hoteling T2, and Roys largest root with post-hoc ANOVAs (last part is not recommended, but just for illustrative purposes).

16