3nd meeting:

Multilevel modeling: introducing level 1 (individual) and level 2 (contextual) variables + interactions

Subjects for today:

Intra Class Correlation once more

Including individual (level 1) variables

Including contextual (level 2) variables

(cross level) interactions

Last meeting:

Intra Class Correlation: a fraction, it tells us what the relative share of level 2 variance is in the total variance:

ICC = level 2 / level 2 + level 1 variance

ICC is always between 0 (only level 1 variance, no clustering) and 1 (only level 2 variance)

We test with a Chi-square test: difference in -2 loglikelihood between model with both level 1 and level 2 variance and model with level 2 variance set to zero.

2

Multilevel null model: ICC = 24.849 / (24.849 + 81.238) = .23

3

Test whether ICC is significant or whether level 2 variance is significant different from zero we perform a Chi-square test:

-2 loglikelihood from 1 level model - -2 loglikelihood from 2 level model:

3933.064 – 3800.776 = 132 with 1 df. Which is highly significant. This test is superior to Z-test in SPSS because the latter assumes normal distribution.

Note: we test one sided because outcome is always zero or higher

Model with one level only:

4

Testing in MLwin with Chi-square (more info in document about testing, see “statistical testing in Mlwin.pdf”

Type cprob 10 1 and press [Enter]

Note that 0.0015654 must be divided by 2 voor level 2 variance testing

5

So we have taken one step in the whole procedure: we tested whether there is significant level 2 variance. In the example with schools we indeed have a level 2 variance.

The next step is to add level 1 variables to the equation:

Y = b0j + b1 * level 1 variable + eij + uoj

Why?

First, we might have hypotheses about level 1 variables, for instance we assume that white students perform best on a math test compared to other students.

Second, we might expect that level 1 variables can explain level 2 variance.

How is that?

6

Suppose So we have had one step in the whole procedure: we tested whether there is level 2 variance. In the example with schools we indeed have a level 2 variance

The next step is to add level 1 variables to the equation:

Y = b0 + b1 * level 1 variable + eij + uoj

Why?

First, we might have hypotheses about level 1 variables, for instance we assume that white students perform better on a math test.

Second, we might expect that level 1 variables can explain level 2 variance.

How is that?

7

Well, suppose being white indeed increases the chances to have a higher grade at a math test.

Let us further assume that the fraction whites across schools varies.

So assume school 1 has 80% whites and school 2 has 40% whites and we have a positive effect for being white on a math test.

This must result in a lower average grade for school 2 compared to school 1, all other things being equal.

This is called a compositional effect: the mere fact that whites perform better on a math test, school 1 (with 80% whites) will perform better than school 2 (with 40% whites).

This eventually means that part of the difference between the school’s average is a result of the race composition. As a consequence introducing the individual level variable ‘race’ will REDUCE level 2 variance!!8

Including the dummy ‘white’ reduces level 2 variance

18.67 while it was 24.849 in the null model

Level 1 variance is also reduced (was 81.238)

Note that -2 loglikelihood dropped to 3785.204 was 3800.776

Being white increases Grade by 4.7!

9

Testing the effect of being white: 2 options:

One is traditional t-test: b / se = t with df = n – p (where n=sample, p=number of level 1 predictors:

4.717 / 1.163 = 3.946 with 519 – 1 = 518.

Data manipulation Command interface and type: tprob 3.9466 1

Second option: take the -2 loglikelihood difference from model with and without level 1 predictor and use Chi-square test:

3800.776 - 3785.204 = 15.572 with df = 1 (in linear models it is equal to t-test because square root taken from 15.572 = 3.946)

So type cprob 15.572 1

Take half of that because directional hypothesis!

10

Of course one will introduce more level 1 variables. First, to test hypotheses, second to find a (partly) explanation for level 2 variance (we now use the full dataset: LOGSCHOOL.SAV: 415 schools, >8000 pupils.

After we included all relevant level 1 predictors we move on to step number 2: introduce level 2 variables: here the % whites (variable name: race2_mean)

415 schools, t-test: 7.555/ .8 = 9!! With df = n2 – 1 – 1= 413

11

You see there is a difference between a compositional effect from being white and the level 2 effect of %whites. The first effect is just due to individual behavior that taken together influences the school’s average on a math test.

The second effect tells us that individuals perform better when a school has a high percentage of whites. Now let us finally include public versus private school (students in public schools do worse: estimate -3.133).

415 schools, df=415 – 1 - 2

12

Now we have done 3 things: 1) estimate ICC, 2) included level 1 variables, 3) include level 2 variables.

But now…. We might think that the effect of homework varies across schools:

13

Let us get back to the model with no random effect of homework. So effect of homework is constant across schools:

14

There are two extra parameters: 1) the variance of the slope for Homework (not all with the same slope angle, even -!

2) a co-variance between slope and intercept (high on intercept, lower slope)

Effect of homework set random across schools:15

Ok, so we know that the effect of homework depends on what school you are in: in some schools the effect is low, while in others it is high.

WHY is that? One explanation: it is so because in private schools quality of staff is higher so making homework is less important. This would imply that the effect of homework is stronger when school is public:

Effect of homework when school = private equals .914.

When public the effect doubles = .914 + .917 = 1.831!!

16