Latent Structural Equation Modelinglipas.uwasa.fi/~sjp/Teaching/sem/lectures/semc4.pdfLatent Structural Equation Modeling Seppo Pynn onen Department of Mathematics and Statistics,

Latent Structural Equation Modeling

Seppo Pynnonen

Department of Mathematics and Statistics, University of Vaasa, Finland

As of March 28, 2016Seppo Pynnonen Latent Structural Equation Modeling

SEM Applications

Contents

1 SEM Applications

MIMIC models

Multi Group Analysis

Comparing Factor Models Between Groups

Analysis of Latent Mean Structures

Seppo Pynnonen Latent Structural Equation Modeling

SEM Applications

MIMIC models

Contents

1 SEM Applications

MIMIC models





SEM Applications

MIMIC models

Multiple Indicators Multiple Causes (MIMIC) model involves usinglatent variables that are predicted by observed variables.

Example 1

Social status and social participation (Hodge and Traiman 1968,American Sociological Review 33. 723–740).

In the cited relationship between social status and social participation ina sample of 530 women was studies.

It was hypothesized that income, occupation, and eduction explain

social participation. Social participation (SocalP) was measured by

church attendance (church), memberships (member), and friends

seen (friends).


SEM Applications

MIMIC models

Example 1: Social Status and Social Participation

The following model was specified:


SEM Applications

MIMIC models


Mathematical representation: y1

y2

y3

=

λ1

λ2

λ3

η +

ε1

ε2

ε3

(1)

η = γ1x1 + γ2x2 + γ3x3 + ζ, (2)

where y1 = church, y2 = member, and y3 = friends are indicators ofthe latent variable η (= social p) and x1 = income, x2 = occup, andx3 = educ are causes of η.

In particular equation (1) says that y ’s are congeneric. I.e., the true

values τi = λiη (= yi − εi ), i = 1, 2, 3 are linear combinations of each

other and hence have pair-wise correlations equal to unity.


SEM Applications

MIMIC models


* Example 4.1: Social Status and Social Participation;

* (Hodge and Treiman 1968);

data hodge(type = corr);

infile cards missover;

input _type_ $ _name_ $ income occup educ church member friends;

datalines;

corr income 1.000

corr occup .304 1.000

corr educ .305 .344 1.000

corr church .100 .156 .158 1.000

corr member .284 .192 .324 .360 1.000

corr friends .176 .136 .226 .210 .265 1.000

n . 530

;

run;

proc calis data = hodge;

path

Socialp <-- income occup educ,

Socialp --> church member friends,

/* Identification: Scale variance of the latent variable to unity */

<--> Socialp = 1; /* Latent variable variance scaled to 1 */

pathdiagram exogov

title = "Scocial Status and Participation";

run;


SEM Applications

MIMIC models


Estimation results:


SEM Applications

MIMIC models


========================================================

Standard

Estimate Error t Value Pr > |t|

--------------------------------------------------------

Socialp <=== income 0.26947 0.06613 4.0752 <.0001

Socialp <=== occup 0.11291 0.06457 1.7485 0.0804

Socialp <=== educ 0.38721 0.07000 5.5317 <.0001

Socialp ===> churc 0.40118 0.04613 8.6972 <.0001

Socialp ===> member 0.63347 0.06000 10.5576 <.0001

Socialp ===> friends 0.34597 0.04580 7.5533 <.0001

Socialp <==> Socialp 1.00000

========================================================

========================================================

Squared Multiple Correlations

Variable Error Variance Total Variance R-Square

--------------------------------------------------------

church 0.78313 1.00000 0.2169

friends 0.83871 1.00000 0.1613

member 0.45926 1.00000 0.5407

Socialp 1.00000 1.34752 0.2579

========================================================


SEM Applications

MIMIC models


Although the overall fit is not too bad (χ2 = 12.6 with df = 6), it seemthat this model is not very successful because most of the relationshipsare rather poorly determined as shown by the low R-squares.

Furthermore the low t-value of occup reveals that occupation may not

be a significant determinant of social participation although income and

education are.

Remark 1

SEM analysis should be based on covariance matrix. Using correlation matrix may biasstandard errors. For more details, see e.g. Cudeck, Robert, 1989, Analysis ofcorrelation matrices using covariance structure models, Psychological Bulletin 105,317–327. (http://psycnet.apa.org/journals/bul/105/2/317/).

See also discussion at semnet faq: http://www2.gsu.edu/∼mkteer/covcorr.html


SEM Applications


Contents

1 SEM Applications

MIMIC models





SEM Applications


Example 2

Nine psychological variables: Comparison of Measurement Modelsbetween Boys and Girls.

vicperc: Visual perception scorescubes: Test of spatial visualizationlozenges: Test of spatial orientation

paragraph: Paragraph comprehension scoresentence: Sentence completion scorewordmean: Word meaning score

addition: Add test scorecountdot: Counting groups of dots score

sccaps: Straight and curved capitals test score


SEM Applications


Contents

1 SEM Applications

MIMIC models





SEM Applications


Sometimes it may be of interest to test whether the factor modelsare similar between different groups.

For example are the indicators measuring same underlying factorsin different groups.

The comparison can be done on different levels.


SEM Applications


Letx(g) = Λ(g)ξ(g) + δ(g) (3)

denote the factor model for group g , g = 1, 2 (we assume for thesake of simplicity that we have only two groups).

The covariance matrices factor accordingly

Σ(g) = Λ(g)Φ(g)Λ(g)′ + Θ(g)δ . (4)


SEM Applications


If there are no constraints across the groups, each group can beanalyzed separately.

If there are constraints across groups, the data must be analyzedsimultaneously.

The analysis can be proceeded in a hierarchical manner:

Test first where the covariance matrices are equal:

HΣ : Σ(1) = Σ(2). (5)

If HΣ is accepted, then there is no need to proceed, because itimplies that the factor model are identical in the groups.


SEM Applications


If HΣ is rejected, we can proceed in a hierarchical manner.

First we test whether the number of factors, q, is the same inboth groups.

Next we test whether the factor patters are of the same formacross groups (i.e., equally many factors and the diagram is ofthe same form: Λ(g) have zeros (or fixed elements) in thesame positions).

If the above hypotheses are accepter, we can imposeadditional restrictions on individual parameters or matrices,for example, that all factor structure coefficients are the same,i.e.,

HΛ : Λ(1) = Λ(2). (6)


SEM Applications


Example 2: Data preparations

Coping SAS code fromhttp://www.psych.yorku.ca/friendly/lab/files/psy6140/data/psych24r.sas we get theinitial data set.2

Next we restrict ourselves to grant school students and make separate data sets forboys and girls

data hb(keep = visperc--sccap); /* boys, grant school*/

set holzinger;

visperc = v1; /* rename for more descriptive variables */

cubes = v2; lozenges = v4; parcomp = v6; sencomp = v7;

wordmean = v9; addition = v10; countdot = v12; sccap = v13;

if sex = "M" and grp = 1 then output; /* output only grant school boys */

run;

data hg(keep = visperc--sccap); /* girls, grant school*/

set holzinger;

visperc = v1; cubes = v2; lozenges = v4; parcomp = v6; sencomp = v7;

wordmean = v9; addition = v10; countdot = v12; sccap = v13;

if sex = "F" and grp = 1 then output;

run;

2Other SAS data sets: http://www.psych.yorku.ca/lab/psy6140/ex/data.htm


SEM Applications


Example 2: Test equality of covariance matrices.

There are 72 boys and 73 girls in the sample.

First we test (5)HΣ : Σ(1) = Σ(2)

Testing for equality of the covariance matrices can be easiest done forexample by SAS CALIS procedure.

proc calis covpattern = eqcovmat;

var visperc cubes lozenges parcomp sencomp

wordmean addition countdot sccap; /* variables used from both data sets */

group 1 / label = "Girls" data = hg; /* girls data set */

group 2 / label = "Boys" data = hb; /* boys data set */

fitindex noindextype on(only) = [chisq df probchi aic sbc]; /* subset of statistics to output */

run;


SEM Applications


Example 2: Equality of covariance matrices.

=========================================

Fit Summary

-----------------------------------------

Chi-Square 58.1922

Chi-Square DF 45

Pr > Chi-Square 0.0897

RMSEA Estimate 0.0640

Akaike Information Criterion 148.1922

Schwarz Bayesian Criterion 282.1453

=========================================

The test is a chi-square test.

The value χ2 = 58.2 with df = 45 and p-value .0897 indicates that thereis no strong empirical evidence that the covariance matrices would bedifferent.

Thus, we can presume that there should not appear big differences in the

factor model.


SEM Applications


Example 2: Test whether there are 3 factors

In testing for the number of factors, we specify exploratory factor models(EFA) in both groups.

To make the models identified the program imposes (default)identification constraints (details given in the classroom).

In both groups we have 9 observed variables and 3 factors thatimply 21 freely estimated factor loadings and 3 freely estimatedfactor variances in each group.

In addition there will be 9 error variances in each group.

So that the total number of estimated parameters is2 × (21 + 3 + 9) = 66.

As there are 2 × [(9 + 1) × 9/2] = 90 variances and covariances, we have90 − 66 = 24 degrees of freedom in the χ2 testing for the hypothesis

Hq : q = 3 (7)


SEM Applications


Example 2: Test whether there are 3 factors

/* test whether there are 3 common factors */

proc calis;


wordmean addition countdot sccap;

group 1 / label = "Girls" data = hg;

group 2 / label = "Boys" data = hb;

model 1 / group = 1;

factor n = 3; /* 3 exploratory factor model for girls */


factor n = 3; /* 3 factors for boys */

fitindex noindextype on(only) = [chisq df probchi rmsea aic sbc];

run;

==============================================

Fit Summary

----------------------------------------------

Chi-Square 23.2751

Chi-Square DF 24





==============================================


SEM Applications


Example 2: Number of factors

The test results with χ2 = 23.3 with df = 24 and p-value of .503indicates that the null hypothesis of 3 common factors in (7) cannot berejected.

Thus, in both groups a three factor model seems to explain the

covariances between the measures.


SEM Applications


Example 2: Factor structure

Next we test whether both groups share the following similar 3-factorpattern (configural invariance):


SEM Applications


Example 2: Similarity of Factor Structures (Configural Invariance)

Estimate the factor structures and produce modification indexes.

/* test similarity of factor patterns */

proc calis mod; /* produce also modification indicies */




path

Visual --> visperc cubes lozenges = 1,

Verbal --> parcomp sencomp wordmean = 1,

Speed --> addition countdot sccap = 1;

pathdiagram exogcov

model = [1] /* display group 1 */

fitindex = [chisq df probchi rmsea cfi aic sbc nobs]

title = "Girls";


path

Visual --> visperc cubes lozenges = 1,




pathdiagram exogcov

model = [2]


title = "Boys";

run;

run;


SEM Applications


Example 2: Factor structure (girls)


SEM Applications


Example 2: Modification indices (girls)

=========================================================

Rank Order of the 10 Largest LM Stat for Path Relations

---------------------------------------------------------

Parm

To From LM Stat Pr > ChiSq Change

---------------------------------------------------------

sccap visperc 12.55178 0.0004 2.38677

sccap Visual 11.15010 0.0008 5.06390

Visual sccap 10.89057 0.0010 0.14834

Speed sccap 7.27894 0.0070 -0.56212

countdot addition 7.27266 0.0070 0.27844

addition countdot 7.27152 0.0070 1.02967

visperc sccap 6.06943 0.0138 0.05992

addition sccap 5.08355 0.0242 -0.46932

sccap addition 5.08164 0.0242 -0.58640

Speed countdot 5.08147 0.0242 0.85994

=========================================================


SEM Applications


Example 2: Factor structures (boys)


SEM Applications


Example 2: Modification indices (boys)

========================================================

Rank Order of the 10 Largest LM Stat for Path Relations

--------------------------------------------------------

Parm

To From LM Stat Pr > ChiSq Change

--------------------------------------------------------

sccap Visual 12.30018 0.0005 3.80710

sccap Verbal 10.01694 0.0016 4.52170

sccap parcomp 9.09985 0.0026 3.46155

Speed addition 8.43438 0.0037 1.27872

countdot sccap 8.42548 0.0037 -0.64102

sccap countdot 8.41718 0.0037 -4.98659

addition visperc 8.26307 0.0040 -0.97515

Visual addition 8.15343 0.0043 -0.11172

countdot addition 7.55653 0.0060 2.26536

Speed sccap 7.55094 0.0060 -0.63300

========================================================

The goodness-of-fit is not satisfactory.According to the modification indices the fit would improve materially if sscapis allowed to be load on the Visual factor as well (mi predicts degrease in χ2 by12.3 for 1 degree of freedom).Also the girls’ data support this (mi predicts 11.1 decrease in χ2 for 1 degree offreedom).Thus, freeing that parameter in both group should decrease χ2 by 23.4 in tradeof only 2 degrees of freedom (”pretty good deal”).


SEM Applications


Example 2: Modified model

proc calis mod; /* produce also modification indicies */




path

Visual --> visperc cubes lozenges sccap = 1, /* sccap added */



pathdiagram exogcov

model = [1] /* display group 1 */


title = "Girls";


path

Visual --> visperc cubes lozenges sccap = 1, /* sccap added */




pathdiagram exogcov

model = [2]


title = "Boys";

run;


SEM Applications


Example 2: Modified model for girls


SEM Applications


Example 2: Modified model for boys


SEM Applications


Example 2: Configural invariant model for boys and girls

The goodness of fit improves in terms of all fit indexes and showgood fit.

Also AIC and SBC improve.

AIC and SBC improve also form the unrestricted 3-factor model,and the chi-square difference of ∆χ2 = 49.13 − 23.27 = 25.86 with∆df = 46 − 24 = 22 (p-val = .258) shows that the restrictions onthe factor patterns are acceptable.

We can proceed with this specification.

Accordingly, boys and girls seem to share similar factor patterns, i.e., theindicators seem to measure the same underlying factors in both groups.

Next we can test whether the hypothesis in (6), i.e., HΛ : Λ(1) = Λ(2)

holds (Weak Factorial Invariance).


SEM Applications


Remark 2

It is important that in a multi-sample analysis with constraints over

groups, one must not standardize the variables within groups (neither the

observed nor the latent variables).


SEM Applications


Example 2: Equality of factor patterns, HΛ : Λ(1) = Λ(2) (Weak factorial invariance)

proc calis;




path

Visual --> visperc cubes lozenges sccap = 1 cVi lVi sVi, /* coefficient labels */

Verbal --> parcomp sencomp wordmean = 1 sVe wVe,

Speed --> addition countdot sccap = 1 cSp sSp;

pathdiagram exogcov

model = [1]


title = "Girls";


path /* same coefficient labels across groups impose the desired restrictions*/

Visual --> visperc cubes lozenges sccap = 1 vVi lVi sVi,


Speed --> addition countdot sccap = 1 cSp sSp;


pathdiagram exogcov

model = [2]


title = "Boys";

run;


SEM Applications


Example 2: Equality of factor patterns (Girls)


SEM Applications


Example 2: Equality of factor patterns (Boys)


SEM Applications


Example 2: Equality of factor patterns, HΛ : Λ(1) = Λ(2)

The chi-square difference is 55.43 − 49.13 = 6.30 and the degrees of

freedom difference is 53 − 46 = 7 implying p-value 0.505 and we can

accept the additional restrictions.

Next we can proceed to test whether in addition the groups share thesame factor covariances structure (Strong Factorial Invariance), in whichthe null hypothesis is

HΛΦ : Φ(1) = Φ(2). (8)


SEM Applications


Example 2, Strong factorial invariance; HΛΦ : Φ(1) = Φ(2)

ML-estimation runs into troubles with this restriction by producingnegative error variance for sccap.

Using robust covariance matrix obtained by statement PROC CALIS

robust = twostage; in ML estimation solves the problem.

Remark 3

Robust methods can be only used when full sample data are available.


SEM Applications



************** Strong Factorial Invariance **************************;

proc calis method=ml robust = twostate; *residual plots=caseresid; *ml robust=sat mod;






path

Visual --> visperc cubes lozenges sccap = 1 cVi lVi sVi,


Speed --> addition countdot sccap = 1 cSp sSp,

/* Factor covariances */

<--> [Visual Verbal Speed] = c11 c21 c31 c22 c32 c33;

pathdiagram exogcov nomeans

model = [1] fitindex = [chisq df probchi rmsea cfi aic sbc nobs] title = "Girls";


path






/* factor covariances */


* nlincon 0. <= v8;


model = [2] fitindex = [chisq df probchi rmsea cfi aic sbc nobs] title = "Boys";

run;


SEM Applications



=========================================

Chi-Square 64.81

Chi-Square DF 59





=========================================

∆χ2 = 64.81 − 55.43 = 9.38, df = 6, p = .153, thus strong factorialhypothesis is not rejected.

Finally, we can proceed to test whether in addition the error variances arethe same in both groups (Strict Factorial Invariance): Given Λ(1) = Λ(2)

and Φ(1) = Φ(2),HΛΦΘ : Θ(1) = Θ(2). (9)


SEM Applications


Example 2, Strict invariance; HΛΦΘ : Θ(1) = Θ(2)







path





<--> [Visual Verbal Speed] = c11 c21 c31 c22 c32 c33,

/* error variances */

<--> visperc cubes lozenges parcomp sencomp wordmean addition countdot sccap = v1-v9;

pathdiagram exogcov



path





<--> [Visual Verbal Speed] = c11 c21 c31 c22 c32 c33,

/* error variances */

<--> visperc cubes lozenges parcomp sencomp wordmean addition countdot sccap = v1-v9;


pathdiagram exogcov


run;


SEM Applications


Example 2, Strict factorial invariance; HΛΦΘ : Θ(1) = Θ(2)

Results:

=========================================

Chi-Square 84.06

Chi-Square DF 68





=========================================

The difference of Chi-square is 19.25 with 9 degrees of freedom and

p-value 0.023. Thus we reject these restrictions at the 5% level.


SEM Applications


Example 2, Summary

Summary of the results are in the table below:

Hypothesis χ2 df ∆χ2 ∆df p-val Decision

A Hq 23.3 24 na na 0.503 AcceptedB Hpattern 49.1 46 25.9 22 0.258 AcceptedC HΛ 52.1 53 6.3 7 0.505 AcceptedD HΛΦ 64.8 59 9.4 6 0.153 AcceptedE HΛΦΘ 84.3 68 19.3 9 0.023 Rejected

Thus far the conclusion is that otherwise the structures are invariant (the same forboys and girls) but at some error variances are different (that is, reliability of themeasures differ for boys and girls).

We leave fixing individual error variances across groups as an exercise to find out

measurements in particular differ in that respect.


SEM Applications


Contents

1 SEM Applications

MIMIC models





SEM Applications


Mean structures

So far it has been assumed that the random variables (and inparticular the latent variables) have zero means.

In single sample the latent variable means are unidentified (thusset to zero for convenience).

In multi-samples the individual means of the latent variables areagain unidentified, but the group differences can be estimated(provided that the factors have the same scales across groups).


SEM Applications


Mean structures

The idea is that the group differences in observed variables can beexplain by the differences in the latent factor means.

This implies that one must assume that the factor structures (thefactor regression coefficients) are invariant across groups.

The differences in the means can be measured by fixing the meanin one group equal to zero.


SEM Applications


Example 2, Mean structures

/********* Analysis of Mean Structures ************/







path






/* means */

mean

Visual Verbal Speed = 0 0 0, /** set factor means in group 1 to zero */

visperc cubes lozenges parcomp sencomp wordmean addition countdot sccap = m1-m9;

/* equality accross groups */



/******** continues ************/


SEM Applications



/**** continues ***********/


path





<--> [Visual Verbal Speed] = c11 c21 c31 c22 c32 c33; /* equality accross groups */

/* means in group 2 */

mean

Visual Verbal Speed = mVi mVe mSp, /** in group 2 factor means are freely estimated */

visperc cubes lozenges parcomp sencomp wordmean addition countdot sccap = m1-m9;




run;


SEM Applications



Model 2. Means and Intercepts

==================================================================

Standard

Type Variable Parameter Estimate Error t Value Pr > |t|

-----------------------------------------------------------------

Mean Visual mVi 0.70175 0.97379 0.7206 0.4711

Verbal mVe -0.89456 0.51079 -1.7513 0.0799

Speed mSp 1.45170 2.99023 0.4855 0.6273

==================================================================

==================================================================

Fit Summary

------------------------------------------------------------------

Chi-Square 76.6710

Chi-Square DF 65





==================================================================

None of the mean differences are statistically significant.

Thus, there is no empirical evidence for latent variable mean differences.


Documents

Latent Structural Equation Modelinglipas.uwasa.fi/~sjp/Teaching/sem/lectures/semc4.pdfLatent Structural Equation Modeling Seppo Pynn onen Department of Mathematics and Statistics,