Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

SEM 2: Structural Equation ModelingWeek 1 - Causal modeling and SEM

Sacha Epskamp

18-04-2017

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Course Overview

• Mondays: Lecture

• Wednesdays: Unstructured practicals

• Three assignments• First two 20% of final grade, last 10% of final grade

• Final project• Presentations and report, 50% of final grade

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

The SEM 2 team

• Sacha Epskamp (sacha.epskamp@gmail.com)

• Gaby Lunansky (G.Lunansky@uva.nl)

• Eiko Fried (eiko.fried@gmail.com)

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Schedule

Week 1 – Introduction to Structural Equation Modeling

• Monday May 8 – Lecture

• Wednesday May 10 – Practical

Week 2 – Causality and equivalent models

• Monday May 15 – Lecture

• Wednesday May 17 - Practical

Week 3 – Latent variable models and network models

• Wednesday May 22 – Lecture

• Wednesday May 24 - Practical

Week 4 – Presentations

• Monday May 29 – Lecture

• Wednesday May 31 - Practical

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Individual AssignmentsEach week, the assignment will be made available 11:00 onWednesday, and will be due 11:00 the next Wednesday. Eachassignment will contribute to 20% or 10% (last week) of yourgrade.• Work on the assignments alone.• Hand in a PDF file and an .R file (in case R was used). If you

use Jasp, hand in the Jasp object as well as a screenshot ofthe options used.

• Make sure your PDF report is as standalone readable aspossible. E.g., if you are asked to report a factor loadingmatrix, then report it in the PDF and not just say “look at .Rfile”.

• Assignments are due before 11:00. If you do not hand in anassignment before 11:00, you will get a 1.

• If you encounter any problems, or have any feedback, pleaselet me know before the deadline, as then I can take it intoaccount or help you.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Final Project

Three options:

1. Perform a SEM analysis on your own data and write a report(individual)

2. Write a manual for semPlot, Onyx, Jasp or Lavaan (individualor with a partner)

3. Research an area or a topic of SEM in more detail and teachfellow students about it

See syllabus on blackboard

• Claim your project using the discussion board on blackboardas soon as possible!

• If you have another idea on a project not listed above, talk tome

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Causal modeling

• This course will introduce structural equation modeling (SEM)

• In SEM, we will discuss modeling complex causal hypotheses

• Again, all variables are assumed normally distributed and allassociations are assumed linear

• Causal hypotheses can be specified between observed andlatent variables

• CFA is a special case of SEM

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Causal models

X Y

X causes Y

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Endogenous and exogenous

• Exogenous (independent) variables are variables of which thecausal origin are not modeled

• Exogenous variables have a variance (sometimes not drawn)• Exogenous variables, except residuals, are allowed to covary

(sometimes not drawn)• Latents: ξ (xi); observed: x (x is also used for indicators of

latent exogenous variables)• Residuals are exogenous

• Endogenous (dependent) variables are variables of which thecausal origin are modeled

• Simply stated: endogenous variables have incoming arrows• Endogenous variables do not have a variance by themselves• Latents: η (eta); observed: y• The causal equation for endogenous variables can be derived

from the path diagram by summing all incoming edges

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

X Y

X is exogenous, Y is endogenous

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

X Y

yi = xi

Causal effect goes from right hand side to left hand side.Experimentally changing x will change y , experimentally changingy will not change x

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

βX Y

yi = βxi

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

βX Y ε

yi = βxi + εi

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Exogenous variables have a variance (often not drawn)

βσx2 θX Y ε

yi = βxi + εi

x ∼ N(µx , σx)

ε ∼ N(0, θ)

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

yi = βxi + εi

x ∼ N(µx , σx)

ε ∼ N(0, θ)

Three observations (variance of x and y and covariance between xand y), three unknowns. Solvable!

Var(x) = σ2x

Var(y) = β2σ2x + θ

Cov(x , y) = βσ2x

But why?

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Covariance Algebra

Let Var(x) indicate “the variance of x” and Cov(x , y) indicate“the covariance between x and y”. The following rules can bederived:

Var(x) = Cov(x , x)

Cov(x , α) = 0

Cov(x , y) = Cov(y , x)

Cov(αx , βy) = αβCov(x , y)

Cov(x + y , z) = Cov(x , z) + Cov(y , z)

Where α and β are constants (parameter) and x , y , and z arerandom variables.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Covariance Algebra

Some consequences:

Cov(αx + βy , z) = Cov(αx , z) + Cov(βy , z)

= αCov(x , z) + βCov(y , z)

Var(x + y) = Var(x) + Var(y) + 2Cov(x , y)

Var(βx) = β2Var(x)

Where α and β are constants (parameter) and x , y , and z arerandom variables.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Matrix Covariance Algebra

Let Var(xxx) indicate “the variance–covariance matrix of vector xxx”and Cov(xxx ,yyy) indicate “the covariance matrix between xxx and yyy”.Then the following rules can be derived:

Var(xxx) = Cov(xxx ,xxx)

Cov(AAAxxx ,BBByyy) = AAACov(xxx ,yyy)BBB>

Var(BBBxxx) = BBBVar(xxx)BBB>

Cov(xxx + yyy ,zzz) = Cov(xxx ,zzz) + Cov(yyy ,zzz)

Where AAA and BBB are constant (parameter) matrices.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

yi = βxi + εi

Var(x) = σ2x

Var(y) = Var(βx + ε)

= Cov(βx + ε, βx + ε)

= Cov(βx , βx + ε) + Cov(ε, βx + ε)

= Cov(βx , βx) + Cov(βx , ε) + Cov(ε, βx) + Cov(ε, ε)

But since x is not correlated with the residuals, Cov(x , ε) = 0 andthus:

Var(y) = β2Cov(x , x) + Cov(ε, ε)

= β2Var(x) + Var(ε)

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

yi = βxi + εi

Cov(x , y) = Cov(x , βxi + εi )

= Cov(x , βxi ) + Cov(x , εi )

= βCov(x , xi )

= βVar(x)

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Path analysis

β1 β2

θ1 θ2

x y1 y2

x is exogenous, and both y1 and y2 are endogenous. θ1 is thevariance of ε1. Causal model for y2:

yi2 = β2yi1 + εi2

yi2 = β2(β1xi + εi1) + εi2

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β1 β2

θ1 θ2

x y1 y2

Number of parameters: 2 regressions +2 residual variances +1exogenous variance (not drawn) = 5, number of observations: 3variances and 3 covariances. 1 degree of freedom!

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β1 β2

θ1 θ2

x y1 y2

Implied covariance between x and y2:

Cov (x , y2) = Cov (x , β2(β1xi + εi1) + εi2)

= Cov (x , β2β1x + β2ε1 + ε2)

= Cov (x , β2β1x) + Cov (x , β2ε1) + Cov (x , ε2)

= β1β2Cov (x , x)

= β1β2σx

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

How many degrees of freedom?

• 7× 8/2 = 28 observed variances and covariances

• 7 regressions +7 variances +1 covariance = 15 parameters

• 13 degrees of freedom

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

How many degrees of freedom?

• 7× 8/2 = 28 observed variances and covariances

• 7 regressions +7 variances +1 covariance = 15 parameters

• 13 degrees of freedom

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Wright’s path tracing RulesThe correlation between any two variables can be expressed as thesum of the compound paths connecting them.To obtain a compound path:

• Trace backwards, change direction at a two-headed arrow,then trace forwards

• Do not go forward and then backward• You can never pass out of one arrow head and into another

arrowhead: heads-tails, or tails-heads, not heads-heads

• Must contain one, and only one, variance or covariance(bidirectional edge)

Then to obtain the implied (co)variance:

• Compute the product of coefficients in each route between thevariables of interest

• Sum over all distinct routes, where pathways are considereddistinct if they contain different coefficients, or encounterthose coefficients in a different order

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between A and B:

A↔ B

Cov (A,B) = h

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between A and B: A↔ B

Cov (A,B) = h

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between A and B: A↔ B

Cov (A,B) = h

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and D:

C ← A↔ A→ D andC ← A↔ B → D

Cov (C ,D) = a(var(A))b + ahc

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and D: C ← A↔ A→ D andC ← A↔ B → D

Cov (C ,D) = a(var(A))b + ahc

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and D: C ← A↔ A→ D andC ← A↔ B → D

Cov (C ,D) = a(var(A))b + ahc

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and itself:

C ←W ↔W → C andC ← A↔ A→ C

Var (C ) = a2(var(A)) + Var(W )

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and itself: C ←W ↔W → C andC ← A↔ A→ C

Var (C ) = a2(var(A)) + Var(W )

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between C and itself: C ←W ↔W → C andC ← A↔ A→ C

Var (C ) = a2(var(A)) + Var(W )

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between F and G :

F ← D ← B ↔ B → E → G ,F ← C ← A↔ B → E → G , and F ← D ← A↔ B → E → G .

Cov (C ,D) = fc(Var(B))dg + fbhdg + eahgd

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between F and G : F ← D ← B ↔ B → E → G ,F ← C ← A↔ B → E → G , and F ← D ← A↔ B → E → G .

Cov (C ,D) = fc(Var(B))dg + fbhdg + eahgd

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Compound paths between F and G : F ← D ← B ↔ B → E → G ,F ← C ← A↔ B → E → G , and F ← D ← A↔ B → E → G .

Cov (C ,D) = fc(Var(B))dg + fbhdg + eahgd

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Structural Equation Modeling

• Structural equation modeling (SEM) extends confirmatoryfactor analysis (CFA) by modeling the variance–covariancematrix of latent variables with a path model

• Allows one to test causal hypotheses on the latent variables

• Includes path analysis for observed variables:• Define one latent per observed variable• Set factor loading to 1• Set residual variance to 0

• In lavaan: use sem() function and define structuralrelationships using the ~ operator (same as used in regressionanalysis)

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Exogenous measurement model (using full LISREL notation, it getseasier!):

xxx i = ΛΛΛxξξξi + δδδi

xxx ∼ N(000,ΣΣΣx)

ξξξ ∼ N(000,ΦΦΦ)

δδδ ∼ N(000,ΘΘΘδ),

Allows you to derive the model-implied variance–covariance matrix:

ΣΣΣx = Var(xxx) = Var(ΛΛΛxξξξ + δδδ)

= ΛΛΛxVar(ξξξ)ΛΛΛ>x + Var(δδδ)

= ΛΛΛxΦΦΦΛΛΛ>x + ΘΘΘδ

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Endogenous model:

yyy i = ΛΛΛyηηηi + εεεi

ηηηi = ΓΓΓξξξi +BBBηηηi + ζζζ i

yyy ∼ N(000,ΣΣΣy )

ξξξ ∼ N(000,ΦΦΦ)

ζζζ ∼ N(000,ΨΨΨ),Note that ΨΨΨ is now diagonal!

εεε ∼ N(000,ΘΘΘε)

Only different from CFA model in the added regression parametersΓΓΓ and BBB. Note that ηηηi appears twice in the structural model, solet’s first solve that:

ηηηi = ΓΓΓξξξi +BBBηηηi + ζζζ i

ηηηi −BBBηηηi = ΓΓΓξξξi + ζζζ i

(III −BBB)ηηηi = ΓΓΓξξξi + ζζζ i

ηηηi = (III −BBB)−1ΓΓΓξξξi + (III −BBB)−1ζζζ i

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Now:

Var(ηηη) = Var((III −BBB)−1ΓΓΓξξξi + (III −BBB)−1ζζζ i

)= Var

((III −BBB)−1ΓΓΓξξξi

)+ Var

((III −BBB)−1ζζζ i

)= (III −BBB)−1ΓΓΓΦΦΦ

((III −BBB)−1ΓΓΓ

)>+ (III −BBB)−1ΨΨΨ(III −BBB)−1>

Which can be used in:

Var(yyy) = ΛΛΛyVar(ηηη)ΛΛΛ>y + ΘΘΘε

And Cov(xxx ,yyy) can similarly be derived. Way too complicated...

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

All-y notation

Much easier, just treat exogenous variables as endogenousvariables. All latents are then contained in ηηη and all indicators inyyy . Only important to note is that ΨΨΨ then contains both exogenousvariances and covariances (all freely estimated) as well as latentresidual variances (usually without covariances).

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

All-y model:

yyy i = ΛΛΛηηηi + εεεi

ηηηi = BBBηηηi + ζζζ i

= (III −BBB)−1ζζζ i

yyy ∼ N(000,ΣΣΣ)

ζζζ ∼ N(000,ΨΨΨ)

εεε ∼ N(000,ΘΘΘ)

Results in:

ΣΣΣ = Var(yyy) = Var (ΛΛΛηηη + εεε)

= Var (ΛΛΛηηη) + Var (εεε)

= ΛΛΛVar (ηηη) ΛΛΛ> + ΘΘΘ

= ΛΛΛVar((III −BBB)−1ζζζ

)ΛΛΛ> + ΘΘΘ

= ΛΛΛ(III −BBB)−1ΨΨΨ(III −BBB)−1>ΛΛΛ> + ΘΘΘ

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

SEM model:

ΣΣΣ = ΛΛΛ(III −BBB)−1ΨΨΨ(III −BBB)−1>ΛΛΛ> + ΘΘΘSimply the CFA model with one extra matrix: BBB encoding

regression parameters. Element βij encodes the effect from variablej to variable i (note, this is opposite of how normally a directednetwork is encoded).The same identification rules as in CFA apply:

• Latent variables must be scaled by setting one factor loadingor (residual) variance to 1

• Model must have at least 0 degrees of freedom

Next week we will discuss equivalent models.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β1 β2

θ1 θ2

x y1 y2

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β21 β32

ψ11 ψ22 ψ33

1 1 1

η1 η2 η3

y1 y2 y3

ΛΛΛ =

1 0 00 1 00 0 1

,ΨΨΨ =

ψ11 0 00 ψ22 00 0 ψ33

,ΘΘΘ =

0 0 00 0 00 0 0

,BBB =

0 0 0β21 0 00 β32 0

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β21 β32

ψ11 ψ22 ψ33

1 1 1

η1 η2 η3

y1 y2 y3

Lavaan automatically adds the latent dummy variables for you!The model is just:

y2 ~ y1

y3 ~ y2

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β21 β32

1 λ21 1 λ42 1 λ63

ψ11 ψ22 ψ33

θ11 θ22 θ33 θ44 θ55 θ66

η1 η2 η3

y1 y2 y3 y4 y5 y6

ΛΛΛ =

1 0 0λ21 0 00 1 00 λ42 00 0 10 0 λ63

,ΨΨΨ =

ψ11 0 00 ψ22 00 0 ψ33

,BBB =

0 0 0β21 0 00 β32 0

ΘΘΘ diagonal as usual.

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

β21 β32

1 λ21 1 λ42 1 λ63

ψ11 ψ22 ψ33

θ11 θ22 θ33 θ44 θ55 θ66

η1 η2 η3

y1 y2 y3 y4 y5 y6

Lavaan model (using sem()):

eta1 =~ y1 + y2

eta2 =~ y3 + y4

eta3 =~ y5 + y6

eta2 ~ eta1

eta3 ~ eta2

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

Structural Equation Modeling (SEM)

• SEM is a more general modeling framework than CFA

• Allows for modeling complex causal hypotheses

• Requires fitting CFA model (as it is nested in CFA model)

• Fitting SEM models using lavaan:• Use sem() instead of cfa()• Specify regressions using the ~ operator

• Same identification rules as in CFA apply

• Models can be assessed and compared in the same way asCFA models

Introduction Causal Modeling Covariance Algebra Path Analysis Structural Equation Modeling Conclusion

This week: start looking for final project topic!