A New Look at the Evaluation of Sociological Theories in International Large Scale Educational Assessments

Daniel Caro and Andrés Sandoval-Hernandez CIES 2012 Annual ConferenceApril 25, San Juan, Puerto Rico

Rationale

Two important criticisms of LSA studies areStudies lack a solid theoretical basis

This affects the selection, operationalization and explanation of the relations between variablese.g., extended use of SES indicators

Techniques for developing and testing theories (EFA and CFA) have several limitations, for example:

EFA lacks model fit indicesCFA is too restrictive

This study addresses these criticisms

Purpose

Evaluate sociological theories of economic, cultural, and social capital postulated by Bourdieu, Bernstein, and Coleman

Investigate if theoretical models are reflected by the data of international student assessments

Demonstrate advantages of ESEMExplore factor structureEvaluate model fitTest MI across countries

Theoretical Framework

Economic capital (Bourdieu, 1983)

Economic capital is defined as the command individuals have over economic resourcesThe concept is commonly understood as exchange values, like income and assets that can be easily transformed into cash

Cultural capital (Bernstein, 1971; Bourdieu, 1983)

Cultural capital is accounted by the cultural long-lasting dispositions embedded in the human mind and body, as well as in cultural goods and educational credentials Cultural capital can appear in three states: objectified, institutionalized, and embodied

Theoretical Framework

Social capital (Coleman, 1988)

Social capital are all those aspects of the social structure that can be used by actors as resources to achieve their interestsFamily social capital is represented by family relations that enhance the transmission of other structural resources (e.g. dispositions towards culture)

EFA and CFA

In general, EFA for generating theoryCFA for testing theory

CFA limitations (Marsh, 2009, 2010):The zero cross-loading restriction

Statistically, leads to inflated interfactor correlationsAlso theoretically problematic , e.g. number of books reflects both cultural capital and economic capital

Often well-defined EFA structures are not supported by CFA

In practice researchers perform exploratory CFA

Exploratory Structural Equation Modeling (ESEM)

Marsh et al. (2009, 2010) introduced ESEM, an integration of EFA and CFAEssentially, ESEM is EFA with

Standard errorsModel fit indicesMI tests

ESEM is appropriate if model fit is better than in CFATo our knowledge, ESEM has not been yet applied to sociological theories

Data

PIRLS 200642 educational systems 21,000 students (500*42)36 items from parent questionnaire

PISA 200914 countries 7,000 students (500*14)66 items from student and (optional) parent questionnaire

Analytical Strategy

EFA with all itemsSelected items (indexes) for 3 factors

14 in PIRLS 200612 in PISA 2009

CFA and ESEM with selected itemsModel fitInterfactor correlationsCross-loadings

Measurement invariance (MI) tests by country’s HDI

Child

ren'

s bo

ok

Pare

nts'

read

ing

for

enjo

ymen

t

Pare

ntal

atti

tude

s to

read

ing

Pare

nts'

read

ing

at

hom

e

Book

s at

hom

e

Visi

ts to

libr

ary

/ bo

okss

tore

Aca

dem

ic

com

mun

icati

on

Com

mun

icati

on &

In

tera

ction

Cultu

ral

com

mun

icati

on

Lite

racy

sup

port

ac

tiviti

es

Pare

ntal

oc

upati

onal

sta

tus

Child

rens

’ ow

n ro

om

Fina

ncia

l sta

tus

Pare

ntal

edu

catio

n

Cultural Capital

Social Capital

Economic Capital

PIRLS 2006: CFA results

0.73 0.19

0.30

CFI= 0.77TLI= 0.71RMSEA= 0.07

Child

ren'

s bo

ok

Pare

nts'

read

ing

for

enjo

ymen

t

Pare

ntal

atti

tude

s to

read

ing

Pare

nts'

read

ing

at

hom

e

Book

s at

hom

e

Visi

ts to

libr

ary

/

book

shop

Aca

dem

ic

com

mun

icati

on

Com

mun

icati

on &

In

tera

ction

Cultu

ral

com

mun

icati

on

Lite

racy

sup

port

ac

tiviti

es

Pare

ntal

oc

upati

onal

sta

tus

Child

rens

’ ow

n ro

om

Fina

ncia

l sta

tus

Pare

ntal

edu

catio

n

Cultural Capital

Social Capital

Economic Capital

PIRLS 2006: ESEM results

0.73 0.41

0.190.08

0.300.24

CFI= 0.77 0.96TLI= 0.71 0.93RMSEA= 0.07 0.03

PIRLS 2006: CFA and ESEM factor loadings

F1 F2 F3 F1 F2 F3Economical Capital (F1)Parental education 0.64 0.09 0.02 0.80 0.00 0.00Parental occupational status 0.62 0.06 -0.04 0.74 0.00 0.00Financial status 0.29 0.02 0.08 0.35 0.00 0.00Children's own room 0.24 0.01 0.03 0.24 0.00 0.00

Cultural Capital (F2)Books at home 0.65 0.27 -0.02 0.00 0.83 0.00Children's books 0.70 0.16 -0.01 0.00 0.76 0.00Parental attitudes toward reading 0.07 0.62 0.04 0.00 0.53 0.00Frequency parents read for enjoyment -0.08 0.78 0.07 0.00 0.49 0.00Time parents spend reading at home 0.17 0.56 -0.02 0.00 0.56 0.00

Social Capital (F3)Parents literacy support activities 0.27 0.20 0.47 0.00 0.00 0.61Parent-child cultural communication -0.05 0.03 0.85 0.00 0.00 0.80Parent-child communication and interaction 0.22 0.10 0.47 0.00 0.00 0.56Parent-child academic communication -0.14 0.01 0.71 0.00 0.00 0.66Parent-child visit library/bookstore 0.11 0.16 0.27 0.00 0.00 0.36

ESEM ICM-CFA

Hom

e ed

ucati

onal

re

sour

ces

Cultu

ral

poss

essi

ons

Pare

nts'

read

ing

m

otiva

tion

Book

s at

hom

e

Visi

ts to

libr

ary

/ bo

okst

ore

Aca

dem

ic

com

mun

icati

on

Com

mun

icati

on &

In

tera

ction

Cultu

ral

com

mun

icati

on

Lite

racy

sup

port

ac

tiviti

es

Pare

ntal

oc

upati

onal

sta

tus

Hou

seho

ld

over

crow

ding

Pare

ntal

edu

catio

n

Cultural Capital

Social Capital

Economic Capital

PISA 2009: CFA results

0.68 0.28

0.36 CFI= 0.87TLI= 0.83RMSEA= 0.07

Hom

e ed

ucati

onal

re

sour

ces

Cultu

ral

poss

essi

ons

Pare

nts'

read

ing

m

otiva

tion

Book

s at

hom

e

Visi

ts to

libr

ary

/ bo

okst

ore

Aca

dem

ic

com

mun

icati

on

Com

mun

icati

on &

In

tera

ction

Cultu

ral

com

mun

icati

on

Lite

racy

sup

port

ac

tiviti

es

Pare

ntal

oc

upati

onal

sta

tus

Hou

seho

ld

over

crow

ding

Pare

ntal

edu

catio

n

Cultural Capital

Social Capital

Economic Capital

PISA 2009: ESEM results

0.680.38

0.280.17

0.360.15

CFI= 0.87 0.96TLI= 0.83 0.93RMSEA= 0.07 0.05

PISA 2009: CFA and ESEM factor loadings

F1 F2 F3 F1 F2 F3Economical Capital (F1)Parental education 0.80 0.02 0.00 0.74 0.00 0.00Parental occupational status 0.59 0.19 -0.04 0.72 0.00 0.00Household overcrowding -0.33 -0.05 -0.05 -0.36 0.00 0.00

Cultural Capital (F2)Parents reading motivation 0.21 0.20 0.33 0.00 0.44 0.00Cultural Possessions -0.01 0.70 -0.07 0.00 0.57 0.00Books at home 0.23 0.58 0.00 0.00 0.74 0.00Home educational resources 0.09 0.48 0.05 0.00 0.51 0.00

Social Capital (F3)Parents literacy support activities 0.22 0.04 0.54 0.00 0.00 0.61Parent-child cultural communication 0.03 0.14 0.59 0.00 0.00 0.64Parent-child academic communication 0.02 -0.08 0.77 0.00 0.00 0.70Parent-child communication and interaction -0.03 0.06 0.52 0.00 0.00 0.53Parent-child visit library/bookstore -0.04 0.16 0.41 0.00 0.00 0.43

ESEM ICM-CFA

Measurement Invariance Tests

Two groups of countries

PIRLS 2006: low/medium HDI versus high HDIPISA 2009: high HDI versus very high HDI

CFI TLI RMSEA CFI TLI RMSEA

Configural Invariance 0.97 0.95 0.04 0.96 0.92 0.05

Weak Invariance 0.97 0.96 0.04 0.88 0.85 0.07

Strong Invariance 0.97 0.97 0.03 0.89 0.86 0.07

Strict Invariance 0.97 0.96 0.03 0.82 0.81 0.08

PIRLS 2006 PISA 2009Group Invariance

Discussion

Evaluated theories of economic, cultural, and social capital are reasonably reflected by the dataESEM performs better than CFA to capture these theories

Better model fitGreater discriminant validity (lower interfactor correlations)Cross-loadings are statistically and theoretically significant

Educational/cultural possessions and parental education for cultural and economic capitalParent-child cultural interactions for social and cultural capital

Discussion

But theoretical models seem not to hold between countries

SES measures (economic capital) are not comparable

This raises theoretical and methodological questionsTheoretical

Are other theories more appropriate?Global or contextualized theories?

MethodologicalWhat should be the groups of comparison?Can ESEM manage comparisons among a large number of groups/countries?

Future Steps

TheoreticalConsider competing theories, e..g., rational action theory

Reproduction theories for developing countries and rational action theory for developed countries?

MethodologicalMI

Identify problematic variables for MICountrywise comparison matrixEvaluate MI within regions

What to do if no MI?

Questions?

Thank you for your attention!

:)