1

The Evolution of SES Gradients in Skills in the School Years: Evidence from the US and the UK

Katherine MagnusonUniversity of Wisconsin-Madison

Jane WaldfogelColumbia University & London School of Economics

Elizabeth WashbrookUniversity of Bristol

Royal Statistical Society, 4th April 2011

We gratefully acknowledge funding from the Russell Sage Foundation’s CRITA project. Waldfogel and Washbrook also gratefully acknowledge funding from

the Sutton Trust. We also thank YouGeon Lee for excellent research assistance with the ECLS-K data.

Introduction

Many studies have shown sizeable SES disparities in skills at school entry in both the US and the UK, but less is known about whether these gaps hold constant, widen or diminish as children move through the school years.

Do schools play an equalizing or disequalizing role? Does this differ • Across the two countries?• Across schooling stages (e.g. the primary/secondary

transition in the UK?)

Does the evolution of disparities in socio-emotional skills mirror that of more commonly studied academic achievement outcomes?

Aims

Aim: Systematically compare the mean differences in outcomes at ages 4 to 14 between different socio-economic groups along a number of dimensions.

How does the gradient differ:

By outcomes: academic achievement (reading and maths) versus socio-emotional behaviour (internalizing and externalizing symptoms)

By measure of SES: parental education or family income

By the outcome metric: absolute values versus standardized scores

In size at a given age and in changes over time

In the US versus the UK

Repeated cross-sections are often used in this context for practical data reasons. But studies of this type have a number of drawbacks.

Age 3 (MCS) Age 5 (MCS) Age 7 (ALSPAC) Age 11 (ALSPAC/

LSYPE)

Age 14 (LSYPE) Age 16 (LSYPE)20

30

40

50

60

70

80

Highest Quintile 4 Quintile 3 Quintile 2Lowest

Perc

entil

e of

the

test

scor

e di

strib

ution

Figure 1.2. in Alissa Goodman and Paul Gregg (eds) Poorer children educational attainment: How important are attitudes and behaviour? Joseph Rowntree Foundation, March 2010.

Academic outcomes by SEP quintile from three cohort studies

Methodological issues

Missing data

Non-random attrition in the longitudinal sample may lead to biases when making statements about population averages (and differently for the two countries).

Non-response affects not only who is observed in the sample, but also

The estimated outcome variance used to norm the scores at each age

The boundaries used to define quantiles of income or an SES index

Methodological issues

Raw vs standardized scores Why standardize? To allow comparison of outcomes measured in different metrics z-scores give estimates of effect sizes (intuitive sense of magnitudes)

But what do we really care about?Typically children’s academic skills show increasing variance over the course of development, which is removed by standardization. A standard deviation difference at 16 may equate to much larger disparities in the skills that matter for future success than a standard deviation gap at 7. The two methods may lead to different characterizations of whether gaps are widening or narrowing with age.

Datasets and assessmentsUK US

Dataset Avon Longitudinal Study of Parents and Children (ALSPAC)

Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K)

YOB 1991/1992 1993

Scope Census of children born to pregnant women resident in Avon with EDD between 4/91 & 12/92

Nationally representative sample of children enrolled in kindergarten fall 1998.

SES measures

Highest qualification of parent in pregnancyQuintile of average net income at 2.8 and 3.9 years*

Highest qualification of parent at Fall K (5.7 years)Quintile of gross income at Fall K

Math and Reading

National Curriculum Levels from Key Stage assessments. Plus Entry Assessments at age 4. Assessed at a single point in time for 3 cohorts.

Bespoke ECLS-K assessments using IRT methods. Assessed at a single point in time for a single cohort.

Behaviour Parent reports (SDQ). Externalizing (20-pt scale; Hyperactivity + Conduct Problems) and Emotional Symptoms (10-pt scale). Age-specific assessments.

Teacher reports of externalizing (5-pt scale) and internalizing (4-pt scale). Assessed at the same time as academic outcomes.

Assessment agesUK USAge in

yrsMath/

ReadingBeh-avior

Age in yrs

Math/ Reading

Beh-avior

Entry Assessment 4.5 √*Kindergarten Fall 5.7 √ √Kindergarten Spring 6.2 √ √

SDQ6 6.8 √Key Stage 1 7.2 √ Spring 1st grade 7.2 √ √SDQ8 8.2 √SDQ9 9.6 √ Spring 3rd grade 9.2 √ √Key Stage 2 11.2 √ Spring 5th grade 11.2 √ √SDQ11 11.7 √SDQ13 13.2 √Key Stage 3 14.1 √ Spring 8th grade 14.2 √

Non-response in ALSPACThe mechanisms leading to missing data differ strongly for academic achievement and behavioural outcomes.

Core ALSPAC cohort = 13,988 children alive at 1 year.

Key Stage outcomes are available for all children in English state schools, even if they left the study. Just 7% were never observed in state school from 7 to 14; but only 66% were always in a state school. Requiring EA reduces this further to 53%.

SDQ outcomes require parental completion of a postal questionnaire at 5 dates. 28% have complete records, 68% have at least 2 out of 5. Non-response is non-monotonic.

Parental education comes from a single questionnaire in pregnancy; income from at least one of two questionnaires at 33 and 47 months.

Multiple imputation

The ice command in Stata was used to impute values for the full 13,988 cohort five times. Standard errors of all estimates are adjusted to the process.

Additional variables used in the imputation:

Cohort year, month of birth, gender, mother’s age and its square, birth weight, non-white dummy (98%+ observed)

Parental education, FSM and SEN status at age 11 (85%+)

Family structure, family income at 33 & 47m, nursery attendance, family income at 85m, 97m & 11y

Additional SDQ sub-scores reported by parents at 47 months and by teachers in Years 3 and 6.

Math imputation results, by education

Parental ed Math outcomes – Mean (SD)No quals Degree EA (4y) KS1 (7y) KS2 (11y) KS3 (14y)

Fully observed 11.2% 13.2% 5.30 2.14 4.06 5.816680 (47.8%) (1.07) (0.64) (0.81) (1.32)Total sample 10.4% 20.6% 5.28 2.13 4.05 5.8013988 (100%) (1.11) (0.68) (0.85) (1.41)

Known ed, missing outcome 7.7% 31.2% 5.38 2.18 4.14 5.985863 (41.9%) (1.12) (0.69) (0.85) (1.44)Known outcome, missing ed 17.8% 10.5% 4.81 1.88 3.67 5.04740 (5.3%) (1.07) (0.71) (0.89) (1.40)Missing outcome & ed 18.5% 13.0% 4.80 1.80 3.63 5.01705 (5.0%) (1.15) (0.75) (0.93) (1.53)

Numbers in red calculated wholly or partly from imputed values

Externalizing behaviour imputation results, by education

Parental ed Behaviour outcomes – Mean (SD)No quals Degree 6y 8y 9y 11y 13y

Fully observed 3.6% 31.7% 4.52 4.39 3.72 3.53 3.783821 (27.3%) (3.12) (3.23) (2.92) (2.93) (3.01)Total sample 10.4% 20.6% 5.22 5.11 4.49 4.32 4.5613988 (100%) (3.30) (3.42) (3.17) (3.19) (3.23)

Known ed, missing outcome 12.2% 17.2% 5.42 5.31 4.70 4.54 4.778722 (62.4%) (3.34) (3.46) (3.21) (3.24) (3.25)Known outcome, missing ed 14.6% 14.6% 4.49 4.93 3.66 4.00 3.6641 (0.3%) (3.37) (3.57) (2.68) (2.53) (2.35)Missing outcome & ed 18.2% 11.6% 5.94 5.83 5.27 5.17 5.411404 (10.0%) (3.27) (3.38) (3.20) (3.21) (3.23)

Numbers in red calculated wholly or partly from imputed values

Income quintile boundaries, by sample

Numbers in red calculated wholly or partly from imputed values

Math outcomeExternal behaviour

outcome20 p’ile 80 p’ile 20 p’ile 80 p’ile

Fully observed 148 424 246 519

Total sample (13,988) 148 437 148 437

Known income, missing outcome 195 519 148 432

Known outcome, missing income 118 351 203 519

Missing outcome & income 118 432 118 387

EA (4y) KS1 (7y) KS2 (11y) KS3 (14y)0

0.5

1

1.5

2

2.5

Parental education gradients in raw maths scores (degree vs no quals gaps)

UnimputedImputed

Poin

t gap

in a

vera

ge sc

ores

EA (4y)

KS1 (7y)

KS2 (11y)

KS3 (14y)

0

0.5

1

1.5

2

2.5

Parental income gradients in raw maths scores (top vs bottom income quintiles)

UnimputedImputed

Poin

t gap

in a

vera

ge sc

ores

EA (4y) KS1 (7y) KS2 (11y) KS3 (14y)0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Parental education gradients in standardized maths scores (degree vs no quals gaps)

UnimputedImputed

Gap

in st

anda

rdize

d sc

ores

EA (4y)

KS1 (7y)

KS2 (11y)

KS3 (14y)

00.20.40.60.8

11.21.41.6

Parental income gradients in standardized maths scores (top vs bottom income quintiles)

UnimputedImputed

Gap

in st

anda

rdize

d sc

ores

The effects of imputation on estimated SES gradients in Maths

Black lines are 95% CIs

The effects of imputation on estimated SES gradients in Externalizing behaviour

Black lines are 95% CIs

6y 8y 9y 11y 13y

-2-1.8-1.6-1.4-1.2

-1-0.8-0.6-0.4-0.2

0

Parental education gradients in raw externalizing behaviour scores (degree vs no quals gaps)

UnimputedImputed

Poin

t gap

in a

vera

ge sc

ores

6y 8y 9y 11y 13y

-2-1.8-1.6-1.4-1.2

-1-0.8-0.6-0.4-0.2

0

Parental income gradients in raw externalizing behaviour scores (top vs bottom income quintiles)

UnimputedImputed

Poin

t gap

in a

vera

ge sc

ores

6y 8y 9y 11y 13y

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

Parental education gradients in standardized ex-ternalizing behaviour scores (degree vs no quals

gaps)

UnimputedImputed

Gap

in st

anda

rdize

d sc

ores

6y 8y 9y 11y 13y

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

Parental income gradients in standardized externaliz-ing behaviour scores (top vs bottom income quintiles)

UnimputedImputed

Gap

in st

anda

rdize

d sc

ores

UK Raw Achievement Scores

UK Standardized Achievement Scores

US Raw Achievement Scores

US Standardized Achievement Scores

Summary of standardized achievement gaps

Math ReadingAge Education gap Income gap Education gap Income gap

US UK US UK US UK US UK

4.5 1.03 0.80 1.01 0.74

5.7 1.19 1.06 1.20 1.04

6.2 1.11 0.97 1.08 0.91

7.2 0.99 1.02 0.91 0.74 1.08 1.12 0.91 0.82

9.2 1.12 1.03 1.22 1.06

11.2 1.14 1.18 1.05 0.84 1.21 1.18 1.05 0.83

14.2 1.20 1.42 1.06 1.02 1.28 1.30 1.12 0.95

High – Low gaps in mean z-scores (ISCED 5A/6 vs ISCED 2; top vs bottom income quintile groups)

UK Raw Behavior Scores

UK Standardized Behavior Scores

US Raw Behavior Scores

US Standardized Behavior Scores

Summary of standardized behavior gaps

Externalizing InternalizingAge Education gap Income gap Education gap Income gap

US UK US UK US UK US UK

5.7 -0.26 -0.18 -0.30 -0.14

6.2 -0.32 -0.25 -0.34 -0.17

7.2/6.8 -0.42 -0.44 -0.28 -0.38 -0.41 -0.09 -0.19 -0.25

8.2 -0.39 -0.38 -0.07 -0.21

9.2/9.6 -0.53 -0.48 -0.33 -0.43 -0.45 -0.15 -0.24 -0.22

11.2/11.7 -0.48 -0.42 -0.28 -0.42 -0.32 -0.23 -0.19 -0.24

13.2 -0.48 -0.46 -0.21 -0.24

High – Low gaps in mean z-scores (ISCED 5A/6 vs ISCED 2; top vs bottom income quintile groups)

Conclusions

We find evidence of widening academic achievement gaps in the UK between 7 and 14, with greater widening after age 11.

This holds for maths and reading, for income and education, and for raw and standardized scores.

We hypothesize this is related to greater sorting at the secondary than the primary level.

The US results are more sensitive to the choice of raw or standardized scores. Raw scores reveal constant gaps in the first years of schooling, with widening thereafter. Standardized scores reveal gaps that narrow in the first years of schooling, then return to their original values by age 14.

Either way, the rapid growth in inequality of outcomes in the UK after age 11 does not appear to hold in the US in the same way.

Conclusions

Cross-national comparisons on gradients in behaviour must be tentative due to measurement differences across the surveys.

However, the results suggest that social gradients are much smaller in socio-emotional outcomes than in academic achievement.

We find little evidence of systematic widening or narrowing of behaviour gradients during the school years in either country.

Careful thought needs to be given to the methodological details of constructing even the simplest descriptive statistics in work that compares outcomes over time and across countries!