Fadeout in human capital interventions: Death, miracles and resurrection

Fadeout in human capital interventions: Death,

miracles and resurrection

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Greg J. DuncanDrew BaileyWinnie Yu

School of Education

University of California, Irvine

IQEarnings

Fadeout in human capital interventions: Death, miracles and resurrection

ImpactDeath

IQ impacts in Perry

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

-0.25

0

0.25

0.5

0.75

1

AgeSolid marker denotes p<.05

End of program

Achievement impacts for Head Start 3 year olds

3 4 K 1st 3rd-0.10

0.00

0.10

0.20

0.30

0.40

Eff

ect

siz

e in s

d u

nit

s

Solid marker denotes p<.05

Letter-WordMath

End of program

Cognitive impacts in 67 ECE studies

0 0-1 1-2 2-4 4+0.00

0.10

0.20

0.30

0.40

0.23

0.10 0.090.05 0.06

Eff

ect

siz

e in s

d u

nit

s


End of pro-gram

0 5 11 17 23 29 35 41 47

-10.0

0.0

10.0

20.0

30.0

Months From Random Assignment

Perc

enta

ge E

mplo

yed F

ull-

Tim

e

Program pe-riod

Employment impacts for the 36-month Canadian Self-Sufficiency Program

ImpactPersistence

IQ impacts in Perry and Abecedarian

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

-0.25

0

0.25

0.5

0.75

1

Age

Perry

Abecedarian


IQ and Earnings impacts in Perry

3 5 7 9 11 13 15 17 19 21 23 25 27 29

-0.25

0

0.25

0.5

0.75

1

Age

Age 27 earn-ings


IQ

A “noncognitive”

miracle occurs…

Earnings impacts for the Job Training Partnership Act Program

7-18 19-30-$500

$0

$500

$1,000

$1,500

$2,000

$2,500Adult women Adult men Female youth

Months since random assignmentSolid marker denotes p<.05

Pro-gram

period

Patterns of fade-out

A mess!

• Impacts fade out in some interventions but don’t in seemingly similar interventions

• Sometimes, for the same intervention, some impacts fade out but others emerge decades later

Outline

I. OPTIMAL CONDITIONS FOR IMPACT PERSISTENCE

II. OTHER AVENUES FOR IMPACT PERSISTENCE

III. HOW TO RECONCILE ECE PROGRAM FADEOUT ON IQ WITH IMPACTS ON ADULT OUTCOMES?

Outline




What conditions lead to impact persistence?

• the “right kinds” of skills or capacities, or

• the “right kinds” of environments

When interventions change:





“Right kinds” of skills:

• Skills or behaviors fundamental for success in adulthood or for childhood attainments…

• that are malleable…

• and would not develop eventually in counterfactual conditions

“Right kinds” of environments:

• Malleable features of environments that are fundamental for promoting the “right kinds” of skills and behaviors

Fundamental and malleable skills

Fundamental Peripheral

More mallea

ble

Lessmallea

ble

• Conscientiousness (grit)

• g (IQ)

Who cares?

• Teaching to the test SAT test prep Flash cards• FAFSA rule knowledge?...

Fundamental AND malleable skills?

• Math• Literacy

• Implicit theories (Dweck)

• Self-concept (Cohen) • Academic motivation

• Executive function• Emotional self-

regulation

• Background knowledge

• Fixed vs. malleable intelligence (Dweck)

• Self-worth (Cohen et al.)

• Math: number line, fractions, algebra

• Literacy• Background

knowledge• Executive function• Prosocial behaviors

“Right kinds” of skills:

Skills or behaviors fundamental for success in adulthood or for childhood attainments…

that are malleable…

and would not develop eventually in counterfactual conditions

Fundamental AND malleable skills?

• Math• Literacy




regulation


• Fixed vs. malleable intelligence (Dweck)

• Self-worth (Cohen et al.)

• Math: number line, fractions, algebra

• Literacy• Background

knowledge• Executive function• Prosocial behaviors

Which would develop

eventually and therefore generate impact

fadeout?

Pace of development in counterfactual conditionsNull/slow => no

fadeout?Eventually =>

fadeout?• Math• Literacy




regulation




fadeout?• Fractions, algebra• Large vocabulary



• EF working memory• Emotional self-reg for

some


• Counting• Alphabet knowledge



• EF impulse control• Emotional self-reg for

most• Background knowledge

Examples of impact persistence

• Algebra

– Chicago’s double-dose algebra

• Self-concept

– Value affirmation

Persistent impacts of 9th grade double-dose algebra in Chicago

Source: Cortres et al. (2011)

ImpactB or higher in 9th-grade algebra

+13% *

A in 9th-grade algebra ns

Passed geometry in 10th grade +12% *Grade 11 math scores +.24 sd *

Graduated within 5 years +12% *Enrolled in any college +11% *

Cohen values affirmation impacts on low-GPA Black students

Prein-terven-

tion

Term 1 Term 2 Term 3 Term 4 Term 1 Term 2 Term 3 Term 4

-0.25

0.00

0.25

0.50

0.75

1.00

Me

an

GPA

in

Co

re C

ou

rse

s

Year 1 Year 2

Program period

Cohen caveats

• No impacts on higher-achieving Blacks and whites

• Some replication attempts show no consistent impacts (Dee, 2014)

Conditions where “eventual” development in counterfactual may

lead to fadeout

– Impulse control by age

– Did the Canadian SSP accelerate return to labor force that would have happened anyway?

Distraction time by age

3 4 5 6 7 8 9 100

100

200

300

400

500

600

700

Age

Tim

e t

o s

olv

e c

on

-fl

ict

in m

s

Adult lev-els

Posner and Rothbart (2007)

-12 -6 1 7 13 19 25 31 37 43 49

-10

0

10

20

30


Perc

enta

ge E

mplo

yed F

ull-

Tim

e

Program pe-riod

Did Canadian SSP speed up employment that would have occurred anyway?

-12 -6 1 7 13 19 25 31 37 43 49

0

10

20

30

40

50


Perc

enta

ge E

mplo

yed F

ull-

Tim

e

Program pe-riod

Con-trol

group


-12 -6 1 7 13 19 25 31 37 43 49

0

10

20

30

40

50


Perc

enta

ge E

mplo

yed F

ull-

Tim

e

Program pe-riod

Con-trol

group

Treat-ment group






“Right kinds” of environments:

• Malleable features of environments that are fundamental for promoting the “right kinds” of skills and behaviors

School quality can be a fundamental environmental feature

• Winning the lottery to enter one of NYC’s small high schools of choice (Untermann et al., 2014)

Outcomes SSC Control group Effect

Graduation

Graduated from high school 71.6 62.2 9.4 **

Regents diploma granted 50.2 43.5 6.7 **

Advanced Regents diploma granted

8.2 7.3 0.9

College readiness

Passed English Regents Exam at 75+

42.1 35.8 6.3 **

Passed Math Regents Exam at 75+

25.1 24.5 0.5

Post-secondary enrollment

Enrolled in post-secondary education

49.0 40.7 8.4 **

Winning the lottery to enter a NYC Small High School of Choice (n=14,608)

Is neighborhood quality a fundamental environmental feature?

• Moving to Opportunity suggests not for many outcomes in the US

MTO: Huge Differences in Neighborhood Poverty (Duration-Weighted)

01

23

45

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1

Con Compliers (Exp) Exp Compliers

De

nsity

Neighborhood Poverty Rate

Experimental Compliers vs Control Compliers

38

Outcome

Interventions

ImpactsBaseline Ages 0 to 5

Reading Assessment ns

Math Assessment ns

Baseline Ages 6 to 11

Reading Assessment ns

Math Assessment ns

Took SAT/ACT? ns

No Impacts on School Achievement

39

What else can sustain impacts?

When interventions:

• (from before) boost the “right kinds” of skills or environment

• are supported by post-TX sustaining environment

• lead to foot-in-the-door access to sustaining environments

• are sufficiently intensive to change foundational skills for children with bad counterfactual conditions

• treat enough children to generate positive peer effects

What can sustain impacts?

When interventions:






Building Blocks and sustaining environments

K-1 TX follow-through

No K-1 TX follow-through

High math in K-1

Low math in K-1

End of pre-K TX

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0.17

0.19

0.66

Impact on math in sd units




High math in K-1

Low math in K-1

End of pre-K TX

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0.17

0.15

0.19

0.66


nsSimilar results for low vs. high math home environments




High math in K-1

Low math in K-1

End of pre-K TX

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0.32

0.17

0.15

0.19

0.66


ns

p=.07

Sustain environments

Building Blocks suggest that environmental supports must be tailored explicitly to the nature of the prior treatment


When interventions:






Foot-in-the-door examples?

• SAT prep may affect college quality, which is known to have a positive impact on earnings

• Can FAFSA knowledge lead to college and later success?

• Was some of Abecedarian’s long-run success caused by lower rates of special ed and grade retention?

• Can pro-social behavioral interventions reduce or delay first arrests?

Foot-in-the-door

Foot-in-the-door links to the emerging literature on developmental cascades (Dodge et al. 2008)

• Sequence of positive or negative conditions that cumulate to good or bad outcomes

• Since <100% probabilities multiply, relying on cascades seems like a risky intervention strategy


When interventions:




• are sufficiently intensive to change foundational skills for children with difficult counterfactual conditions


IQ impacts in Perry and Abecedarian

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

-0.25

0

0.25

0.5

0.75

1

1.25

Age

Perry

Abecedarian

Perry vs. Abecedarian

• 1 or 2 years• Part-day ECE +

weekly home visits• Urban setting

• 5 years• Year-round full-day

ECE• Cumulative

curriculum• Rural setting

Perry Abecedarian

• African American children with low tested IQs

• High risk/low SES families


When interventions:






Generating peer effects

• Measles vaccination!• Deworming treatments in

Kenya generated large benefits for untreated children in treated schools (Miguel and Kremer, 2004)

• County spending on preschool?

More county spending on ECE boosts grades 3-5 achievement

Ladd, Dodge, & Muschkin (2014, JPAM), Muschkin, Dodge, & Ladd (in press, EEPA), Dodge, Ladd, Muschkin, & Bai (under review)

Grade 3 Grade 4 Grade 50

1

2

3

4

5

6More At Four Reading Scores

More At Four Math Scores

Month

s of

Learn

ing

Gain

ed

Outline




Death

Resurrection

4 5 6 7 8 9 10111213141516171819202122232425262728

-0.25

0.00

0.25

0.50

0.75

1.00

.44 sd

.87 sd

Age

IQ

Earnings

Perry’s IQ Swan Dive and Earnings Resurrection

A miracle

occurs…

Perry and Abecedarian both affected adult earnings

Do they have common (statistical) mediators?

Are they “cognitive” or “non-cognitive”?

ECE treatment

MediatorsAdult

earnings

Perry’s IQ Effects by Age

4 5 6 7 8 9 1011121314151617181920

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Eff

ect

Siz

e in s

d

unit

s

IQ

Source: Schweinhart et al., 2005; Effect sizes >.30 are p<.05, one-tailed test

Perry’s IQ and Achievement Effects by Age

4 5 6 7 8 9 1011121314151617181920

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Eff

ect

Siz

e in s

d

unit

s

IQLan-

guage

Math

Reading Adult liter-acy

Source: Schweinhart et al., 2005; Effect sizes >.30 are p<.05, one-tailed test

Perry’s Noncognitive Effects Ages 6-9

Source: Pinto, based on Heckman et al. (2014)

Index Sample item Impact

Academic potential

Creativity .31ns

Academic motivation

Alert and interested; motivated; persists

.37ns

Classroom conduct

Disobedient; impulsive; blames others

.40*

Personal behavior Absences; swears .36ns

Teacher dependence

Seeks constant reassurance .03ns

Emotional state Depressed; withdrawn .29ns

Emotional adjustment

Trust; level of emotional adjustment

.30ns

Perry (statistical) mediators

• Perry generated a raft of potential mediational effects

– achievement, but not IQ

– a number of potentially important “noncognitive” domains

Abecedarian’s IQ and Achievement Effects by Age

4 5 6 7 8 9 101112131415161718192021

-0.50

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Eff

ect

Siz

e i

n s

d

un

its

IQ

Source: Campbell et al., 2001; all effect sizes are p<.05.

Abecedarian’s IQ and Achievement Effects by Age

4 5 6 7 8 9 101112131415161718192021

-0.50

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Eff

ect

Siz

e i

n s

d

un

its

IQMath

Reading

Source: Campbell et al., 2001; all effect sizes are p<.05.

Abecedarian’s Cognitive and Noncognitive Effects by Age

4 5 6 7 8 9 101112131415161718192021

-0.50

-0.25

0.00

0.25

0.50

0.75

1.00

Age

Eff

ect

Siz

e i

n s

d

un

its

IQMath

Reading

Cognitive compe-tence

Global self-worth

Social compe-tence

Source: Campbell et al. (2001) and Campbell et al. (2002)

Abecedarian (statistical) mediators

• Abecedarian generated a number of potential mediational effects

– achievement and IQ

– but not a limited number of “noncognitive” measures

Two other relevant studies

• Both the Chicago Parent-Child program (Reynolds et al.) and kindergarten class quality (Chetty) affected adult earnings

– CPC affected reading and math achievement more than its “noncognitive” measures

– Chetty et al. found more impact on a noncognitive index than on achievement

Bottom lines on impact resurrection

An even bigger mess!

• Q: haven’t developmental psychologists invented a word for a big mess like this?• A: YES – “equifinality,” when many roads lead to the same outcomes

Summary

• Impact persistence requires treating fundamental and malleable skills that would not develop eventually in counterfactual conditions

• Other avenues are possible– Foot-in-the-door cascades from

peripheral skills are possible but risky– Environmental or intensive individual

interventions may work but are expensive

– Targeting children in the worst counterfactual conditions may be the best strategy

Bottom linesResearch priorities

• Long-run follow-ups, perhaps using administrative data

• Design better post-TX sustaining environment

• For interventions involving implicit theories, self-concept and motivation, we need more independent replications and longer-run follow-ups

That’s it!

Fundamental and malleable skills

Fundamental Peripheral

More mallea

ble

Lessmallea

ble

• Conscientiousness (grit)

• g (IQ)

Who cares?

• Teaching to the test SAT test prep Flash cards• FAFSA rule knowledge?...



fadeout?• Fractions, algebra• (much) Vocabulary



• EF working memory• Emotional self-reg for

some


• Counting• Alphabet knowledge



• EF impulse control• Emotional self-reg for

most• Background knowledge

Documents

Fadeout in human capital interventions: Death, miracles and resurrection