Does increasing children’s access to books at home mediate the long-term effects of scaffolded summer

reading on reading comprehension?

James Kim, Harvard University Jonathan Guryan, Institute for Policy Research,

Northwestern University David M. Quinn

Helen Chen Kingston Harvard University

July 18, 2015 SSSR – 22nd Annual Meeting

The contents of this presentation were developed under a grant from the U.S. Department of Education, Investing in Innovation (i3) Program. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal government.

Goals

1.  Estimate long-term effects on the NC End of Grade reading comprehension test (replicate ITBS and less missing data)

2.  Examine mediating effects of reading engagement - multidimensional construct and individualized for each child

The Theory Behind READS •  Low-SES (socioeconomic status) children fall behind in the

summer months. •  We can remedy this by providing:

–  Access to books at home; –  Books matched to child’s reading level and interests; and –  Comprehension routines in school: (1) end-of-year

classroom lessons, (2) afterschool family literacy event with parents

•  Comprehension routines direct attention to text structure –  understand author’s purpose, understand central ideas (v. non-central

ideas), save processing time by using authors’ schema

What is “scaffolding” and why is it important in the context of READS?

•  The notion of scaffolding is grounded in sociocultural theory; teachers, experts help children learn in the zone of proximal development – distance between actual and potential development (Vygotsky)

•  In READS, we scaffold summer reading in 2 ways –  (1) Simplify routine in classrooms: Teachers take

whole routine and break into parts with lesson texts, promote situational interest

–  (2) Simplify routine in homes: Books matched to child’s interest and reading level, and trifold tool breaks routine into parts, tap personal interest

Scaffolding comprehension routines in classrooms

•  Narrative text – 3 lessons with chapter book – Jeremy Bean – Teachers teach - content-based prediction

routine with a trifold tool •  Informational text

– 3 lessons with 3 informational texts – Focus on getting “main idea” (macrostructure) of

3 genres of informational text

Classroom Narrative lesson •  Before reading

–  Words listed in the order they appear in text

–  Words are anchored to narrative text structure (main character, setting)

•  During reading –  Teacher reads aloud, stops

and asks literal and inferential questions

•  After reading – check prediction

Classroom - Informational lesson •  Before reading

–  Three types of informational books: books about animals, nature, or biography

–  We’ll read about each text of book

–  Informational books have different text structures vs. narrative

–  Authors focus on main idea in informational texts

Before reading a biography: Info impressions (~ story impression)

Lesson 4 passage © Mary Burkhauser

Classroom - Informational lesson 4 •  During reading

–  Teacher reads aloud, stops and asks literal and inferential questions

•  After reading (home work, simulate summer context)

–  check information guesses

–  2 literal, 1 main idea –  Well-matched text?

Scaffolding comprehension routines with home book reading

•  Scaffolding home book reading – Children mailed 10 books (2 lesson texts, 8

matched texts – algorithm match on reading level and interest survey)

– Over 519 books (153 narrative, 366 informational)

– Trifold scaffolds home book reading (break the whole routine into parts)

Bethany

School Activities Home Activities School-Based Reading Tests

Classroom lessonsTeacher-directed comprehension routines involving children and peers Proximal outcomes Distal outcomes

Increased Quantity and Quality Improved reading comprehensionof Summer Book Reading NC - EOG Test

School family literacy nightTeacher-directed comprehension routinesinvolving children and parents

Socioeconomic Status (SES) of Children's Neighborhoods, Schools, and Families

Hypothesis: The trifold is designed to foster children’s engagement with text in school lessons and home book

reading activities.

Teacher-child: Trifold

Teacher-child-parent:

Trifold

Child-book: Trifold

Evaluation Timeline

!"#$%&'()*+' ''!,--.#'()*+' /011'()*+'' 2$%3.#'()*4' ''!"#$%&'()*4

!3,5.%36'307.89:!'3.636;

<=>

?<@A!$%3.#B.%3$C%

!3,5.%36'307.89:!'3.636;<=>'D

6..'.EE.F36'CE?<@A!'

$%3.#B.%3$C%

!GH==I !GH==I

13

Year 3-4 Design: 59 NC Schools in 7 districts and 463 homeroom blocks (5,569 students G2/G3)

(1) block teachers and students by homeroom,

and (2) randomly assign to

(1) Control group (6 math lessons)

(2) READS Treatment Condition

(3 narr, 3 info lessons) (after school literacy) (10 matched books +

Reading trifolds)

Measures •  Reading Comprehension

– North Carolina End-of-Grade Reading Test (Grades 3 and 4, Spring 2014)

•  Reading Engagement – Conceptual definition: Behavioral, Motivational,

Cognitive dimension (Fredricks, Blumenfeld, & Paris, 2004; Wigfield et al., 2008)

– Operational measure: trifold – specific to mailed book

Summer book trifolds returned

•  N = 3,174 (treatment group) •  Trifold returns

– 0 returns (36%) – 1 trifold (3%) – 2-5 trifolds (28%) – 6-10 trifolds (33%)

•  Mean comp. items correct – M = 7.50, SD = 7.92, Min = 0, Max = 30

Long-term ITT effects of READS

•  ITT model: •  Delayed posttest outcomes (spring 2014)

– Reading comprehension

𝑦↓𝑖↑𝑃𝑜𝑠𝑡 = α↓0 + α↓1 𝑅↓𝑖 + α↓𝑦 𝑥↓𝑖↑𝑆𝑝2013 + 𝜆↓𝑐 + 𝜀↓𝑖 

ITT Analysis: Positive main effect (driven mostly by high poverty schools = 39, FRL 75 = 100%)

0.04

0.05

0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

Overall High Poverty Schools Moderate Poverty Schools

𝛽_1=  .01 (se = .029) (t = 0.68)

𝛽↓1 =  .04 (se = .017) (t = 2.35) (p < .05)

𝛽↓1 =  .05 (se = .021) (t = 2.38) (p < .05)

Long-term TOT effects of READS

𝑦↓𝑖↑𝑃𝑜𝑠𝑡 = 𝛽↓0 + 𝛽↓1 𝑀↓𝑖 + 𝛽↓𝑦 𝑥↓𝑖↑𝑆𝑝2013 + 𝜆↓𝑐 + 𝜀↓𝑖 

𝑀↓𝑖 = 𝜋↓0 +𝜋↓𝑐 𝑅↓𝑖 + 𝜋↓𝑦 𝑥↓𝑖↑𝑆𝑝2013 + 𝛿↓𝑐 + 𝜇↓𝑖  •  Estimate TOT using 2SLS :

The Effect of Tri-folds on EOG (MOL) 0

.51

1.5

22.

5

Effe

ct o

n E

OG

in M

onth

s of

Lea

rnin

g

1 Trifold 5 Trifolds 10 Trifolds

by Number of Trifolds Returned in Summer 2013 (Grades 2 & 3)READS Effect on EOG 2014

21

The Effect of Tri-folds by School Poverty

Conclusion •  We can improve reading comprehension by

fostering children’s engagement with books – Positive long-term effects on NC EOG test –  Identify mechanisms (trifold and summer book

reading) linked to student outcome •  RCT 4: How do we sustain evidence-based

literacy interventions? – Compare this version of READS (scripted) to

adaptive READS (teachers learn research principles and adapt program)

literacy-reforms.gse.harvard.edu

Delayed effects on home access to book (0 to 50 scale)

25 24

28 28 27 30

0

5

10

15

20

25

30

All Schools High Poverty Schools Moderate Poverty Schools

Control READS