The Preschool Curriculum Evaluation Research (PCER) Program The 2005 OSEP National Early Childhood...

The Preschool Curriculum Evaluation Research (PCER) Program

The 2005 OSEP National Early Childhood Conference

February 8, 2005

Caroline Ebanks

James Griffin

Institute of Education

Sciences

AcknowledgementsPCER 2002 and 2003 Researchers

RTI PCER 2002 National Evaluation Coordinator

Ina Wallaceand Holly Rhodes

Mathematica Policy Research PCER 2003 National Evaluation Coordinator

John Love

Institute of Education Sciences (IES)

IES Statutory Mission

• Condition and progress of education in the United States

• Education practices that improve academic achievement & access to education opportunities

• The effectiveness of Federal and other education programs

Institute of Education Sciences

N a tio n a l C e n te r fo r E d u c a tio n S ta tis tic s

N a tio n a l C e n te r fo r E d u c a tio n R e se a rc h

N a tio n a l C e n te r fo r E d u c a tio n E v a lu a tio n

N a tio n a l C e n te r fo r S p e c ia l E d u c a tio n R e se a rc h

O ffic e o f th e D ire c to r N a tio n a l B o a rd fo r E d u c a tio n S c ie n c e s

Preschool Curriculum Evaluation Research

• Immediate Program Goals– Provide practitioners with evidence for selecting

preschool curricula– Support rigorous evaluations of preschool curricula using

random assignment

• Additional Goals– Support complementary research studies to supplement

the evaluation– Build capacity for collaboration between researchers and

practitioners

PCER: General Framework

• Three year longitudinal studies across the transition

from preschool to kindergarten and first grade

• Grantees were funded to implement curricula and conduct complementary research studies at each project site

• Focus on supporting school readiness, especially for disadvantaged children

• Common core of evaluation data collected by an outside contractor

Basic Design of Evaluation

• Two groups of grantees (PCER 2002 and PCER 2003) implementing a variety of preschool curricula

• Randomly assigned either schools, classrooms, or children to the treatment condition

• Baseline and three follow-up assessments—end of preschool, end of kindergarten, and end of first grade

Primary Research Questions

• What outcomes do specific curricula produce?

• Are outcomes sustained over time?

Secondary Research Questions

• What is the relationship between level of implementation and participation and curriculum effectiveness?

• How are experiences in kindergarten and first grade related to child outcomes?

PCER Project Cohorts

IES

Cohort 1: PCER 2002

National Evaluator:

RTI

7Grantees

Cohort 2: PCER 2003

National Evaluator:

MPR

5Grantees

Basic Design Features

• 14 Experimental Curricula

• 12 Grantees in 13 geographic locations

-California-Florida-Georgia-Kansas-Missouri-New Hampshire-New Jersey

-New York-North Carolina-Tennessee-Texas-Virginia-Wisconsin

PCER Curricula

• PCER 2002 Curricula

– Bright Beginnings (TN)– Creative Curriculum (NH, NC,GA,

TN)– Doors to Discovery (TX)– Early Literacy and Learning Model

(ELLM) (FL)– Ladders to Literacy (NH)– Let’s Begin with the Letter People

(TX)– Pre-K Mathematics with DLM

Express Math (CA/NY)– Project Approach (WI)

• PCER 2003 Curricula

– Curiosity Corner (FL, NJ, KS)

– The Language Focused Curriculum (VA)

– Literacy Express (FL-FSU)

– Open Court with DLM Early Childhood (FL-FSU)

– Project Construct (MO)

– Ready, Set, Leap! (Newark, NJ)

PCER 2002 CohortGrantee Research Site(s)

Dale Farran

Vanderbilt University

Tennessee

Cheryl Fountain

University of North Florida

Florida (Bay, Jacksonville, and Miami)

Richard Lambert

UNC Charlotte

Georgia and

North Carolina

Susan Landry

University of Texas Health Science Center, Houston

Houston, Texas

Doug Powell

Purdue University

Milwaukee, Wisconsin

Jeff Priest

University of New Hampshire

New Hampshire

Prentice Starkey

University of California, Berkeley

California and New York

PCER 2002 Sites and Curricula

Research Site CurriculumTennessee Creative Curriculum vs.

Bright Beginnings vs. Comparison

Florida Early Literacy and Learning Model vs.

Comparison

North Carolina/Georgia Creative Curriculum vs. Comparison

Texas Doors to Discovery vs. Let’s Begin with the Letter People vs. Comparison

Wisconsin Project Approach vs. Comparison

New Hampshire Creative Curriculum with Ladders to Literacy vs. Comparison

California/New York Pre-K Mathematics vs. Comparison

PCER 2003 Cohort

Grantee Research Site(s)

Bette Chambers

Success for All Foundation

Florida, Kansas, New Jersey

Anne Cunningham

University of California, Berkeley

Newark, New Jersey

Laura Justice

University of Virginia

Virginia (Culpeper and Wise counties)

Christopher Lonigan

Florida State University

Florida (Taylor and Leon counties)

Kathy Thornburg

University of Missouri

Missouri

PCER 2003 Sites and Curricula

Research Site Curriculum

Florida, New Jersey, Kansas

Curiosity Corner vs. Comparison

Newark, New Jersey Ready, Set, Leap! vs. Comparison

Virginia The Language Focused Curriculum vs. Comparison

Florida Literacy Express vs. Open Court/DLM vs. Comparison

Missouri Project Construct vs. Comparison

Grantees’ Complementary Research

• FSU (Chris Lonigan): Implementation of Open Court/DLM and Literacy Express

• NC/GA (Richard Lambert): Implementation of Creative Curriculum, 4th edition

• UNH (Jeff Priest): Implementation of Ladders to Literacy

Cross-Site Study Sample Size• Total sample size

– 2913 children and 317 classrooms

• PCER 2002:

– 1,686 Children and 180 Classrooms

• PCER 2003:

– 1,227 Children and 137 Classrooms

Types of Measures in the Cross-Site Study

• Child Assessments

• Classroom Observations

• Teacher interviews

• Teacher reports of child behavior

• Parent interviews

Overview

• Preschool program type

• Demographics

• Academic Outcomes

• Social Outcomes

• Teacher Characteristics

Types of Preschool Programs in the PCER 2002 Cohort

Head StartPublic SchoolPrivate Pre-K50%

44%

6%

Types of Preschool Programs in the PCER 2003 Cohort

Head StartPublic SchoolPrivate Pre-K

65%

23% 12%

Demographics

From the Parent Interview

Children’s Age (in years)

4

4.5

5

2002 Cohort 2003 Cohort

TreatmentControl

Children’s Gender (% Male)

10

20

30

40

50

60

70

80

2002 Cohort 2003 Cohort

Treatment

Control

Race/Ethnicity

0%

20%

40%

60%

80%

100%

2002 2003

White (non-Hispanic) African American Hispanic/Latino Other

Disability Status (% Yes)(Parent report)

5

10

15

20

25

30

35

2002 Cohort 2003 Cohort

TreatmentControl

Maternal Employment Status

0%

20%

40%

60%

80%

100%

2002 2003

Other

Not Working

Part-Time

Full-Time

Maternal Educational Level

0%

10%

20%

30%

40%

50%

60%

< HS HS Some postHS

BA orhigher

PCER 2002PCER 2003

Sample Comparisons: Poverty Indicators

0%

10%

20%

30%

40%

50%

60%

Medicaid (Child) AFDC/TANF

PCER 2002PCER 2003FACESECLS-K

PCER Sample Comparisons-Child

PCER 2002 PCER 2003 FACES ECLS-K

Gender (% Male) 53% 48% 51% 50%

Race/Ethnicity

-White (non Hispanic) 30% 36% 31% 57%

-African American (non-Hispanic)

44% 52% 26% 14%

-Asian/Pacific Islander 2% 1% 1% 7%

-Hispanic/Latino 17% 5% 30% 18%

-American Indian/Eskimo

0.4% 1% 2% 2%

-Other 6% 6% 18% 3%

Disability Status 15% 19% 9% --

Teacher Education

0%

20%

40%

60%

80%

100%

2002Treatment

2003Control

2003Treatment

2003Control

BA/BS or >< BA/BSHS or less

Academic Outcomes

From the Child Assessment

-Baseline Data-

Child Math Assessment (CMA) Composite

0.25

0.3

0.35

0.4

0.45

0.5

2002 Cohort 2003 Cohort

Child Math Assessment (CMA)Composite score,

by treatment status

00.05

0.10.15

0.20.25

0.30.35

0.40.45

0.5

PCER 2002 PCER2003

TreatmentControl

Woodcock Johnson-III: Applied Problems

80

85

90

95

100

2002 Cohort 2003 Cohort

Woodcock Johnson-III: Applied Problems, by treatment status

80

85

90

95

100

PCER 2002 PCER 2003

TreatmentControl

Test of Early Reading Ability (TERA): Reading Quotient

80

85

90

95

100

2002 Cohort 2003 Cohort

Test of Early Reading Ability (TERA): Reading Quotient,

by treatment status

80

85

90

95

100

PCER 2002 PCER 2003

TreatmentControl

Woodcock-Johnson-III: Letter Word Identification

90

95

100

105

2002 Cohort 2003 Cohort

Woodcock-Johnson-III: Letter Word Identification,

by treatment status

90

92

94

96

98

100

102

PCER 2002 PCER 2003

TreatmentControl

Woodcock-Johnson-III: Spelling

85

90

95

100

2002 Cohort 2003 Cohort

Woodcock-Johnson-III: Spelling,by treatment status

80

85

90

95

100

PCER 2002 PCER 2003

TreatmentControl

Peabody Picture Vocabulary Test (PPVT) III

80

85

90

95

2002 Cohort 2003 Cohort

Peabody Picture Vocabulary Test (PPVT) III,

by treatment status

80

85

90

95

100

PCER 2002 PCER 2003

TreatmentControl

Test of Language Development (TOLD): Grammatical

Understanding

5

6

7

8

9

10

2002 Cohort 2003 Cohort

TOLD: Grammatical Understanding,

by treatment status

5

6

7

8

9

10

PCER 2002 PCER 2003

TreatmentControl

Social Outcomes

From Teacher Report Form

-Baseline Data-

Social Skills Rating System (SSRS): Social Skills

90

95

100

105

2002 Cohort 2003 Cohort

Social Skills Rating System (SSRS): Social Skills,

by treatment status

90

95

100

105

PCER 2002 PCER 2003

TreatmentControl

SSRS: Problem Behaviors

90

95

100

105

2002 Cohort 2003 Cohort

SSRS: Problem Behaviors, by treatment status

90

95

100

105

PCER 2002 PCER 2003

TreatmentControl

Preschool Learning Behaviors Scale (PLBS)

45

47

49

51

53

55

2002 Cohort 2003 Cohort

Preschool Learning Behaviors Scale (PLBS),

by treatment status

45

47

49

51

53

55

PCER 2002 PCER 2003

TreatmentControl

What we can learn from PCER

• Which curricula appear to better support school readiness

• The type of support teachers need in order to effectively implement curricula

Additional Benefits of PCER

• Support collaboration between practitioners and researchers which can produce:

– Research that is more relevant to practice– Researchers and practitioners who understand

how to be good collaborating partners– Model for other education research programs

PCER: Contact Information

• Program Officers:– Caroline Ebanks, PhDE-mail: Caroline.Ebanks@ed.gov

– James Griffin, PhDE-mail: James.Griffin@ed.gov

• PCER Websites:– PCER 2002: http://pcer.rti.org/– PCER 2003: www.pcer-mpr.info