Study of the Impact of Pre-Kindergarten Experiences on ......consecutive years (SY 2011-12 through SY 2013-14) of 4percent increases in the number of students served (Barnett, Carolan,

Study of the Impact of Pre-Kindergarten Experiences

on FCPS Students

Final Report

May 2016

Office of Program Evaluation

FAIRFAX COUNTY PUBLIC SCHOOLS Karen Garza, Ed.D.

Superintendent of Schools

Marty K. Smith Chief of Staff

OFFICE OF PROGRAM EVALUATION Recardo V. Sockwell, Ph.D., Director Ludmila Z. Hruda, M.S.Ed., Manager

Janine Lacina, M.A., Evaluator Chantal Follett, Ph.D., Evaluator

Acknowledgements

The Office of Program Evaluation would like to thank the following Fairfax County Public Schools staff and early childhood authorities for their support and engagement in the research process as participants of the Evaluation Team:

Maura Burke, Director, Instructional Services Department, Fairfax County Public Schools

Debra Fulcher, Instructor, College of Education and Human Development, George Mason University

Mary Miller, Principal, Brookfield Elementary School, Fairfax County Public Schools

Grace Taylor, Executive Principal, Region 3, Fairfax County Public Schools

Cecilia Vanderhye, Principal, Belvedere Elementary School, Fairfax County Public Schools

Table of Contents

Report Overview ........................................................................................................................................... 1

Introduction .................................................................................................................................................. 2

Study Background ....................................................................................................................................... 11

Study Findings ............................................................................................................................................. 13

Conclusions ................................................................................................................................................. 39

Recommendations to Decision Makers ...................................................................................................... 45

Appendixes

Appendix A – Qualities of Good Preschool Programs

Appendix B – The Debate about Preschool

Appendix C – Study Design

Appendix D – Methodology

Appendix E – On Parent Survey and Additional Details on Programs

Appendix F – Additional Information on Formal Pre-Kindergarten Experiences

Appendix G – Pre-K Experience by Pyramid

Appendix H – Additional Outcomes for Pre-Kindergarten Versus No Formal Pre-Kindergarten

Appendix I – Additional Outcomes for FECEP Matched

Appendix J – Additional Outcomes for Former FECEP Students with One and Two Years of

Participation

Appendix K – Outcomes for Five Categories of Pre-Kindergarten Experience

Appendix L – Bridge to Kindergarten

Appendix M – Additional Information on Funding Preschool Programs

Appendix N – Additional Information About Return on Investment Calculations

Appendix O – Program Management Response to OPE’s Recommendations

Fairfax County Public Schools, Office of Program Evaluation May 2016

ii

Report Overview

The Study of the Impact of Pre-Kindergarten Experiences on FCPS Students is organized into five sections:

Introduction provides a selected review of the literature on the benefits of high-quality pre-kindergarten programs, as well as the specific benefits accrued by students. The introduction also provides an extensive discussion about the types of pre-kindergarten experiences available to Fairfax County Public School students prior to Kindergarten, especially those provided by FCPS.

Study Background details the purpose of the study (context and design) as well as the data sources used to answer the study questions.

Study Findings presents findings about both student outcomes and FECEP program costs, including cost avoidance the school division accrues for students who have participated in the program.

Conclusions summarizes the study findings, often in light of what other research has found.

Recommendations to Decision Makers details the Office of Program Evaluation’s (OPE) recommendations to the Superintendent and School Board when considering an expansion of funding and support for pre-kindergarten services for the division’s eligible students.

Appendixes are provided to supplement and support discussions in the report. Several appendixes provide background information to help the reader understand critical concepts or findings, such as a description of the qualities of a good preschool experience and an overview of research on the effectiveness of pre-kindergarten programs.

In addition to this report, an Executive Summary is available that highlights key findings, conclusions and recommendations.


1

Introduction

In the United States, entering kindergarteners include students with an assortment of prior experiences that can impact their success as elementary students and beyond. Some students have spent most of their early years at home with family or a caregiver. Others have had more formal experiences outside the home. Generally, formal experiences for young children are categorized in one of two ways: (a) childcare (i.e., providing supervision and safety, and managing well-being while the parent works, attends school, or pursues other activities) or (b) preschool (i.e., providing organized experiences that promote the child’s development of academic, social, emotional, physical, and other knowledge and skills).1 There is, nonetheless, a great deal of overlap between the two as quality caregiving promotes a child’s development. Throughout this report, the term “pre-kindergarten” or “pre-k” is used to describe a formal experience outside the home and is intended to encompass both childcare and preschool. In contrast, this report reserves the term “preschool” for organized experiences that promote a child’s development.

While parents can emphasize academic, social, emotional, physical, and other knowledge and skills at home, many rely on preschool experiences to support such development and pay tuition for their young children to participate. Other parents rely on external support. For instance, since 1965, when Head Start began, the federal government has funded tuition-free preschool experiences for children living in poverty towards the aim of improving the developmental outcomes of participating children. And, many states and localities now choose to supplement such funding for some or all students in the one to two years prior to Kindergarten entry. Thus, when not available universally to all children, these publicly-funded programs typically target children at-risk for school failure (e.g., from low-income family, foster child, English learner, and child with a disability, as required by the Individuals with Disabilities Education Act-IDEA).

Recent participation data indicate that nationally, approximately 68 percent of four-year-olds and 42 percent of three-year-olds are enrolled in publicly- or privately-funded preschool programs (NCES, 2015). While the overall percentage of three- and four-year-old students enrolled in preschool programs has remained steady over the past decade, the percentage of students in publicly-funded programs increased by 15 percent between 2001-02 and 2013-14. Moreover, in general, three-year-olds are less likely than four-year-olds to be served in state-funded programs (4 percent versus 29 percent). Among the states offering funding for pre-k programs (N=40), Virginia ranks 27th in the percentage of preschool-aged students served, enrolling approximately 18 percent of four-year-olds and 9 percent of income-eligible three-year-olds (Barnett, Carolan, Squires, Brown, & Horowitz, 2015). Consistent with national trends, Virginia recently began expanding its state funding (Virginia Preschool Initiative [VIP]) as reflected in three consecutive years (SY 2011-12 through SY 2013-14) of 4 percent increases in the number of students served (Barnett, Carolan, Squires, Brown, & Horowitz, 2015).

While proponents have sought to expand pre-k based on a body of supporting evidence, a sufficient number of studies have cast doubts on its value (See Appendix B: The Debate about Preschool). Thus, school divisions are well advised to investigate which types of pre-kindergarten programs or experiences have the desired impacts on their incoming kindergartners. Such an understanding is especially true within FCPS, given the division’s inclusion in its Strategic Plan, Ignite, to “Provide quality early childhood experiences aligned with Portrait of a Graduate outcomes that are designed to prepare students to successfully enter Kindergarten” (p.18).

1 See, for example, http://www.babycenter.com/0_how-preschools-differ-from-daycare-centers_5958.bc. Fairfax County Public Schools, Office of Program Evaluation May 2016

2

http://www.fcps.edu/news/docs/FCPS%20Strategic%20Plan%202015-20_Final_Revised1a.pdfhttp://www.babycenter.com/0_how-preschools-differ-from-daycare-centers_5958.bc

Pre-Kindergarten Experiences of FCPS Students

Types of Pre-Kindergarten Experience

Reflecting the national trend, students entering Kindergarten in FCPS come from a wide range of pre-k experiences. However, a limited amount of information is known about these experiences since FCPS gathers this information via a single form (known as the Pre-Kindergarten Experience form) completed by parents when registering students for Kindergarten. The primary purpose for the data gathered from this form is to provide required information to the Virginia Department of Education. The form asks parents to identify the setting of the child’s pre-k experience (which, as categorized on the form, typically reflects the source of funding for the child’s experience), as well as the number of hours the child participated in the experience (see http://fcpsnet.fcps.edu/it/offices/eis/ doc_mgmnt/information /se228.pdf for the form). For the study, pre-k experiences were grouped into two broad categories: Formal Pre-K and No Formal Pre-K. Table 1 indicates the parent responses from the form that fell into each of these groups.2

The Formal Pre-K group includes students who were likely to have received some sort of preschool curriculum, whether publicly- or privately-funded.3 The final column of Table 1 shows the percentage of current kindergarteners (SY 2015-16) from the Formal Pre-K and No Formal Pre-K groups.4 The vast majority of parents (75 percent) reported their children attended a formal pre-kindergarten setting outside the home. Parents reported a smaller percentage (11 percent) of incoming Kindergarten students in SY 2015-16 had not participated in a formal pre-kindergarten experience (i.e., they were cared for at home). It should be noted that both the “Formal Pre-K” and “No Formal Pre-K” groups may represent any income level. That is, these groups may be composed of a range from the most needy to the least needy students.

2 Finer categories (Center, Tuition; Center, No Tuition; FECEP; Licensed Home Care; No Pre-K) were also analyzed for this study as reported in Appendix E. 3 The choice to include only the parent responses from the Pre-Kindergarten Experience form that are listed on Table 1 as students with formal pre-k experience (and who likely received some sort of preschool curriculum) was made in conjunction with central office staff from the Office of Early Childhood Curriculum and Grant Management in ISD. More specifically, students whose parents responded “Licensed Family Daycare Provider” were not included in the group because these settings most often reflect childcare without an early childhood curriculum. In contrast, although some of the responses included in the Formal Pre-K group reflect attending a childcare center, such settings typically do have some type of early childhood curriculum. 4 Although the literature has often used care setting as a proxy for quality, it is important to note that the pre-kindergarten setting alone is not sufficient for understanding the quality of what the child experiences or how well students are prepared for Kindergarten. Instead, these five categories provide insight into cost to the family rather than the quality of the pre-kindergarten experience the child received.


3

http://fcpsnet.fcps.edu/it/offices/eis/%20doc_mgmnt/information%20/se228.pdf

Table 1 Description of Pre-Kindergarten Experience Groups and

Distribution Among FCPS’ SY 2015-16 Kindergarteners (n=12,953)* Pre-Kindergarten Experience Group

Included Pre-Kindergarten Experience Form Responses Percent (n) Students

Formal Pre-K • Government Tuition Charged • Private Provider • Coordinated Pre-Kindergarten Classroom • VA Pre-Kindergarten Initiative (VPI) • Title I Preschool • Head Start • Coordinated Special Education • Special Education Only

Note: FCPS Student Information System data was used to override any selections the parent had made on the Pre-Kindergarten Experience form to identify students who had participated in FECEP and include them in this group

75 (9,766)

No Formal Pre-K • No Formal Institutional Pre-Kindergarten Program 11 (1,429) Data Source: Decision Support/DIT (DIT), SY 2015-16 Fall student membership (which includes the Pre-Kindergarten Experiences Form data).

*Students who were not identified as formal or no pre-k were excluded from the analyses comparing formal/no formal pre-k students since too little was known about their experiences to interpret related findings. These students include those whose parents responded “Licensed Family Daycare Provider” (n=316) or “Other” (n=324) on the Pre-Kindergarten Experience form or who did not have information provided on the form (n=1,118). Excluded students represent 14 percent of the SY 2015-16 Kindergarten class.

Types of Pre-Kindergarten Experience by Student Subgroups

Demographic differences exist among the most recent FCPS students who enter Kindergarten with and without a formal pre-kindergarten experience. Table 2 shows demographic information for the two groups of pre-k experience (Formal Pre-K, No Formal Pre-K) for entering kindergartners during SY 2015-16. Table 2 shows the students in the Formal Pre-K and No Formal Pre-K groups are dissimilar in their subgroup percentages for the majority of subgroups. More specifically, the No Formal Pre-K group is largely Hispanic (53 percent), English learners (67 percent), and low-income families (FRM: 59 percent). In contrast, there is no subgroup that forms a majority in the Formal Pre-K group, though almost half this group is White (46 percent). Also of note, students with disabilities form a sizeable portion of the Formal Pre-K group (10 percent) but a negligible portion of the No Formal Pre-K group (1 percent).

Types of Pre-Kindergarten Experience by County Location

The particular geographic region in which a student lives in the county is associated with the student’s pre-k experience. Figure 1 provides the percentages of students in each pyramid in the No Formal Pre-K group. That is, this figure represents concentrations of students not participating in a formal pre-kindergarten activity prior to entering Kindergarten. Pyramids with the highest percentages of students with no formal pre-k experience upon Kindergarten entry include Mount Vernon (with 27 percent No Formal Pre-K), Herndon (24 percent), and Falls Church (21 percent). The Edison (17 percent), Lee (16 percent), and Stuart (16 percent) pyramids also have more than 15 percent of incoming kindergartners


4

with no formal pre-kindergarten experience. Pyramids with the lowest percent of students with no formal pre-k experience upon Kindergarten entry include Langley (3 percent), Oakton (3 percent), and Mclean (4 percent). A similar graphic depicting the Formal Pre-K group in each pyramid is available in Appendix G.

Table 2 Demographic Descriptives (percent, n) of Students

With and Without Formal Pre-Kindergarten Experience in SY 2015-16 Kindergarten Pre-Kindergarten Experience Type

Race/Ethnicity Students with

Disabilities (SWD)

English Learner

(EL)

Free/ Reduced

Price Meals (FRM)

Asian Black Hispanic White Other

Formal Pre-K (n=9,766)

19 (1,848)

9 (899)

20 (1,911)

46 (4,451)

7 (657)

10 (999)

28 (2,739)

24 (2,350)

No Formal Pre-K (n=1,429)

14 (203)

9 (9,131)

53 (9,760)

19 (265)

5 (70)

1 (15)

67 (957)

60 (857)

All Students Entering Kindergarten (n=12,953*)

18 (2,372)

9 (1,224)

26 (3,389)

40 (5,150)

6 (818)

8 (1,044)

36 (4,698)

31 (4,024)

Data Source: DIT, SY 2015-16 Fall student membership (which includes the Pre-Kindergarten Experiences Form data).

*Includes students whose parents responded “Licensed Family Daycare Provider” (n=316) or “Other” (n=324) on the Pre-Kindergarten Experience form or who did not have information provided on the form (n=1,118).

Pre-Kindergarten Experiences Offered by FCPS

FCPS offers four pre-k programs5 mostly targeted towards students in the year prior to Kindergarten entry: (1) Family Early Childhood Education Program/Head Start (FECEP), (2) Bridge to Kindergarten (B2K), (3) Preschool Special Education services, and (4) Home Instruction for Parents of Pre-Kindergarten Youngsters (HIPPY). Each program is designed to promote development of skills and knowledge necessary to be successful upon entering school.

Only the first two programs were included in this study for several reasons. First, Preschool Special Education services are individualized to the needs of the child and family rather than a unified program that provides a uniform set of experiences for students. Second, HIPPY provides no direct intervention for children. Rather, the program works directly with parents to build their capacity to be their child’s first teacher. Parents are likely to vary in how they apply their knowledge and use the resources. Also, students from participating families are not identified in FCPS’ student records data. Thus, evaluation staff did not

5 FCPS also serves preschool students with disabilities through various individualized approaches under the umbrella of Early Childhood Identification and Services. Some students participate in FECEP for part of the day and receive services more specifically targeted to their disability for the other half of the day in a special education classroom. Others attend a preschool autism class (PAC) while others receive resource services in the community as determined by an IEP team. Since all of these services can take on a variety of possibilities, rather than a unified programmatic approach, they were not investigated for this study. If not participating in FECEP, students receiving classroom-based services should have been counted as part of the Center No Tuition group. Among SY 2015-16 kindergartners, FCPS served approximately 400 students with disabilities this way, which would represent about 25 percent of the Center No Tuition students. The remaining students in the Center No Tuition group participated in some type of publicly-funded center experience not offered by FCPS (likely through Fairfax County).


5

feel this program’s effect could be measured with validity. (For more information on FCPS’ HIPPY program, see http://www.fcps.edu/is/fam/hippy.shtml).

Furthermore, given the brief intervention period (three weeks) for students in the B2K program and its intended impact, the evaluation staff felt the available measures were not sufficient to reach valid conclusions about the success of this program. However, exploratory findings showed no advantages for B2K students versus their matched peers. These data are available in Appendix L. Thus, the focus of this study is on FECEP, which is FCPS’ most extensive preschool offering.

Figure 1 Percent of Students Who Had No Formal Pre-K Experience by Pyramid

SY 2015-16 Kindergartners



6

http://www.fcps.edu/is/fam/hippy%20%20%20%20.shtml

Family Early Childhood Education/Head Start Program (FECEP)

FECEP provides early childhood services to income-eligible three- and four-year-olds by offering comprehensive services to support children’s academic, social-emotional, and physical development. During SY 2015-16 the program has 99 classrooms located at 65 sites. The goal of the program is to provide high-quality early childhood educational services to students and involve parents in the process. Using a blended funding stream from federal, state, and local contributions, all FECEP classrooms operate under both the Federal Head Start Performance Standards and the Virginia Pre-Kindergarten Initiative (VPI) Guidelines.

Students are admitted into FECEP based on a formula that assigns “points” to students for various factors (poverty level, being a foster child, etc.). Within this design, the student with the highest point total at a specific site receives the next slot as it becomes available. This point system means that at times a three-year-old is prioritized above a four-year-old.6 Furthermore, federal guidelines encourage FECEP to serve eligible Early Head Start participants to maintain continuity of services. Approximately 50 three-year-olds each year transition from FCPS’ Early Head Start to FECEP and the remaining three-year-olds enter FECEP primarily from Fairfax County’s similar early intervention services. As shown in Figure 2, the number of participating students has increased by approximately 200 students over the last three years, with the increase attributable to rising numbers of four-year-old participants. Over the course of the current school year (SY 2015-16), 1,755 children (1,612 four-year-olds and 143 three-year-olds) have participated in FECEP.

Figure 2 Number of Three- and Four-Year-Old Students

Attending FECEP, SY 2013-14 to SY 2015-16

As shown in Table 3, the current Kindergarten class (SY 2015-16) had 1,410 former FECEP students, which was approximately 11 percent of the incoming Kindergarten class. The majority of former FECEP students were living in low-income families (FRM: 95 percent), English learners (72 percent), and Hispanic (57 percent). Thus, former FECEP participants were demographically more similar to students described

6 Early Childhood Learning & Knowledge Center (ECLKC). Head Start Policy and Regulation: Eligibility, Recruitment, Selection, Enrollment and Attendance. See http://eclkc.ohs.acf.hhs.gov/hslc/standards/pc/i_pc_actual.htm.

Data Source: Early Childhood Curriculum and Grant Management/ISD (ECCGM), FECEP Enrollment, SY 2013-14 through 2015-16.


7

http://eclkc.ohs.acf.hhs.gov/hslc/standards/hspps/45-cfr-chapter-xiii/45-cfr-chap-xiii-eng.pdfhttp://www.doe.virginia.gov/instruction/early_childhood/preschool_initiative/guidelines.pdfhttp://www.doe.virginia.gov/instruction/early_childhood/preschool_initiative/guidelines.pdfhttp://eclkc.ohs.acf.hhs.gov/hslc/standards/pc/i_pc_actual.htm

above as not having participated in formal pre-kindergarten experiences than to students who had participated in formal pre-kindergarten.

Table 3 Demographic Descriptives (percent, n) of Former FECEP Students

In SY 2015-16 Kindergarten Pre-Kindergarten Experience Type

Race/Ethnicity Students with

Disabilities (SWD)

English Learner

(EL)

Free/ Reduced

Price Meals (FRM)

Asian Black Hispanic White Other

FECEP (n=1,410)

12 (167)

17 (236)

57 (810)

12 (174)

2 (23)

8 (118)

72 (1,018)

95 (1,337)

All Students Entering Kindergarten (n=12,953*)

18 (2,372)

9 (1,224)

26 (3,389)

40 (5,150)

6 (818)

8 (1,044)

36 (4,698)

31 (4,024)

Data Source: DIT, SY 2015-16 Fall student membership.

Concentrations of former FECEP students as they enter Kindergarten vary in different parts of the school division. Figure 3 indicates the percentage of the SY 2015-16 Kindergarten class in each pyramid that had participated in FECEP. As seen in the figure, much of the FECEP participation is concentrated in the southeast portion of the county, with 21 to 25 percent of the Kindergarten class this year in the Stuart, Falls Church, Annandale, and Lee pyramids being former FECEP students. Many pyramids, however, have under 5 percent of their incoming kindergartners having participated in FECEP.


8

Figure 3 Percent of Students Who Had Participated in FECEP by Pyramid

SY 2015-16 Kindergartners

As the number of low-income students has grown in the county, FCPS has had difficulty providing sufficient numbers of FECEP slots to handle all eligible students. Thus, a waiting list of students has been created each year. During SY 2014-15, a total of 301 students (39 three-year-olds, 262 four-year-olds) were on FCPS’ FECEP waitlist. Figure 4 depicts the waitlist in different parts of the county for three- and four-year-olds in relation to each FECEP site. The larger the circle, the larger the waitlist for that site. As can be seen in the figure, the largest concentration of the waitlist was in the southeastern portion of the county though other areas also had relatively large concentrations of students on the waitlist. For a table with the specific number of students on the waitlist at each FECEP site, see Appendix G.



9

Figure 4

SY 2014-15 FECEP Waitlist at Each FECEP Site

Three-Year-Olds (n=39)

Four-Year-Olds (n=262)

FCPS places capacity limits on the Three-Year-Old waitlist because the program has limited spaces for these students. Thus, some three-year-olds whose parents/guardians inquire about FECEP are not placed on the waitlist and not included in this figure.

Data Source: ECCGM, Waitlist data per FECEP site, SY 2014-15.


10

Study Background

Study Purpose

The purpose of this study is (a) to identify to what extent differences exist in academic and behavioral outcomes for students with and without formal pre-kindergarten experiences as students enter and continue through the elementary grades and beyond in FCPS, and (b) to investigate funding for FCPS’ FECEP program, including return on investment. This report differs from prior FECEP data monitoring reports (such as http://www.fcps.edu/pla/ope/docs/fecep/sy1415_analyses.pdf), which supply detailed annual data about FCPS’ former FECEP participants, in that it (a) takes a broader view to encompass all pre-kindergarten experiences, not just those of FECEP participants, (b) uses a more methodologically rigorous approach to examine impacts of FECEP participation on students, and (c) includes findings on FECEP costs and return on investment to the school division for FECEP expenditures.

Study Design

For this study, OPE staff developed an initial set of study questions based on the information needs identified by the Superintendent. The questions were refined based on recommendations from an advisory team composed of representatives from central office and school- and region-based staff. The study will provide the FCPS School Board and Leadership Team with information about the impact of pre-kindergarten experiences on student outcomes, with a specific look at FCPS’ FECEP program, including its costs and potential return on investment.

The study gathered evidence to address five questions:

• To what extent does participation in FECEP benefit students once they enter Kindergarten and progress through school?

o FECEP students versus matched peers who did not attend pre-kindergarten? o FECEP students who experienced one versus two years of the program?

• To what extent do students with and without formal pre-kindergarten experience differ in educational outcomes once they enter Kindergarten and progress through school?

• What costs and funding are associated with FECEP, FCPS’ preschool offering? • To what extent is FCPS’ approach to funding pre-kindergarten programs similar to other

jurisdictions? • What is FCPS’ Return on Investment (ROI) from the FECEP program?

Data Sources

Evidence in this report came from the following sources:

• For Study Questions 1 and 2, the study examined areas that mirror both the emphasis in preschool education on the whole child as well as the types of benefits that research has indicated accrue from participation in high-quality preschool programs. However, due to time constraints, only existing data regularly collected divisionwide were used for this study. As such, questions about some conditions and outcomes (e.g., executive functioning, physical development, health and


11

http://www.fcps.edu/pla/ope/docs/fecep/sy1415_analyses.pdf

nutrition) were beyond the scope of the study. Table 4 provides specifics on the student outcomes analyzed, categorized into five areas (achievement, need for support services, participation in advanced academic opportunities, academic-related behaviors and habits, and aptitude)

Table 4 Description of Student Outcomes and Grades Analyzed in the Study

Area Assessment/Measure Grade(s)*

Achievement • DRA Word Analysis K • Developmental Reading Assessment (DRA2) 1,2 • Standards of Learning (SOL) mathematics and reading tests 3, 5, 8, 12 • Retention K, 1, 2, 3, 5, 8, 12 • High school grade point average 12 • High school graduation rates (federal and state definitions) 12

Need for support services

• ESOL K, 1, 2, 3, 5, 8, 12 • Special Education K, 1, 2, 3, 5, 8, 12

Participation in advanced academic opportunities

• Participation in Elementary Advanced Math 3, 5 • Advanced Academic Program (AAP) Eligibility 3, 5 • Young Scholars K, 1, 2, 3, 5 • Completion of Algebra I in middle school 8 • Completion of a course beyond Algebra 2 in high school 12 • Passing one Advanced Placement (AP) or International

Baccalaureate (IB) course in high school 12

• Passing three Advanced Placement (AP) or International Baccalaureate (IB) courses in high school

12

• Plans to attend 4-year institution (from Senior Survey) 12 • Verified enrollment in 4-year institution (Student Tracker

data) 12

Academic-related behaviors and habits

• Attendance K, 1, 2, 3, 5, 8, 12 • Elementary Progress Report Life, Work, and Citizenship

Ratings K, 1, 2, 3, 5

• VDOE-Reported Discipline Offenses 8, 12 Aptitude • Cognitive Abilities Test (CogAT), verbal and quantitative 2 *Grade 8 and 12 data were only analyzed for Study Question 1 as pre-kindergarten experience data was not available at these grade levels. For details on which cohorts were used for which analyses, please see Appendix D.

• For Questions 3 and 5 on costs and cost avoidance (ROI), the study used cost and funding information from the Policy Committee Annual Report 2015-16 for Early Head Start/FECEP, the FY 2016 Program Budget Book, and the FY 2016 Approved Budget. The Policy Committee Annual Report was used to calculate the expenditures associated with the FECEP program as well as the funding sources for FECEP. The FY 2016 Program Budget Book provides detailed information regarding how funds were spent (e.g., on staff salaries or operating expenses). The FY 2016 Approved Budget documents provided cost-per-student of Special Education (SPED) and English for Speakers of Other Languages


12

(ESOL) services that allowed calculation of costs above and beyond the cost to educate a general education student.

• For Question 4, information was collected from federal and state documents located online, as well as broader online searches of funding strategies in order to determine the range of approaches that can be used to fund pre-kindergarten programs. This information was supplemented by interviews with early childhood coordinators of comparison school divisions both within Virginia and across the U.S. to understand how other divisions face the challenge of funding pre-kindergarten programs. The Office of Program Evaluation (OPE) also consulted with staff from FCPS’ Office of Government Relations to understand the current status of state funding for preschool.

For further details about the study's methodology, see Appendix D.

Study Findings

This section summarizes the findings for each study question and provides a discussion of the evidence used to address the question. Both Study Questions 1 (FECEP comparisons) and 2 (Formal Pre-K versus No Formal Pre-K comparisons) are organized around the following four areas of student outcomes:

• Achievement • Need for support services • Participation in advanced academic opportunities • Academic-related behaviors and habits

Study Question 1. To what extent does participation in FECEP benefit students once they enter Kindergarten and progress through school?

Measure of Differences Between Comparison Groups

In the discussion of findings for this question, determination of advantages for attending or not attending a formal pre-k experience relies heavily on effect size analyses.7 Effect size (ES) is distinguished from a

7 Analyses were conducted with four cohorts of students (in Kindergarten and Grades 2, 3, and 5 during SY 2014-15) to provide a broad picture of available student outcomes across elementary school. In some cases, the performance of these students in prior

The FECEP program is associated with stronger reading outcomes for participating students upon Kindergarten entry and continued advantages in reading through the end of Grade 1. FECEP is also associated with stronger mathematics outcomes during later elementary and middle school and higher on-time graduation rates for participating students. In addition, FECEP is related to needing fewer Special Education services. Two years of FECEP were found to advantage participants even further. FECEP participants who had been in FECEP for two years had even better achievement, lower need for ESOL services, and higher levels of participation in some advanced high school coursework when compared to one-year participants.

Summary Finding


13

statistical significance test in that ES estimates the size of a difference (e.g., between participation in formal pre-k and non-participation), rather than simply determining from statistical significance if a difference exists. That is, while significance testing reflects the confidence with which one can state that there are differences between two groups, effect sizes quantify the size (or magnitude) of the difference between the two groups. Consequently, effect sizes reflect the meaningfulness of the difference between the groups, which was at the heart of the study question asked here (e.g., To what extent do students with and without FECEP differ once they enter Kindergarten and progress through school?). Or stated another way, the study question of interest here and in the other comparative questions about student outcomes imply a concern about magnitude (“to what extent…”) so effect size statistics were selected as the emphasis for reporting about student outcomes.

Effect sizes reported in this study represent the extent to which there are differences between the two groups being compared in each analysis. The study relied on a categorization of effect sizes supported by research on appropriate cutoffs for making meaning of observed effects in education and set these levels at 0.1 (small effect), 0.3 (medium effect), and 0.5 (large effect).8 The study considers effect size values falling below 0.1 as trivial and, therefore, not representative of meaningful differences between students who did and did not attend a formal pre-kindergarten experience. All other values should be considered meaningful, with the labels small, medium, and large used to differentiate the relative size of advantages demonstrated by the groups. Each group’s (Former FECEP, Matched/No Pre-K) observed pass rates or means on outcome(s) are also provided with the finding evidence.

Comparison Groups

The response to this question is organized around two sets of comparisons: • FECEP versus matched peers • One year versus two years of FECEP.

Matched Comparisons

Five cohorts of former FECEP students (those in Kindergarten and Grades 3, 5, 8, and 12 during SY 2014-15) were compared to demographically similar matched students who had not participated in FECEP. For the earliest grades investigated (Kindergarten, Grades 3 and 5), the matched students had not participated in a formal pre-kindergarten experience prior to entering Kindergarten (based on their

grades was also examined (i.e., longitudinal analysis). So for example, the Grade 1 and 2 reading performance of the SY 2014-15 Grade 3 cohort in SY 2012-13 and SY 2013-14, respectively, was also analyzed. See Appendix D for additional information on each of the grade-level cohorts included in the analyses.

8 The National Center for Special Education Research (NCSER) suggests that when it comes to interpreting effect sizes, Cohen’s (1988) traditional categories of small (0.2), medium (0.5), and large (0.8) are not always appropriate for research on education, particularly education intervention studies. Cohen’s cutoff points were derived from the range of effect sizes found in social and behavioral science research, but average effect sizes specifically in education research tend to be lower than these averages. Across a sample of 181 studies of achievement outcomes in education, NCSER found mean effect sizes of 0.40 for one-on-one interventions (e.g., tutoring), 0.26 for small group interventions, 0.18 for classroom-level interventions, and 0.10 for school-level interventions. Moreover, researchers from the National Survey of Student Engagement (NSSE) at Indiana University Bloomington analyzed effect sizes in the context of empirical data and found that few results fit within Cohen’s traditional cutoff points. Instead, they proposed alternative cutoffs of 0.1 (small effect), 0.3 (medium effect), and 0.5 (large effect). These suggestions are aligned with findings from NCSER regarding the average effect sizes among education research studies, allowing for a more meaningful interpretation of results.


14

parents completion of the Pre-Kindergarten Experience form). However, the availability of pre-kindergarten experience data for students at the middle and high school level was limited because FCPS did not collect pre-kindergarten experience data at the time these students entered elementary school. Thus, the only thing that is known about Grades 8 and 12 matched students’ pre-kindergarten experiences is that they did not attend the FECEP program.

This approach provided a statistically identical sample of students who differ only in their participation in FECEP. The matching technique provides a more rigorous examination of outcomes associated with the program than using intact comparison groups (see, for example, FECEP monitoring reports found at http://www.fcps.edu/pla/ope/docs/fecep/sy1415_analyses.pdf – which encourage comparison of FECEP results with the division-level FRM subgroup’s performance – or findings for Question 2 – which statistically equalized student characteristics [described more fully in Appendix D]). Stated another way, the analyses reported for this question were intended to isolate the effects of FCPS’ program on the types of students targeted by the program.

One-Year Versus Two-Years Comparisons

The dosage, or length of time in a program, also can indicate whether a program is having its intended impacts. Typically, unless there is a limit to the program’s ability to impact a participant, the greater the length of time in the program, the better the outcomes should be. Thus, in this study, outcomes for FECEP participants who had received two years of the program were compared with outcomes for FECEP participants who had received the program for one year. Students who had participated in FECEP for two years, starting as three-year-olds (many of whom had also taken part in Early Head Start) were compared to students who had participated in FECEP for one year, starting as four-year-olds. This type of analysis allows the intensity (one versus two years) to be considered as a measure of FECEP’s impact. One would expect that with greater intensity (i.e., two years of FECEP) students would demonstrate more benefits than with lesser intensity (i.e., one year of FECEP), especially given that prior research on Head Start programs has found such impacts.

FECEP Versus Matched Peers

Achievement

Finding 1: Students who participated in FECEP demonstrated stronger early reading skills upon Kindergarten entry than those who had not participated in a formal experience.

Based on analyses of three cohorts of students who entered FCPS Kindergarten during the last five years (SY 2009-10, SY 2012-13, SY 2014-15), students who had participated in FECEP prior to Kindergarten performed better on early reading skills when entering FCPS than a matched group of comparison students who had not participated in a formal experience. Figure 5’s depiction of the early reading skills performance of SY 2014-15 kindergarteners indicates that a large majority (80 percent) of former FECEP students met the benchmark for early reading skills upon Kindergarten entry. In contrast, 59 percent of matched students with no formal pre-k experience met the same benchmark. Thus, in SY 2014-15, former FECEP students demonstrated a 21 percentage point advantage in early reading skills upon entering Kindergarten. The effect size statistic, which indicates the size (or magnitude) of the difference across all three cohorts analyzed for the study, fell within the small range (ES = .13; see circle marker at Kindergarten entry in Figure 6). Thus, as students entered FCPS for Kindergarten, those who had participated in FECEP


15

http://www.fcps.edu/pla/ope/docs/fecep/sy1415_analyses.pdf

demonstrated, on average, stronger skills than similar students who had no formal pre-kindergarten experience.

Figure 5 Percent of Former FECEP and Matched Students Meeting the

Early Reading Benchmark Upon Kindergarten Entry, SY 2014-15

Figure 6

Average Effect Size Differences Between Former FECEP and Matched Students for Reading and Mathematics Achievement by Grade

Data Source: Office of Student Testing (OST), Fall DRA-WA.

Data Source: OST, Fall and Spring DRA-WA; Spring DRA2; SOL.


16

Finding 2: The reading skill differences that favored former FECEP students when entering school was evident through the end of Grade 1.

The early advantage in reading skills demonstrated by former FECEP students as they entered kindergarten remained evident at the end of Kindergarten (DRA Word Analysis assessment) and through Grade 1 (DRA2 assessment), but was no longer evident in SOL assessment results in later elementary grades. Figure 7 shows the pass rates for students in different grades on reading assessments taken at the end of the year during elementary school. As shown in Figure 6, the size of these differences indicate a similar small effect at the end of the Kindergarten year through the end of Grade 1. Thus, after the first two years in school, differences between students who had and had not attended FECEP narrowed and continued to be negligible from Grade 2 through Grade 8.

Figure 7 Percent of Former FECEP and Matched Students

Who Met Reading and Mathematics Benchmarks by Grade

In comparison to reading, mathematics performance differences between FECEP and matched students were evident in later grades, though small. More specifically, as seen in Figure 7, former FECEP students in Grades 3, 5, and 8 met mathematics benchmarks (i.e., passed the SOL mathematics test) consistently more often (by approximately 5 percentage points at each grade) than matched students. Figure 6 shows that the size of this difference was considered a small advantage for former FECEP students at all grade levels.

Finding 3: FECEP program participation had a small effect in on-time high school graduation.

Former FECEP students demonstrated higher rates of on-time high school graduation than matched students, using either the federal or state definitions (see Figure 8). As shown in Figure 9, the size of the difference between students who had taken part in FECEP and those who had not was categorized as a small effect (ES = .11 for federal definition; ES = .12 for state definition). Thus, when matching former FECEP participants to demographically similar students in Grade 12, FECEP students were more likely to have taken less time to graduate from high school than matched students who had not attended.

Reading Mathematics

Data Source: OST, Spring DRA-WA; Spring DRA2; SOL.


17

Figure 8 On-Time Graduation Rates for

Former FECEP and Matched Students, SY 2014-15

Figure 9 Effect Size Differences Between

Former FECEP and Matched Students for On-Time Graduation Rates, SY 2014-15

Data Source: DIT, On-time Graduation file.

Data Source: DIT, On-time Graduation file.


18

Need for Support Services

Finding 4: Former FECEP students required fewer Special Education services in elementary and middle school.

As shown in Figure 10, in the early elementary grades, differences in need for Special Education services did not surface between former FECEP and matched peers. However, at Grade 3, rates among former FECEP students begin to be lower and at Grade 5, significantly fewer former FECEP students were in need of Special Education services when compared to similar peers who had not attended formal pre-kindergarten. In Grade 8, the difference was even larger (and, again, significant). Figure 11 highlights that the reduced Special Education needs among former FECEP students at both Grades 5 and 8 amounted to a small effect (ES = .12 and ES = .18, respectively). At Grade 12, the difference in need for Special Education was approaching zero (ES = .07), indicating it was not meaningful.

Figure 10 Percent of Former FECEP and Matched Students Needing Support Services by Grade

Also in Figure 10, the need for ESOL services did not differ between former FECEP and the matched student groups at any of the grade levels (1 through 12) investigated for this study. At Grade 1, 78 percent of both groups required ESOL services with percentages of need for ESOL only slightly lower at Grades 2 and 3. As depicted in Figure 10, at both Grades 5 and 8, the need for ESOL services for the former FECEP group was slightly lower than that for the matched students but this difference was not statistically significant, meaning that the rate for both groups could not be reliably distinguished from each other. By Grade 12, the rates had fallen to an equivalent 6 percent for both groups. Figure 11 also reflects the similarity between the two groups, with effect sizes hovering close to zero at all grades, indicating no meaningful effect on the need for support services from FECEP participation.

ESOL Special Education

Data Source: DIT, Student Membership.


19

Figure 11

Average Effect Size Differences Between Former FECEP and Matched Students in Need for Support Services by Grade

Participation in Advanced Academic Opportunities

Finding 5: Participation in FECEP was not related reliably to more advanced academic opportunities during elementary school.

Starting in Grade 3, FCPS students have the opportunity to participate in Advanced Mathematics (the school division’s offering of extended and above grade-level mathematics standards) and the Advanced Academic Program ([AAP] the school division’s offering designed to meet the needs of advanced learners with a strong emphasis on higher-level thinking, problem-solving, and decision-making). Academic opportunities afforded to students favored former FECEP students at Grade 5, which may reflect that year’s cohort more than a specific impact of FECEP. As depicted in Figure 12, former FECEP students in the Grade 5 cohort participated in Advanced Math and were eligible for AAP participation at significantly higher rates than matched students. Figure 13, which describes student participation in advanced opportunities at the middle high school level, indicates equivalent percentages of former FECEP and matched students participating in advanced coursework. Thus, only the Grade 5 former FECEP cohort demonstrated higher percentages of advanced opportunities than the matched student group. The magnitude of the participation difference favoring former FECEP students was small (ES = .19 for Advanced Math, ES = .14 for AAP eligibility) at Grade 5, with all other effect sizes hovering around zero and, therefore, not meaningful. With only one cohort showing an advantage, no reliable differences could be presumed from FECEP participation.



20

Figure 12 Percent of Former FECEP and Matched Students Participating

in Advanced Academic Opportunities During Elementary School by Grade

Figure 13 Percent of Former FECEP and Matched Students Participating

in Advanced Coursework During Middle and High School

AAP Eligibility (Counts)

Former FECEP Matched Students

Grade 3 692 692Grade 5 598 578

Advanced Math Advanced Academic Program


Data Source: OST, SAG Algebra by 8th grade; SAG Passing Course above Algebra II; SAG Passing AP/IB Courses.


21

Figure 14 Effect Size Differences Between Former FECEP and Matched Students’

Participation in Advanced Opportunities and Coursework

Academic-Related Behaviors and Habits

Finding 6: FECEP participation was not associated with reliable differences in academic-related behaviors and habits.

Figure 15 shows that observed attendance rates for former FECEP students were slightly higher at most elementary grades studied (i.e., Kindergarten, Grades 1, 3, and 5) compared to the matched sample of non-preschool peers. Progress report ratings of elementary student work habits for the former FECEP students group, though varying somewhat between former FECEP and matched students in Grade 5, were considered statistically equivalent at all other grade levels. The higher performance demonstrated at Grade 5 may be a reflection more of that year’s cohort than the FECEP program. As shown in Figure 16, the higher attendance rates of FECEP students was categorized as not meaningful (i.e., close to zero) at all elementary grades. Taken together, the attendance results in Figures 15 and 16 mean that, while FCPS can be confident that attendance advantages will continue to exist for former FECEP students, the magnitude of the difference is negligible. In contrast, the differences in work habits of students found at Grade 5, which could not be confirmed as reliable for former FECEP students, reflected a small effect (ES = .33) that would be meaningful if it could be confirmed.

Similarly, Figures 15 and 16 show that in middle and high school (Grades 8 and 12) attendance did not differ meaningfully between the two groups. And, while matched students committed a slightly higher average number of VDOE-reported discipline offenses during middle and high school (Figure 15), Figure 16 shows that the difference in offenses was not meaningful (i.e., close to zero).

Elementary School Middle and High School

Data Sources: DIT, Student Membership. Data Source: OST, SAG Algebra by 8th grade; SAG Passing Course above Algebra II; SAG Passing AP/IB Courses.


22

Figure 15 Former FECEP and Matched Students’ Academic-Related Behaviors and Habits by Grade

Attendance Work Habit Ratings

VDOE-Reported Discipline Offenses

Data Source: DIT, Student Attendance; Elementary Progress Report; VDOE Discipline.


23

Figure 16 Average Effect Size Differences Between

Former FECEP and Matched Students in Academic-Related Behaviors and Habits

One Year Versus Two Years of FECEP

Finding 7: Attending the FECEP program for two years, rather than one, was associated with higher achievement, lower need for ESOL services, and greater participation in advanced coursework during high school.

In looking across the many outcomes that were investigated for this study, three areas yielded at least one outcome that benefitted from participation in FECEP for two years rather than only one year. That is, students who had begun FECEP as three-year-olds (two years of FECEP) demonstrated better outcomes than students who had begun FECEP as four-year-olds (one year of FECEP) in the areas of achievement, need for support services, and participation in advanced opportunities. No meaningful differences were found between one year and two year participants in the area of behavior and work habits.

The strongest differences between one-year and two-year FECEP participants came from analyses of need for ESOL services. As shown in Figure 17, analyses indicated medium to large effects in most grades analyzed for the study on need for ESOL. The medium to large differences favored students who had participated in FECEP for two years over those who received one year of the program. Thus, for example, at Grade 2, when the largest difference in need for ESOL appeared, 82 percent of students who had FECEP for one year required ESOL support, while 50 percent of students who had FECEP for two years required such support (see Appendix J for percentages of students with one and two years of FECEP requiring ESOL services at each grade level).

Data Source: DIT, Student Attendance; Elementary Progress Report; VDOE Discipline.


24

Further, when former FECEP students first entered Kindergarten, the difference in early reading skills between participants who had spent two years in the program and those who had one year in FECEP reflected a small, almost moderate, effect (ES = .29) favoring students who had been in the program for two years. The small effect translates into an increase of nine points in the percentage of two-year participants meeting the Kindergarten entry benchmark than what was seen among the one-year participant group (88 percent versus 79 percent, respectively). The advantage for two-year participants was also evident at the end of the Kindergarten year, where a relatively similar small effect surfaced (ES = .23). Subsequent grades (other than Grade 5), saw a similar fading of the advantage for two-year participants as students moved into later grades. Not shown in Figure 17, analyses also uncovered a small effect in on-time graduation rates (ES = .20 for State definition; ES = .12 for federal definition) favoring two-year FECEP participants.

Small differences that favored two-year participation in FECEP were also found in the percent of students who participated in advanced coursework during high school. More specifically, students with two years of FECEP participation were more likely to take a mathematics course beyond Algebra 2 (ES = .22) and pass three or more AP/IB courses (ES = .15). Similar advantages in advanced opportunities were not found in elementary and middle school.

Lastly, most comparisons of behaviors and work habits showed no difference between students with one and two years of FECEP participation. Attendance at all grades examined showed no differences, as did disciplinary offenses in middle and high school. The only differences to emerge from the analyses indicated small and medium advantages for students with two years of FECEP in work habit ratings at Grades 1 and 5, respectively (see Figure 17). While two other grades are just below the cutoff for a small effect, the findings make it difficult to support a consistent advantage for students who had participated in FECEP for two years. See Appendix J for greater details on all analyses comparing students with one and two years of FECEP.


25

Figure 17 Average Effect Size Differences Between

Former FECEP Students with One and Two Years of Participation

Achievement Need for Support Services

ES Advanced Opportunities MS/HS Advanced Opportunities

Behaviors/Work Habits Data Source: OST, Fall and Spring DRA-WA; Spring DRA2; SOL. Data Source: DIT, Student Membership. Data Source: OST, SAG Algebra by 8th grade; SAG Passing Course above Algebra II; SAG Passing AP/IB Courses. Data Source: DIT, Student Attendance; Elementary Progress Report; VDOE Discipline.


26

Study Question 2. To what extent do students with and without formal pre-kindergarten differ in educational outcomes once they enter Kindergarten and progress through school?

Measure of Differences Between Comparison Groups

While the earlier discussion focused on the impacts of FECEP participation, the following discussion tracks the broad impacts of formal versus no formal pre-kindergarten experiences. The formal pre-k experiences are those described in the introduction to the report and generally include any student who attended an organized experience outside the home, whether childcare or preschool. Readers should note that observed values reported in the findings for this question represent those actually demonstrated by the groups and, thus, do not adjust for demographic differences between the groups. Furthermore, they represent one cohort’s data (i.e., a specific grade level in a specific year), rather than all cohort’s used to compute average effect sizes. Thus, a comparison of the observed means for the two groups will not reflect the true estimated effect of preschool attendance described by the effect sizes since the observed values include outcome differences based on who is in the groups, not just the pre-kindergarten experience in which the students participated, and do not represent multiple cohorts (when available for analysis). For example, it is well-established that without intervention students from low-income families typically perform less well on early reading and school readiness skills than students from middle- and upper-income families. Given that the percentage of students from low-income families in the No Formal Pre-K group is much higher (60 percent in SY 2015-16) than the percentage in the Formal Pre-K group (24 percent in SY 2015-16), differences in the observed pass rates on the reading benchmarks could be due solely to the demographics of the two groups, rather than participation in a pre-kindergarten experience. Therefore, the effect sizes on which the findings are based, which do make the adjustment for the demographic differences, and do so for multiple cohorts, as available, are the best representation of the impact of pre-kindergarten on students. Determination of advantages in the discussion below relies heavily on analyses that included both significance testing and effect sizes, just as was done for Question 1. Please see Appendix D for more information on the procedures and rationale for the methodology used to answer Question 2.

Overall, students who participated in a formal pre-kindergarten experience demonstrated advantages over students who had no formal pre-kindergarten experience (controlling for demographics of the two groups). Achievement results found stronger early reading skills upon Kindergarten entry and continued stronger reading skills through the end of Grade 2 among students with formal pre-kindergarten experience, the period during which students are typically learning to read. Starting in Grade 3, however, this advantage was no longer evident, meaning students who had and had not participated in formal pre-kindergarten demonstrated similar reading skills in later elementary school. Students with formal pre-kindergarten experience also required fewer ESOL services throughout their FCPS schooling and had higher participation rates in advanced academic opportunities during elementary school. Differences in academic-related behaviors and habits did not differ meaningfully based on pre-kindergarten experience.

Summary Finding


27

Achievement

Finding 8: A reading achievement advantage for students who had participated in formal pre-k was highest upon Kindergarten entry and diminished afterwards.

Figure 18’s depiction of percentage of SY 2014-15 kindergarteners indicates that 91 percent of students who had participated in a formal experience met the benchmark for early reading skills upon Kindergarten entry, while 64 percent of students with no formal pre-k experience met the benchmark, yielding a 27 percentage point advantage among students who had participated in a formal pre-k experience. However, some of this advantage is due to the lower numbers of English learners and students living in poverty among those with formal pre-kindergarten experience. The effect size statistic, which is based on an adjusted (estimated) mean, statistically controls for these differences. Even after statistically equalizing the groups, the Formal Pre-K group demonstrates an advantage in meeting the early reading skills benchmark, as captured by the medium effect size average of .44 (see Figure 19). Thus, just as seen among former FECEP students, as students enter FCPS for Kindergarten, those who have had formal experiences outside the home demonstrate, on average, stronger skills than those who have not had a formal pre-kindergarten experience even after statistically equalizing the demographic characteristics of the students in the two experience groups.

Figure 18 Percentage of Incoming Kindergartners Meeting the

Early Reading Benchmark Upon Entry, By Pre-K Experience

The early advantage in reading skills demonstrated by students entering FCPS with formal pre-kindergarten experience remained evident at the end of Kindergarten (DRA Word Analysis assessment) and through Grades 1 and 2 (DRA2 assessment), but was no longer evident in SOL assessment results in later elementary grades. Figure 7 shows the pass rates for students in different grades on reading assessments taken at the end of the year during elementary school. The size of these differences, after statistically controlling for variations in demographics of students within each group, indicate a considerable drop in the size of the effect (from medium to small) at the end of the Kindergarten year through Grade 2. More specifically, as shown in Figure 6 (which depicts the size of the effects at each

Data Source: OST, Fall DRA-WA.


28

grade) by the end of Kindergarten the size of the difference is small, and it continues to decrease in each subsequent grade until it is almost zero at the end of Grade 8 (circle markers on graph). Thus, after the first year in school, differences between students who had and had not participated in a formal pre-kindergarten experience narrowed and continued to decrease each year thereafter to virtually no difference by the end of Grade 5.

Figure 19 Average Effect Size Differences* Between Formal Pre-K and No Formal Pre-K

for Reading and Mathematics Achievement

As shown by the triangle markers in Figure 19, analyses of mathematics performance that controlled for student characteristics in the pre-kindergarten experience groups found a small effect (difference) at Grade 3 (ES = .15) and no meaningful difference at Grade 5 (ES = .05) based on the pre-kindergarten experience of students, once the groups were statistically equalized for the types of students in each group. Figure 20 also shows the observed percentages of students meeting mathematics benchmarks, which does not control for demographic differences between the two groups. Further details about the SOL reading and mathematics performance of students by Kindergarten experience are available in Appendix J.

Data Source: DIT, Fall and Spring DRA-WA; Spring DRA2; SOL.


29

Figure 20 Percent of Students Who Met End-of-Grade Reading and Mathematics Benchmarks,

By Pre-Kindergarten Experience and Grade*

Need for Support Services

Finding 9: Throughout elementary school, students who had attended a formal pre-kindergarten experience required fewer ESOL services than students with no formal pre-kindergarten experience.

Beginning in Kindergarten and continuing throughout elementary school, students who had formal pre-kindergarten experiences required fewer ESOL services than students who had not, even after controlling for differences between the two groups. Figure 21 shows the actual percentages of students with and without pre-k requiring ESOL services at various grades, not accounting for demographic differences. Among students who had participated in a pre-kindergarten experience, the need for ESOL services ranged from 28 percent decreasing to 9 percent by Grade 5. In contrast, among students with no formal pre-kindergarten experience, the need for ESOL remained at a steady 69 percent from Kindergarten through Grade 3, declining to 28 percent for Grade 5. Not only is the actual need for ESOL services among the No Formal Pre-K students more than twice that of the other experiences combined, but also Figure 21 indicates no movement of students exiting ESOL until Grade 5. While those are the actual percentages that make no adjustment for the differences between the two groups, as shown in Figure 22 (circle markers), even after statistically equalizing the groups, the advantage favoring students with formal pre-kindergarten was evident in all grades examined and strongest from Kindergarten to Grade 3. More specifically, after equalizing characteristics (FRM, SWD) of students in the two groups, the differences are large in size from Kindergarten to Grade 3 and small at Grade 5 in favor of participation in formal pre-kindergarten.

Reading Mathematics

Data Source: OST, Spring DRA-WA; Spring DRA2;


30

Figure 21 Percent of Students Needing ESOL and Special Education Services,

by Pre-Kindergarten Experience and Grade*

Figure 22 Average Effect Size Differences* Between Formal Pre-K and No Formal Pre-K

for ESOL and Special Education Services

ESOL Special Education

Data Source: DIT, Student membership.

Data Source: DIT, Student membership. Fairfax County Public Schools, Office of Program Evaluation May 2016

31

Results about differences between the Formal Pre-K and No Formal Pre-K groups in their need for Special Education was quite different than what was found for ESOL services. Students with formal pre-kindergarten experience were found to require Special Education services more frequently than students with no formal experience. This increase in need for Special Education services among students in the Formal Pre-K group may reflect countywide efforts to provide early intervention services to all students with disabilities. Figure 21 shows the actual percentages of students in the two groups requiring Special Education services. After statistically equating the groups for the types of students in each, the size of the differences favored the No Formal Pre-K group at every grade level. However, almost all students with disabilities (999 versus 15) at the start of school were in the Formal Pre-K group – probably due to ChildFind efforts to provide early intervention services to students with disabilities. The preponderance of students with disabilities in one group with so few in the other group is probably a better explanation of the differences favoring the No Formal Pre-K group, since statistical controls to equalize the characteristics of students in the two groups would have been less effective in this analysis. Nonetheless, the size of the differences, as seen in Figure 22 (triangle markers) were considered large in Kindergarten (ES = .56), and small in Grade 1 (ES = .28), Grade 2 (ES = .27), Grade 3 (ES = .26), and Grade 5 (ES = .14). Thus, over time, the two groups became more similar in their need for Special Education services. Further details about the need for support services associated with different types of pre-kindergarten experience are available in Appendix J.

Participation in Advanced Academic Opportunities

Finding 10: Students who participated in formal pre-kindergarten experiences received more advanced academic opportunities than students without formal pre-kindergarten.

Starting in Grade 3, FCPS students have the opportunity to participate in Advanced Mathematics (the school division’s offering of extended and above grade-level mathematics standards) and the Advanced Academic Program (the school division’s offering designed to meet the needs of advanced learners with a strong emphasis on higher-level thinking, problem-solving, and decision-making). Figure 23 shows the percentages of students who participated in each of these programs based on their pre-kindergarten experience group. Both Advanced Math and Advanced Academic Program (AAP) participation rates were higher for the Formal Pre-K group (yellow columns) than the No Formal Pre-K group in Grades 3 and 5 when not equalizing the characteristics of the two groups. Effect size analyses statistically equalizing the two groups, indicated that the differences between the groups were meaningful (small) for participation in both advanced opportunities at Grades 3 and 5 (see Figure 24). Further details about advanced academic opportunities associated with different types of pre-kindergarten experience are available in Appendix J.


32

Figure 23 Percent of Students Participating in Advanced Opportunities,

by Pre-Kindergarten Experience and Grade, SY 2014-15

Figure 24

Average Effect Size Differences* Between Formal Pre-K and No Formal Pre-K for Participation in Advanced Academic Opportunities

Advanced Math Advanced Academic Program




33

Academic-Related Behaviors and Habits

Finding 11: Participation in a formal pre-kindergarten experience did not advantage students in attendance rates or progress report ratings of work habits.

As shown in Figure 25, the observed attendance rates and ratings of student work habits assigned by teachers on the elementary progress reports were consistently higher among students with formal pre-k experience than no formal experience. For example, at Kindergarten, the Formal Pre-K group had an average attendance rate of 96 percent while the No Formal Pre-K group’s attendance rate was 94 percent. Similarly, work habit ratings for the Formal Pre-K group were consistently one-tenth of a percentage point higher than for the No Formal Pre-K group (3.5 versus 3.4 at Kindergarten, 3.4 versus 3.3 at Grade 2). Although most of these differences were statistically significant, the magnitude of the difference once characteristics of the groups were controlled statistically, indicated negligible differences between the two groups of students. As shown in Figure 26, the effect of the attendance and work habit differences fell in the trivial range, indicating no meaningful differences in these outcomes related to whether students had or had not attended a formal pre-k experience. Thus, after statistically controlling for demographics of the two groups, both groups should be considered equivalent in academic-related behaviors and habits. Further details about academic-related behaviors and habits associated with different types of pre-kindergarten experience are available in Appendix J.

Figure 25 Attendance Rates and Work Habit Ratings by Pre-Kindergarten Experience and Grade, SY 2014-15

Attendance Work Habit Ratings

Data Source: DIT, Student Attendance; Elementary Progress Report.


34

Figure 26 Effect Size Differences Between Formal Pre-K and No Formal Pre-K

for Attendance Rates and Work Habit Ratings, SY 2014-15

Study Question 3. What program costs and funding sources are associated with FECEP, FCPS’ preschool offering?

Finding 12: The total direct cost to operate the FECEP program was $22.8 million in FY 2016, approximately $9.7 million of which was funded by FCPS.

The FECEP program cost approximately $22.8 million dollars to operate in FY 2016.9 FCPS funded approximately $9.7 million dollars (43 percent) of these costs through its School Operating Fund. Remaining costs were funded from federal (Head Start, Title I), state (VPI, VPI+), and county contributions

9 Excluding Early Head Start, transportation services, and central overhead costs (e.g., indirect costs such as instructional support, facilities management, general support, and central administration costs).

The direct cost to operate the FECEP program was $22.8 million dollars in FY 2016, approximately $9.7 million of which was funded by FCPS’ School Operating Fund (including Project Momentum). The remaining $13.1 million to fund the program came from federal (Head Start, Title I), state (VPI, VPI+), and county contributions. With budgeting for 1,680 student placements in FY 2016, the cost-per-pupil of FECEP totaled $13,598.

Summary Finding

Data Source: DIT, Student Attendance; Elementary Progress Report.


35

(see Table 5). During SY 2015-16 (FY 2016), the FECEP budget provided funding for 1,680 student placements10 (excluding Early Head Start11), resulting in a direct per student cost of $13,598 for FECEP in FY 2016 (see Figure 27).12

Table 5 FECEP Funding Sources, FY 201613

Source Type FY2016 % of FECEP Funding

FCPS VPI Match $7,959,645 35% State VPI $4,044,000 18% County FECEP Contribution $3,943,407 17% Federal Head Start $1,536,984 7% Federal USDA Food Program $1,310,669 6% Federal Title I $1,303,594 6% FCPS Head Start Supplement $968,237 4% State VPI+ $929,687 4% FCPS Project Momentum (PSI) $765,765 3% County Head Start Supplement $82,699

As shown in Table 6, the FECEP program budget (which includes Early Head Start costs), was $21.0 million in FY 2016 (calculated by summing the $10.8 million “Work for Others” expenditure credit with the $10.2 “Expenditures”). Approximately $9 million of those dollars came directly from the FCPS School Operating Fund. This calculation does not include Project Momentum funds, which contribute an additional $766,000 from the School Operating Fund toward FECEP (reaching the $9.7 million in costs to FCPS, reported above). Moreover, Table 5 shows that the majority of the program budget—approximately 82 percent—is used toward school-based staff salaries and benefits. Only 8 percent of the program budget is spent on operating expenses.

Funds from outside the division (e.g., those not included in the FCPS School Operating Fund) flow through Fairfax County. This means that these funds appear as an expenditure credit (labeled “Work for Others” in Table 6) due to their allocation to FCPS from the county as credit instead of cash. The “Work for Others” expenditure credit includes funds such as federal Head Start, VPI dollars from the state, and the Fairfax County local contribution (see Table 5 for a detailed breakdown of all funding streams). Additionally, the “Offsetting Revenue” line includes the federal government’s United States Department of Agriculture (USDA) Child and Adult Care Food Program, which further reduces the net cost of the program for FCPS. Therefore, the “School Operating Fund Net Cost” line of the FECEP/Head Start/Early Head Start program budget includes only the costs paid from the FCPS School Operating Fund and not the total cost of operating the program (also excludes Project Momentum funding for FECEP, which is also paid from the FCPS School Operating Fund but captured under the Project Momentum program budget).


37

Table 6 FECEP/Head Start/Early Head Start Program Budget, FY 2015 and FY 2016

FY 2015 Budget FY 2016 Budget

School-Based Nonschool-Based School-Based Nonschool-Based Cost FTE Cost FTE Cost FTE Cost FTE

Administrator $0 0.0 $0 0.0 $0 0.0 $0 0.0

Specialist $2,174,155 31.5 $658,448 10.0 $2,149,087 30.5 $669,653 10.0

Teacher $6,556,307 99.0 $0 0.0 $6,795,649 100.0 $0 0.0

Assistant $2,618,194 94.0 $0 0.0 $2,693,738 94.0 $0 0.0

Office $450,922 10.0 $506,377 10.5 $445,791 10.0 $529,637 10.5

Custodial $0 0.0 $0 0.0 $0 0.0 $0 0.0

Salary Adjustments $0 0.0 $0 0.0 $0 0.0 $0 0.0

Hourly Salaries $399,484 0.0 $0 0.0 $402,068 0.0 $0 0.0

Work for Others ($10,463,064) 0.0 0 0.0 ($10,837,630) 0.0 0 0.0

Employee Benefits $5,215,554 0.0 $457,814 0.0 $5,126,849 0.0 $502,347 0.0

Operating Expenses $1,591,165 0.0 $0 0.0 $1,688,167 0.0 $0 0.0

TOTAL $8,542,717 234.5 $1,622,639 20.5 $8,463,719 234.5 $1,701,637 20.5

84.0% 92.0% 16.0% 8.0% 83.3% 92.0% 16.7% 8.0%

Total Positions 255.0 255.0

Expenditures $10,165,356 $10,165,356

Offsetting Revenue $1,135,780 $1,135,780

Offsetting Grant Funding $0 $0

School Operating Fund Net Cost $9,029,576 $9,029,576

# of Sites 68 65

# Served 1,643 1,752

Data Source: Financial Services (FS), FY 2016 Approved Budget Book, Fairfax County Public Schools, 2016.


38

Study Question 4. To what extent is FCPS’ approach to funding pre-kindergarten programs similar to other jurisdictions?

Finding 13: FCPS “braids”16 funding sources to provide a unified preschool program, unlike many divisions that administer multiple types of classrooms, each reflecting the primary funding source.

Unlike many school divisions in Virginia and across the United States, FCPS combines (or “braids”) its three main funding sources – federal, state, and local funds – together to provide unified preschool services to students. In other words, instead of offering distinct classrooms based on the primary funding source—for example, a Head Start classroom separate from a VPI classroom—students from all funding sources are integrated together. FCPS currently operates 99 FECEP classrooms across 65 sites, all containing a mix of students funded via federal, state, and local dollars. Conversely, Prince William Public Schools operates 17 Head Start classrooms, two VPI classrooms, and eight VPI+ classrooms. Likewise, Chesterfield County Public Schools also separates classrooms by funding and operates 11 Head Start classrooms, seven Title I classrooms, seven VPI classrooms, and nine VPI+ classrooms.17 See Appendix M for further detail regarding the different types of funding sources used to fund FCPS’ FECEP program and other programs at school divisions across Virginia.

The primary advantage of braiding funds is that it allows all students attending a pre-kindergarten program provided by FCPS to receive the same service, putting all students on a level playing field. When funds are braided, every classroom must operate under the most stringent funding guidelines, which regulate program features such as curriculum, teaching staff hiring criteria, and special services offered (e.g., health or dental services). As a result, every classroom is held not only to the highest standard of quality but also to the same standard of quality. Further, braiding funds allows for more students to receive valuable services (such as dental screenings) regardless of whether they are funded under a stream that mandates these additional components. Therefore, while FCPS does not fund its early childhood program in the same way as most other school divisions, the FCPS approach should yield both a consistent and high-quality experience for all participating children.

16 “Braiding” is a funding and resource allocation strategy that taps into existing categorical funding streams and uses them to support unified initiatives in as flexible and integrated a manner as possible. 17 Information was gathered from phone Interviews with early childhood coordinators at school divisions throughout Virginia. Identifying information has been redacted in this report.

Unlike other Virginia school divisions that create separate classrooms based on the funding stream, FCPS blends its main funding sources –federal, state, and local – to provide unified pre-kindergarten services to FECEP students. Similar to other school divisions in Virginia, FCPS is unable take advantage of a portion of the state funding because of a local match requirement and a lack of alignment between Virginia’s estimated cost-per-student and the actual costs to FCPS. This leaves 40 percent of available VPI-funding unused each year and contributes to the underserving of eligible students.

Summary Finding


39

Finding 14: Similar to many divisions across Virginia, FCPS is unable to take full advantage of state preschool funding.

Many divisions in Virginia are unable to accept all VPI funds offered from the state due to an inability to meet the local match requirement. Although FCPS accepts (and matches) roughly 60 percent of the seats allocated and partially funded by VPI, the division ranks second behind Prince William County in terms of the total number of VPI-funded seats it does not accept. In other words, VPI allocates partial funding for over 2,600 pre-kindergarten seats in FCPS, but the division only accepts funding for 1,500 seats—leaving 40 percent of available VPI funding unused every year.18 The primary challenge to accepting the entire amount of available state funding is finding dollars to meet the local match requirement (and exceed it, as is necessary for FCPS in which the cost-per-student of FECEP is more than double the cost-per-student estimated by the state). Unfortunately, a bill calling for the elimination of the required match for VPI funding was put on hold in March 2016.19

This is particularly unfortunate when considered in terms of the number of eligible children who are served by FECEP in comparison to those unserved. Across the 2014-15 school year, FCPS served 1,711 pre-kindergarten students via a process of ongoing enrollments (e.g., filling a seat with a student from the waiting list when a seat becomes vacant, meaning more students are served across the year than there are seats available). At year’s end, there were still 262 four-year-olds and 39 three-year-olds on the waiting list, meaning FCPS is unable to serve all eligible students every school year. Further, waitlist numbers alone are likely an underestimate of FECEP’s reach in serving all eligible students, because it does not take into account students who were income-eligible but did not apply. Many studied divisions reflected a belief that large pockets of students exist who do not apply for pre-kindergarten services (despite being income-eligible) because of a lack of awareness about the opportunities, challenges related to transportation or access to school sites, or a lack of wrap-around childcare services.20 Additionally, waitlist numbers alone do not take into account three-year-old students who not put on a waiting list due to the few number of seats available to students younger than four years of age.

18 Virginia Preschool Initiative (VPI): Program Information, Presentation to the Joint Subcommittee on the Virginia Preschool Initiative. (2015, Aug. 25). Virginia Department of Education. Retrieved from http://hac.virginia.gov/subcommittee/Jt_Preschool_Initiative_Sub/8-25-15/No5_VPI.pdf 19 National Conference of State Legislatures, NCSL Early Care and Education Database, http://www.ncsl.org/research/human-services/child-care-and-early-education-legislation-databas.aspx 20 Information was gathered from phone Interviews with early childhood coordinators at school divisions throughout Virginia. Identifying information has been redacted in this report


40

http://hac.virginia.gov/subcommittee/Jt_Preschool_Initiative_Sub/8-25-15/No5_VPI.pdfhttp://www.ncsl.org/research/human-services/child-care-and-early-education-legislation-databas.aspxhttp://www.nc

Documents

Study of the Impact of Pre-Kindergarten Experiences on ......consecutive years (SY 2011-12 through SY 2013-14) of 4percent increases in the number of students served (Barnett, Carolan,