1

Effects of Race to the Top on Students’ Science Achievement and Closing the

achievement Gap in Science

Kitchka Petrova, kp13b@my.fsu.edu

Patrice Iatarola, piatarola@fsu.edu

Anastasia Semykina asemykina@fsu.edu

Florida State University

Paper prepared for the AEFP 42nd Conference

Washington, DC

March 18th, 2017

This working paper may not be cited or distributed without explicit permission from the authors

2

Introduction

Preparing K-12 students to have strong background in Science Technology, Engineering

and Mathematics (STEM) and inspiring them to pursue STEM careers defined President

Obama’s administration’s efforts to address the issues with K-12 students’ science proficiency

and the existing science achievement gap (Holdren, Lander & Varmus, 2010). RTTT was one of

the educational initiatives of the US Department of Education that was designed to stimulate

innovations in education and improve the educational outcomes of US students by incentivizing

states to make changes in four areas: adopting rigorous standards, building longitudinal state data

systems, ensuring schools have effective teachers and principals and turning around the lowest

achieving schools. The states were encouraged to include STEM component in their applications

and all states that received RTTT funds had such components. The states had to present sound

plans, geared towards offering rigorous studies in STEM, cooperating with industry, universities

and informal science education establishments, improving STEM teaching, preparing more

students for advanced careers in STEM, and addressing the needs of underrepresented groups

(U.S. Department of Education, 2009). RTTT also coincided with other recommendations of the

federal government such as “Educate to Innovate” Campaign for Excellence in Science,

Technology & Math (STEM) Education that aspired to improve STEM education within the next

ten years (The White House, 2009) and the release of the President Obama’s Council of Advisors

on Science and Technology report “Prepare and Inspire: K-12 education in STEM for America’s

future” that contained specific recommendations for improvements by preparing students to have

strong background in STEM and inspiring them to pursue STEM careers (Holdren et al., 2010).

The last report also addressed the existing achievement gap in science and its consequences for

minority and students from low socioeconomic background who would not have the

opportunities to pursue careers in STEM fields that are well paid and highly respected.

3

Furthermore, the report specifically highlighted that due to the lack of minority and women in

STEM fields, these underrepresented groups cannot contribute with their innovative ideas to the

development of STEM fields (Holdren et al., 2010).

These recent events demonstrate how much interest is there to improve STEM education and

one would expect discussions on a national level about the role of initiatives such as RTTT in the

process, because RTTT also targeted improvement in STEM. However, such discussion has been

absent in the education policy research community, a fact that is puzzling considering the goals

of RTTT.

Therefore, investigating how large investments in education such as RTTT contribute to

improving STEM outcomes for students and science education is needed. Our study addresses the

following research questions:

1. Did RTTT improve science achievement?

2. Did subgroups of students, defined by race/ethnicity, English language learner,

disability or income status experience different outcomes of RTTT in terms of science

achievement?

This study reports on the effectiveness of RTTT in improving students’ science

achievement. By evaluating the RTTT effects on science education this study fills the gap in the

literature if large scale education investments, based on a competitive selection process help

improve science education. It also sheds light on how successful RTTT was to decrease the

achievement gap in science for minority, students with disabilities, ELL and students with low

socioeconomic status. It also demonstrates if K-12 public education, when supported with funds

4

is contributing to meeting the National Science Foundation’ s strategic goal to “support

development of a diverse STEM workforce” (National Science Foundation, 2014).

Literature Review

RTTT and student outcomes. Due to the recent end of RTTT program there is a dearth of

peer review articles discussing its effects on student achievement. Scholars have investigated the

political climate of implementing the RTTT policies (Howell, 2015, McGuinn, 2011) or the

legality of the program (Barnes, 2011,). Others have examined issues related to the application

and implementation processes of the program (Scott, 2011) and the awardee states’ capacity to

implement RTTT (Nowicki, 2015). The U.S. Department of Education reported on the impact of

RTTT on students’ graduation rates, enrollment in higher education institutions and participation

in Advanced Placement (AP) classes. (U.S. Department of education, 2015). In the final report,

released in October, 2016 the effect of RTTT on student achievement was reported as

inconclusive and descriptive statistics of changes in the students’ achievement in language arts

and mathematics were reported, but not in science (Dragoset, Thomas, Herrmann, Deke, James-

Burdumy et al., 2016).The evaluators of the Tennessee’s RTTT’s STEM professional

development (PD) programs found out that a significant growth in effectiveness and attitudes of

science and math teachers (Johnson, 2014), but there was no discussion how this growth affected

students’ science and math achievement.

RTTT and closing the achievement gap in science. A goal of RTTT was to decrease the

achievement gap between subgroups of students in reading/language arts and mathematics (US

Department of Education, 2009). Aside from these two subjects’ achievement gap there is also a

science achievement gap that has been explored by several scholars. (Abedi & Gándara, 2006;

Anderman, 1998; Bacherach, Baumeister & Furr, 2003; Cawley, Hayden, Cade & Baker-

5

Kroczynski, 2002). For example, in 2015 4th graders who were English Language Learners

(ELL), Black and Hispanic, had a disability, or were eligible for free and reduced lunch scored

27-37 points below their peers who did not belong to these subgroups when assessed for NAEP

science (National Center for Educational Statistics, n.d ). Large body of knowledge explains that

the racial, ethnic, ELL, disability and socioeconomic status are associated with the achievement

gap in science. Often such children attend schools where they are not usually challenged with

rigorous curriculum or not taught by qualified science teachers (Lieberman & Hoody, 1998;

Ruby, 2006; Lee, Maerten‐Rivera, Penfield, LeRoy & Secada, 2008, Quinn & Cooc, 2015).

Unfortunately, no studies have examined yet the impact of RTTT on science achievement of

these subgroups of students. One report discussed the approaches RTTT states employed to

improve the outcomes for English Language Learners (ELL) and their performance in math and

reading/language arts (Dragoset, James-Burdumy, Hallgren, Perez-Johnson, Herrmann, et al.,

2015), but there was no reference to the ELL students’ science achievement. The achievement

gap in science and it being persistent makes it even more important for us to understand how

RTTT may or may not contribute to improving educational outcomes in science and reducing the

achievement gap. Further investigation of the individual state programs would help to determine

what initiatives were successful, so that they could be used as models for implementation in

other states.

In this study, we use the education production function (EPF) as a theoretical framework

for analyzing the relationship between the educational inputs (RTTT funds and students’

background) and the educational outputs (student achievement scores in science) (Bowles, 1970;

Ferguson, 1991; Hanushek, 1986; MacPhail-Wilcox, & King, 1986; Todd & Wolpin, 2003). The

conceptual framework of RTTT with the study’s logic model is outlined in Figure 1.

6

Fig. 1. Race to the Top Conceptual Framework with the study’s logic model

INPUTS INPUTS OUTPUTS Long-term OUTCOMES

Federal Government

Transparent

Competitive

Selection Process

RTTT grant

Resources

Students’

background

Initiatives,

specific for

each RTTT

state

RTTT

resources

Expected

Increased Student

achievement in

science

Closing the

achievement gap in

science

Excellence in

Science

Education

Competent

STEM Force

Closing the

achievement

gap in Science

Education

Diverse

STEM force

Innovative

Science

Teaching

Learning

Developing 21st

century skills

Phase I - DE, TN

Phase II- DC, FL, GA, HI,

MA, MD, NC, NY, OH, RI

Phase III- AZ, CO, IL, KY,

LA, NJ, PA

Treatment states

Non awardees States Applicants:

AL, AR, CA, CT, IA, ID, IN,

KS, ME, MI, MS, MN, MO,

MT, NE, NH, NM, NV, OK,

OR, SC, SD, UT, VA, WA, WI,

WV, WY

Non awardee states Non-

applicants: TX, ND, AK, VT

Comparison states

Economic and political conditions in each state 2009-2014/2015

RTTT Framework

7

Methodology

Data

The data for this project came from the National Center for Educational Statistics

(NCES) database – the National Assessment of Educational Progress (NAEP). The NAEP

provides data on the independent assessment of student science achievement across the U.S. with

science score data for 4th (2009 and 2015) and 8th (2009, 2011 and 2015) grade students in public

schools. The study’s focus is on 4th and 8th graders’ science achievement, because if students are

not engaged in quality science learning in elementary and middle school that will affect their

future science achievement and interests to pursue careers in STEM disciplines. Publicly

available NAEP data are used for this study and in the future, upon receiving access to the

restricted NAEP data we will refine these analyses. Using the NAEP science achievement data

for 4th and 8th graders in public schools in all 50 states and District of Columbia (DC) and

additional variables from Common Core Data (CCD), an Independently Pooled Cross Sections

(IPCS) data sets (4th and 8th grade) was assembled. This data set contained random samples of

independent observations (science scores of 4th and 8th graders for each state), collected in

different time periods (2009, 2011, 2015). As such, IPCS data are very useful to evaluate the

effect of policies (Wooldridge, 2010) and the described data provide independent assessment

about the changes in students’ science achievement before and after the RTTT. The NAEP also

reports the scores for subgroups of the ELL students, students with disability, students from low

socioeconomic status and different minorities (Black and Hispanics) and these average scores

will be used to estimate the effects of RTTT on the achievement gap in science. The average

national sample size per tested subject, per grade, per year is between 110,000 to 115,000

students from more than 6,000 schools from across the US. Common Core Data (CCD) data and

8

American Community Survey provided information on the, the spending per pupil per state for

2009, 2011 and 2015. Since the data for 2015 spending has not been released yet we are using

the spending per pupil from 2014. Additionally, the second model included dummy variables to

account for the adoption of Common Core State Standards (CCSS) in mathematics and language

arts and the Next Generation Science Standards (NGSS) in the specific states to be able to

control for the influence of such educational initiatives on student achievement.

Data Analysis

The research questions are answered using a difference in differences estimation

technique (Wooldridge, 2010).

Yit= β1+ β2 d2015t + δ (d2015t* dRTTTi) + β3State Dummy variablesi+ uit, , ( t=1,2)

Yit – science scores for 4th or 8th graders for state i at time period t where t=1 for 2009 (phase I

and phase II) OR (t=1 for 2011 for phase III); t=2 for 2015 (phase I, II and III);

β1 – average science scores for the reference state in period t=1;

β2 – change in average science scores for all 50 states and DC between periods 1 and 2 (t=1 &

t=2);

d2015t is a dummy variable for period 2 which is 2015 for all phases. (It is set for 1 for 2015

and 0 for the year before RTTT (2009 for phases I and II; 2011 for phase III). It is important to

include this second-time period dummy variable to account for any general changes that take

place over time. By doing this we ensure that the estimated effect of RTTT is not confused with

the effect of other factors that have changed between the two-time periods for all studied states

and DC);

9

dRTTTi is a dummy treatment variable and it equals 1 for all states that received RTTT grant

and 0 for the states that did not received grants);

β3 – shows the differences in student achievement by state due to various unobservable

factors; State Dummy variables- There will be 50 dummy variables and DC will be a reference

group. By including these dummy variables, we account for the differences between the students

who attend public schools in the different states, the differences between their teachers’

preparation or different state science standards;

δ – shows the effect of the RTTT on students’ science achievement and answers the research

question; uit – error.

The dependent variable is the average student science achievement in state i, at the time

before (t=1) RTTT took place and after (t=2). RTTT was announced in July, 2009 and awards

were allocated to states in three phases (March, 2010, September 2010 and December 2011)

(U.S. Department of Education, 2009). The test allows to determine student achievement for

overall science and for life science, physical science and earth science which gave us an

opportunity to explore if there is some more specific effect depending on the type of science.

Key independent variable is the RTTT award for the specific state. Having states that received

awards and states that did not allows for identifying treatment and comparison groups. Model 1

estimates the effect of RTTT on science achievement without control variables, while Model 2

contains control variables such as spending per pupil per state and adoptions of CCSS and

NGSS, two important educational initiatives that might have influenced the outcomes for the

students.

10

The complete list of variables used for the analyses for both 4th and 8th grade is included

in Table 1.

Table 1. List of Variable Names, Descriptions, and Labels

RQ1. Did RTTT improve science achievement?

RQ2. Were different subgroups of students, defined by race/ethnicity, English language

learner, disability or income status experienced different outcomes of RTTT?

Variable name Description NAEP Label

NAEP sci overall Average sci scores in each state SRPUV

NAEP Physical sci score Average life sci score in each state SRPS1

NAEP Earth sci score Average earth sci score in each state SRPS2

NAEP Life sci score Average life sci score in each state SRPS3

National Public school students Charter, but not BIE and DOD schools TOTAL

Public schools Students attending public schools only SCHTYP2

Science scores of

ELL and non-ELL*

Classified by school LEP

Science Scores of Students with

disabilities*

Classified by SD, IEP, or 504 Plan IEP

Science Scores of students in

the National School Lunch

program

Eligibility based on school records SLUNCH3

Science scores by race and

ethnicity

White, Black, Hispanic, Asian/Pacific

Islander, American Indian/Alaska

Native, Two or more races

SDRACE

Note: BIE = Bureau of Indian Education, DOD = Department of Defense, ELL = English

Language Learners, Sci = Science.

Results

Fourth Grade Science Achievement

Examining the average science scores per state for 4th grade shows that there is a large

variability between states. The average score is 151 points, but Mississippi (MS) has the lowest

average score – 133 points and New Hampshire (NH) has the highest – 163 points in 2009. In

2015 the average science score per state is 155 and the states where students scored lowest and

highest are the same as in 2009 – MS – 135 points and NH – 167 points. The differences in

11

average science scores of subgroups of students in the states are substantial – black 4th graders

scored 31-34 points lower than their white peers; Hispanic 4th graders scored 23-27 points lower

than their white peers; ELL students scored 33-34 points lower than non-ELL students; students

with disabilities scored 24-26 points lower than students not identified as having disabilities;

students who qualified for free and reduced price lunch also scored 25-26 points lower than

students who are not part of the National School Lunch Program (NSLP). These trends of

science achievement of subgroups of students are observed at national and state level. Table 2

contains the summary statistics of 4th graders’ science achievement at a national level only.

The results of the analyses for 4th grade science achievement in the states that received

RTTT grant are presented in Table 3. The column, labeled Model 1 represents the estimated

coefficient of the effect of RTTT on student science achievement for the overall science scores,

and for life, earth and physical sciences. In addition to that, the estimated coefficients of the

effect of RTTT on the science performance of different subgroups of students (ELL, non-ELL,

Black, Hispanic, White, students with and without disabilities and students who qualify for free

and reduced lunch program and the ones who do not ) are also presented in this column. These

results are the estimates of single regressions that did not contain any control variables. Our

results show there is no statistically significant effect of RTTT on the overall science

performance of 4th graders and the different subgroups of 4th graders, except we report

statistically significant positive effect of RTTT on the science achievement of students who

qualify for free and reduced lunch at p<.05 level. To account for differences between the states in

terms of spending per pupil and important educational initiatives such as the adoption of CCSS

and NGSS we use these three controls in the second model. The statistically significant positive

effect of RTTT on science achievement of 4th graders who qualify for free and reduced lunch is

12

Table 2.

Summary Statistics for Fourth Grade for Years 2009

Science Score Science Score by Student Subgroup

Overall Earth Physical Life Black Hispanic White ELL NELL WD WOD FRL NFRL $/pupil

4th Grade

2009

#Obs. 48 48 48 48 41 45 48 39 48 48 48 47 47 52

Mean 151 151 151 151 128 135 162 119 153 130 154 137 163 10748

STD 7 8 7 7 7 8 4 11 7 11 7 7 5 2612

Min 133 132 133 135 116 121 150 98 133 95 135 122 149 2612

Max 163 165 163 163 141 153 172 143 164 148 167 151 172 18126

2015

#Obs. 48 48 48 48 40 45 48 41 48 48 48 47 47 52

Mean 155 156 154 154 133 141 164 124 157 132 158 143 168 11508

STD 6 7 6 5 7 7 4 12 6 10 6 6 4 3293

Min 140 140 141 143 119 128 152 94 141 101 143 128 156 6500

Max 166 168 165 165 148 163 175 150 167 147 170 152 177 20610 ELL = English Language Learners, NELL = Non-English Language Learners, WD = Students with Disabilities, WOD = Students without

Disabilities, FRL = Students on Free/Reduced Lunch, NFRL = Students not Eligible for Free/Reduced Lunch

13

Table 3. Estimated Effects of Race to the Top on 4th grade students’ Science Achievement

4th grade science #

States

N Model 1 Model

2

Science overall 49 92 1.100

(1.172)

1.848*

(1.036)

Life science 49 92 1.148

(1.147)

1.510

(1.097)

Earth Science 49 92 0.807

(1.106)

1.809*

(.902)

Physical Science 49 92 2.064

(1.235)

1.719

(1.136)

Black 43 77 1.833

(1.794)

3.234*

(1.849)

Hispanic 46 86 1.573

(1.389)

2.159

(1.438)

White 49 92 .505

(1.057)

1.143

(1.011)

English Language

Learners (ELL) 43 76

-2.615

(2.519)

-6.753***

(2.715)

Non-ELL 49 92 1.245

(1.102)

2.024**

(1.005)

Students with

Disabilities 49 92

1.433

(1.804)

3.000**

(1.741)

Students without

Disabilities 49 92

0.879

(1.127)

1.506

(1.033)

Students eligible for

Free and Reduced

Lunch

49 92 2.283**

(1.049)

2.610**

(1.055)

Students not eligible

for Free and Reduced

Lunch

49 92 0.663

(1.433)

1.243

(1.212)

Note: Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Model 2 controlled for state spending

per pupil, state adoption of Common Core State Standards, and state adoption of Next Generation Science

Standards.

14

also observed in Model 2, where the control variables are added. In the second model, we

observed positive and statistically significant effect of RTTT on the overall 4th grade students

‘science scores, their earth science scores and black and non-ELL students’ science scores at

p<.1 level. Additionally, we report negative statistically significant effect of RTTT on ELL

students’ science achievement and positive statistically significant effect of RTTT on science

scores of students with disabilities at level p<.05.

Eight grade Science Achievement

The descriptive summary statistics of the science scores of eight graders is presented in

Table 4. 8th graders were tested in 2009, 2011 and 2015, because the decision was made to align

NAEP with TIMSS in 2011. The average science scores per state for 8th graders also show that

there is a large variability between states. The average score is 151 points, but Mississippi (MS)

has the lowest average score – 132 points and North Dakota(ND) and Montana(MT) have the

highest – 162 points in 2009. In 20111, MS still has the lowest score (137) and ND the highest –

164 points. In 2015 the average science score per state is 155 and MS still with the lowest (140

points) and UT with the highest average score (166 points). The differences in average science

scores of subgroups of students in the states are also substantial – black 8th graders scored 33-35

points lower than their white peers; Hispanic 4th graders scored 22-25 points lower than their

white peers; ELL students scored 43-46 points lower than non-ELL students; students with

disabilities scored 31-33 points lower than students not identified as having disabilities; students

who qualified for free and reduced price lunch also scored 24-25 points lower than students who

are not part of the NSLP. Similarly, to the 4th graders the same trends are observed for the 8th

grade subgroups of students’ science performance at national and state level. Summary statistics

of eight graders’ science achievement at a national level only is presented in Table 4.

15

In addition to these differences in student science performance across the different states

there are also vast differences in the spending per pupil in each state, with the state spending the

most per pupil being almost three times higher than the lowest spending state. This discrepancy

in spending is observed in 2009, 2011 and 2014. The decisions to adopt the CCSS and NGSS are

also vastly different. There were incentives to the states to adopt the CCSS while the adoption of

NGSS is without any incentives. CCSS was originally adopted by 48 states, but later some of the

states choose to withdraw. NGSS is rather new and many states have not decided on adopting

them yet. As of November, 2016 the following states have adopted the NGSS – Arkansas,

California, Connecticut, Delaware, Hawaii, Illinois, Iowa, Kansas, Kentucky, Maryland, Nevada,

New Jersey, Oregon, Rhode Island, Vermont, Washington, and DC. The results of the analyses

for 8th grade science achievement and the RTTT effects are presented in Table 5. RTTT had

three phases – April, 2010, September, 2010 and December, 2011. We estimate the overall effect

of RTTT for the three phases. The column, labeled Model 1 represents the estimated

coefficients and robust standard errors of the effect of RTTT on 8th grade student science

achievement for overall science scores and for life, earth and physical sciences and on the

science performance of different subgroups of students (ELL, non-ELL, Black, Hispanic, White,

students with and without disabilities and students who qualify for free and reduced lunch

program and the ones who do not). These results are the estimates of single regressions that did

not contain any control variables. The results show we have no statistically significant effect of

RTTT on the overall performance of 8th graders and most of the different subgroups of 8th

graders except we report statistically significant positive

16

Table 4.

Summary Statistics for Eight Grade for Years 2009, 2011, and 2015

Science Score Science Score by Student Subgroup

Overall Earth Physical Life Black Hispanic White ELL NELL WD WOD FRL NFRL $/pupil

8th Grade

2009

#Obs. 48 48 48 48 41 43 48 31 48 48 48 47 47 52

Mean 151 151 150 151 127 135 160 109 152 123 154 136 161 10748

STD 8 8 8 8 7 8 5 11 7 11 8 8 5 2612

Min 132 131 132 134 111 119 146 86 132 97 135 122 147 6356

Max 162 165 162 163 144 155 170 136 164 140 166 151 169 18126

2011

#Obs. 53 53 53 53 41 46 53 28 53 53 53 51 51 52

Mean 152 152 152 153 128 138 163 109 154 125 156 140 164 11037

STD 9 9 8 9 7 8 5 13 9 12 9 7 5 2982

Min 112 110 117 113 107 116 150 75 113 74 119 127 151 6212

Max 164 166 164 164 149 158 179 133 165 143 168 153 171 19076

2015

#Obs. 48 48 48 48 39 45 48 29 48 48 48 47 47 52

Mean 155 154 154 156 131 142 164 111 157 126 159 142 166 11508

STD 6 7 6 6 7 6 4 12 6 9 6 6 4 3293

Min 140 137 138 143 118 127 150 88 140 102 143 129 154 6500

Max 166 165 167 168 151 162 173 136 169 144 170 152 175 20610

ELL = English Language Learners, NELL = Non-English Language Learners, WD = Students with Disabilities, WOD = Students

without Disabilities, FRL = Students on Free/Reduced Lunch, NFRL = Students not Eligible for Free/Reduced Lunch

17

Table 5. Estimated Effects of Race to the Top on 8th grade students’ Science Achievement

8th grade science #

States

N Model 1 Model

2

Science overall 50 143 .426

(.587)

.697

(.543)

Life science 50 143 .582

(.610)

.776

(.537)

Earth Science 50 143 1.266**

(.565)

1.511***

(.492)

Physical Science 50 143 -.339

(.673)

-.164

(.661)

Black 49 115 -.718

(2.384)

-1.571

(2.863)

Hispanic 48 128 .307

(1.113)

.575

(1.190)

White 50 plus

DC 143

.187

(.697)

.202

(.577)

English Language

Learners (ELL) 35 85

-3.338

(2.557)

-1.322

(2.690)

Non-ELL 50 143 .796

(.601)

1.049*

(.531)

Students with

Disabilities 50 143

0.085

(1.630)

.237

(1.603)

Students without

Disabilities

50 plus

DC 143

0.271

(.182)

.905*

(.476)

Students eligible for

Free and Reduced

Lunch

50 plus

DC 143

1.574**

(.693)

1.654**

(.698)

Students not eligible

for Free and Reduced

Lunch

50 plus

DC 142

0.831

(.725)

1.105*

(.596)

Note: Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Model 2 controlled for state spending

per pupil, state adoption of Common Core State Standards, and state adoption of Next Generation Science

Standards.

effect of RTTT on 8th graders’ earth science scores at p<.05 level and statistically significant

positive effect on 8th graders who qualify for free and reduced lunch science scores at p<.05

level. The column, labeled Model 2 represents the estimated coefficients /robust standard errors

when control variables were added to the model. Adding the control variables increased the

18

estimated coefficient for earth science and it was also statistically significant at p<.05 level. The

estimated coefficient of the RTTT effects on students who qualify for free and reduced lunch

slightly increased and stayed positive statistically significant at p<.05 level. The RTTT effect on

8th graders’ earth science scores became statistically significant at p<.001 level and the RTTT

effects on 8th grade non-ELL students’, students with disabilities and students who do not qualify

for free and reduced lunch and their science scores became statistically significant at p<.1 level

compared to being statistically non-significant in model 1.

Our results show RTTT had some positive and some negative effects on public school

students’ science achievement. To some extent our results agree with the findings included in

the RTTT final report that concluded that there might be positive, negative or no effects of RTTT

on student outcomes in math and reading/ language arts, but the evaluation report refrained from

any conclusions related to causal effect of RTTT on student outcome (Dragoset et al., 2016)

Since no science scores were analyzed as part of the RTTT evaluation we do not have other

studies to compare our results to now.

Discussion

In this study, we sought to find out if RTTT had any effects on science achievement of 4th

and 8th graders in public schools and if different subgroups of students who underperform in

science experienced different effects of RTTT program or if RTTT has contributed to closing the

achievement gap in science. While, we do not report about positive, statistically significant effect

of RTTT on student science achievement across the grades, we do have positive statistically

significant effect of RTTT on 8th graders’ achievement in earth science for all students at p<.05

and for 4th graders’ achievement in earth science at p<.1 level. In terms of RTTT closing the

achievement gap, we report that 4th and 8th graders who are eligible for free and reduced lunch

19

experienced statistically significant positive effect on their overall science scores, which is

encouraging. The ELL in 4th grade experienced statistically significant negative effect of RTTT

on their overall science scores, a finding that should be explored further. The statistically

significant positive effect of RTTT on 4th graders’ who belong to the subgroups of black students

and students with disabilities are optimistic.

Limitations

Estimating causal effect of RTTT on science achievement of elementary and middle

school students requires multilayered approach. Here we presented results that are at state level

and based on comparison of the performances of the states before and after the RTTT. Even

though NAEP aims to provide a national assessment of student progress we still have states that

choose not participate or participate selectively at specific year. This attrition at a state level

might be affecting our estimates. For example, DC and Alaska participated in the NAEP testing

only for 8th graders in 2011. Similarly, some states do not have representative sample of special

subgroups of students and that might also interfere with our estimates.

Conclusion

The importance of providing equal learning opportunities for all students is undisputed.

In this study, we found significant positive effects of RTTT on 8th graders’ performance in earth

science and on the overall science achievement of 4th and 8th graders’, who are eligible for free

and reduced lunch. While some of our results are, inconclusive and require further investigation,

we could say that the large investments in education, such as RTTT could have an impact on

student outcome. Our study was motivated by the lack of information about the effects of RTTT

on students’ science achievement and closing the achievement gap in science. We conclude with

20

how important is to improve science education for all students and to remediate the implications

of the persisting achievement gap in science for the students who are minority, ELL, have

disabilities or are from low socioeconomic background for their future opportunities to pursue

careers in STEM and contribute to these fields with innovative ideas.

21

References

Abedi, J., & Gándara, P. (2006). Performance of English Language Learners as a Subgroup in

Large‐Scale Assessment: Interaction of Research and Policy. Educational Measurement:

Issues and Practice, 25(4), 36-46

Anderman, E. M. (1998). The middle school experience effects on the math and science

achievement of adolescents with LD. Journal of Learning Disabilities, 31(2), 128-138.

Bacharach, V. R., Baumeister, A. A., & Furr, R. M. (2003). Racial and gender science

achievement gaps in secondary education. The Journal of genetic psychology, 164(1),

115-126.

Barnes, C. R. (2011). Race to the Top Only Benefits Big Government. JL & Educ., 40, 393.

Bowles, S. (1970). Towards an educational production function. In Education, income, and

human capital (pp. 11-70). NBER.

Cawley, J., Hayden, S., Cade, E., & Baker-Kroczynski, S. (2002). Including students with

disabilities into the general education science classroom. Exceptional Children, 68(4), 423-

435.

Dragoset, L., James-Burdumy, S., Hallgren, K., Perez-Johnson, I., Herrmann, M., Tuttle, C., ... &

Graczewski, C. (2015). Usage of Policies and Practices Promoted by Race to the

Top (No. 4b296d4310f843acad1f80b5aadd54d2). Mathematica Policy Research.

Retrieved from https://ies.ed.gov/ncee/pubs/20154018/pdf/20154018.pdf

Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., Graczewski, C., Boyle,

A., Tanenbaum, C., Giffin, J., & Upton, R. (2016). Race to the Top: Implementation and

22

Relationship to Student Outcomes (NCEE 2017-4001). Washington, DC: National Center

for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S.

Department of Education. Retrieved on Nov7th, 2016 from https://www.mathematica-

mpr.com/download-media?MediaItemId={75821048-6E9E-456B-A13D-

731A64BE45FD}

Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money

matters. Harv. J. on Legis., 28, 465.

Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public

schools. Journal of economic literature, 24(3), 1141-1177.

Holdren, J. P., Lander, E. S., & Varmus, H. (2010). Prepare and inspire: K-12 education in

science, technology, engineering, and math (STEM) for America's future. Executive

Report). Washington, DC: President's Council of Advisors on Science and Technology.

Howell, W. G. (2015). Results of President Obama's Race to the Top. Education Next, 15(4)

Johnson, C. (2014).THEC STEM Professional Development Program: Final Evaluation.

Tennessee Consortium on Research, Evaluation and Development. Vanderbilt

University’s Peabody College. Nashville, TN 37203

Lee, O., Maerten‐Rivera, J., Penfield, R. D., LeRoy, K., & Secada, W. G. (2008). Science

achievement of English language learners in urban elementary schools: Results of a first‐

year professional development intervention. Journal of Research in Science

Teaching, 45(1), 31-52.

23

Lieberman, G. A., & Hoody, L. L. (1998). Closing the Achievement Gap: Using the

Environment as an Integrating Context for Learning. Results of a Nationwide Study.

MacPhail-Wilcox, B., & King, R. A. (1986). Production functions revisited in the context of

educational reform. Journal of Education Finance, 12(2), 191-222.

McGuinn, P. (2012). Stimulating reform: Race to the Top, competitive grants and the Obama

education agenda. Educational Policy, 26(1), 136-159.

National Science Foundation Strategic Plan for 2014-2018. (2014). NSF. Arlington, VA.

Nowicki, J. M. (2015). Race to the Top: Education Could Better Support Grantees and Help

Them Address Capacity Challenges. Report to the Ranking Member, Subcommittee on

Labor, Health and Human Services, Education and Related Agencies, Committee on

Appropriations, House of Representatives. GAO-15-295. US Government Accountability

Office.

Quinn, D. M., & Cooc, N. (2015). Science Achievement Gaps by Gender and Race/Ethnicity in

Elementary and Middle School Trends and Predictors. Educational Researcher, 44(6),

336-346.

Ruby, A. (2006). Improving science achievement at high‐poverty urban middle schools. Science

Education, 90(6), 1005-1027.

Scott, G. A. (2011). Race to the Top: Reform Efforts Are Under Way and Information Sharing

Could Be Improved. Report to Congressional Committees. GAO-11-658. US Government

Accountability Office.

24

The White House. Office of the Press Secretary. (2009). President Obama Launches” Educate to

Innovate” Campaign for Excellence in Science, Technology, Engineering& Math (Stem)

Education. Washington, DC: The White House. Retrieved from

https://www.whitehouse.gov/the-press-office/president-obama-launches-educate-

innovate-campaign-excellence-science-technology-en

Todd, P. E., & Wolpin, K. I. (2003). On the specification and estimation of the production

function for cognitive achievement. The Economic Journal,113(485), F3-F33.

U.S. Department of Education (2009). Race to the top Program Executive Summary.

Washington, DC 20202. Author: Retrieved: August 4th, 2016 from

https://www2.ed.gov/programs/racetothetop/executive-summary.pdf

U.S. Department of Education (2015). Fundamental change. Innovation in American schools

under the race to the top. U.S. Department of Education. Washington, DC.

Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data. MIT press

423-435.