McMullen 2010 - The Impact of Homework Time on Academic Achievement

7/28/2019 McMullen 2010 - The Impact of Homework Time on Academic Achievement

1/31

The Impact of Homework Time on Academic Achievement

Steven McMullen

May 2010

Working Paper

Abstract

This study takes advantage of nationally representative panel data on student behavior and

academic performance to test two possible policy reforms. First, I examine a policy thatincreases the amount of homework that students complete. Second, I examine the impact ofincreasing the amount of homework assigned. Previous studies have not been able toconsistently estimate the impact of homework because of important omitted variables andmeasurement error, which strongly bias the estimated impact of homework time. This paper,however, uses an instrumental variables approach with student fixed effects to account for bothtime-varying and time-invariant unobserved characteristics and inputs. This approach producesestimates of the impact of homework time on academic achievement that are much larger thanthose of previous studies. Additionally, these findings suggest that assigning additionalhomework primarily improves the achievement of low performing students and students in lowperforming schools. Thus, assigning more homework could help close the gap in achievement

between high and low performing students.

JEL classification: J24; I20; I21Keywords: homework, academic achievement

Steven McMullen is an assistant professor of economics at Calvin College in Grand Rapids, Michigan. The authorwould like to thank Tom Mroz, David Blau, Donna Gilleskie, David Guilkey, and Helen Tauchen for their excellentassistance. Steven can be reached at [email protected] or Department of Economics, Calvin College, 1740Knollcrest Circle SE, Grand Rapids, MI 49546.


2/31

2

I. Introduction

The amount of homework that students should complete has been an issue of debate and

scholarship for decades. A number of popular books and articles have been written recently

arguing that homework should be limited or prohibited entirely (Bennett and Kalish 2006; Kohn

2006; Kralovec and Buell 2001)1. While many scholars have investigated the impact that

homework has on academic achievement, there is no strong consensus in the literature.

Moreover, most studies have done little to correct for the biases caused by omitted variables that

likely influence students choices regarding study time.2

In this paper, I examine two related issues. The first is the effect of students time spent

doinghomework on achievement test scores. The second is the impact ofassigningadditional

homework on achievement. The primary econometric challenge that this study overcomes is that

there are likely a host of unobserved variables, such as student ability, that influence both how

much time a student spends on homework and the students achievement test scores. By

combining instrumental variables estimation with an individual-fixed-effects specification, I

control for both unobserved heterogeneity that is constant over time and time-varying

heterogeneity that might influence the amount of time students spend on homework. This

approach yields estimates of the impact of homework that are much larger than those of previous

studies. The presence of time-varying unobserved factors is especially important to consider

when estimating the impact of education inputs that are under the students control, since

1

Assigning homework seems to be less and less popular. A 2003 Brookings institute report claims that Since2001, feature stories about onerous homework loads and parents fighting back have appeared in Time,Newsweek,andPeople magazines; theNew York Times, Washington Post,Los Angeles Times,Raleigh News and Observer, andthe Tampa Tribune(Loveless 2003). Their research indicates that the homework load for most US students isactually quite low.

2 Recent work done by economists (Aksoy and Link 2000; R. Stinebrickner and T. Stinebrickner 2008; Eren andHenderson 2009) have made progress, but their approaches can only address a subset of the endogeneity problemsinvolved with measuring this effect. The difference between their results and those of previous scholars (Cooper,Robinson, and Patall 2006) indicates that endogeneity problems result in large biases.


3/31

3

students can respond in each period to changing incentives. Moreover in the presence of

measurement error, this approach will yield more accurate results.

I find that one extra hour of mathematics homework per week improves mathematics

achievement by 0.179 standard deviations3. This change is large enough to move a student from

the 50th percentile of math achievement to the 57th percentile4 over the course of a school year.

Moreover, I find evidence that low-performing students and those in lower-performing schools

realize much higher returns to their homework time than other students, and that a policy of

assigning more homework disproportionately benefits these students. These findings lead to two

important conclusions. First, it is possible for students to overcome the lower achievement that

results from attending a low quality school by spending more time doing homework. Second, a

policy that increases the amount of homework assigned is likely to reduce the gap between low

and high achieving students.

II. Background

The limited theoretic literature on homework predicts a positive impact of homework

time on academic achievement (Betts 1997; Neilson 2005). Almost all of the empirical studies

on this topic find evidence that homework time has a positive impact on academic achievement

among high school students, although there is no consensus on the magnitude of the impact. The

literature pertaining to this topic will be presented in two sections: first, a review of literature

3 The impact is reported here in terms of the standard deviation of the mathematics achievement test score used asthe measure of academic achievement in this study. The sample used for this study is nationally representative, andthus the impacts can be interpreted as increases in achievement relative to the performance of similar students in theUS.

4 The impacts are relative to students who do not increase their homework time.


4/31

4

from outside the economics discipline and second, a review of recent work done by applied

economists.

Within the fields of education, psychology, and sociology there has been much research

on the impact of homework. Cooper et al. (2006) provide a good review of recent work in these

fields. All of the published studies that Cooper et al. review that use multivariate regression

analysis find similar results: increased homework causes a small increase in academic

achievement. These studies, however, are limited to cross section analysis and do not try to take

into account omitted inputs or characteristics. A number of scholars have also examined college

students (Schuman et al. 1985; Rau and Durand 2000; Hill 1991); all find similar results when

trying to document the impact of homework time on the performance. Each study either finds

the relationship hard to document or finds a small effect of homework time on college grades.

Applied economists have only recently started to pay attention to the impact of

homework time on achievement, starting with Julian Betts 1996 analysis of the Longitudinal

Study of American Youth. Since then Aksoy and Link (2000), Eren and Henderson (2008,

2009), and Stinebrickner and Stinebrickner (2008) have all examined this question. Each of

these studies makes a serious attempt to address unobserved inputs. Eren and Henderson use a

value added specification in the parametric portion of their 2008 paper,5 and a student and

teacher fixed-effects approach in their 2009 paper. Both Betts and Aksoy and Link use a

specification with student fixed effects, and Stinebrickner and Stinebrickner use the students

roommates videogame ownership as an instrument, estimating the effect with two stage least

squares.

5 The value added specification is a cross section regression with a lagged test score (or other dependent variable)included on the right hand side of the equation to control for past achievement, inputs, and student characteristics.


5/31

5

This study improves the estimation of the effect of homework on academic achievement

in three ways. First, the primary shortcoming in this literature is that previous work fails to

adequately account for unobserved inputs that influence both academic achievement and the

amount of time spent on homework. None of these studies, with the exception of Stinebrickner

and Stinebrickner6 can account for unobserved influences that vary over time or measurement

error. To address these potential problems I use the amount of assigned homework and the

students locus of control7 as instruments for student homework time, as well as a student fixed

effects. Dealing with the endogeneity of students homework time in this manner greatly

influences the estimated effects. Second, previous research has documented that the return to

homework time varies based on a students ability (Eren and Henderson 2008). In this study, I

document that the return to homework also varies with school quality.

[Table 1]

III. DataThe National Education Longitudinal Study of 1988

The primary data used for the empirical work come from the National Education

Longitudinal Study of 1988, a nationally representative longitudinal survey of students who were

in the 8th grade in 1988. Follow up surveys were given in 1990 and 1992, with teacher and

school counselor surveys in each wave, and parent surveys in the first and third waves. With

each survey the students were given achievement exams in mathematics, science, English, and

history. The exams are designed to allow comparison across waves, and to accurately test

6 Stinebrickner and Stinebrickner (2008) do a good job documenting the relationship between homework andachievement, but do so within a selected non-representative college age population. Their instrumental variablestrategy, however, is not reproducible on a large scale.

7 The locus of control measures the extent to which a student believes that she can influence her own futureoutcomes. For a detailed description, see appendix A.


6/31

6

students at different achievement levels. Exam results are reported as a standardized variable

with a t-distribution, with mean 5 and a standard deviation of 1.

Of the 17,580 students who have at least two recorded mathematics exam scores, 12,534

students were not included in the sample because they did not have a mathematics instructor

supplied data.8 Another 389 students were not included because they did not respond to the

questions about homework time or locus of control. Moreover, 597 students were not included

because of missing school-wide data on class size and teachers salaries. Finally 82 students

were dropped because they did not have enough data on peers to construct peer performance

variables. The resulting sample includes 8877 observations on 3978 students. The average

student characteristics in this restricted sample do differ from the general sample on some key

variables, both because of non-random attrition and non-random survey non-response. However,

because the estimation strategy in this study focuses on accounting for unobserved student

heterogeneity, many potential sampling-related biases are likely mitigated. Moreover, the results

reported in this paper are qualitatively robust to significant changes in the sample. Summary

statistics for the variables used in this paper are shown in table 1. For variable definitions, see

appendix A.

IV. Estimating the impact of homework on achievement

The standard approach to finding the impact of an input into a students academic

achievement is to estimate an education production function similar to the following linear

econometric model:

(1)8 Each student had one or more teachers surveyed in each wave, but the teacher was often not their mathematicsinstructor. Only students with a mathematics instructor interviewed in at least two waves can be used for thisanalysis.


7/31

7

whereATit is an achievement test score, which serves as a measure of human capital. Test scores

are assumed to be a function of homework time (Hit), school and district inputs (Sit), teacher

inputs (Tit), and individual characteristics (Xit). The alpha parameters represent the impact of the

inputs on academic achievement,i is an error term that is constant over time, and is a time-

varying error term. In this specification the parameter of interest is 1, which represents the

marginal impact of an additional hour of homework on academic achievement9.

There are three reasons why an ordinary least squares estimation of equation 1 will likely

return a biased estimate of the impact of homework. First, because the amount of homework that

students complete is under the control of students, there may be permanent unobserved (to the

econometrician) variables that have a large impact on academic achievement that may also

influence a students optimal amount of time to spend on homework. For example, a students

innate mathematics ability will certainly impact their mathematics achievement test scores, and

will also likely impact the amount of mathematics homework that students complete. Similarly,

all education inputs that occur before 8th grade will be constant within each student for the

observed period of time, and will likely have a large impact on both achievement and homework

time. This type of unobserved heterogeneity will be a source of correlation between the error

term and the independent variables in the model, and create bias in the estimates of the

associated parameters. A number of different types of possible unobserved heterogeneity are

considered in the next section.

The second type of omitted variable bias comes from transitory education shocks which

will result in time-varying unobserved heterogeneity, such as an illness or other event that only

impacts the student for one period but may influence a students academic achievement, their

9 For a more general discussion of the identification of education production function parameters see Todd andWolpin (2003).


8/31

8

optimal study time, and other education inputs. Stinebrickner and Stinebrickner (2008)

document that this type of shock will likely result in a downward bias in estimates of the impact

of homework.10

The third source of bias is measurement error in the observed amount of homework that

students complete. This variable results from a question asked of the student once per wave, are

recorded categorically, and the questions and categories change slightly across waves.

Moreover, students may not be able to, or desire to, give accurate replies when asked for the

average amount of time that they spent on homework over the past year.11

One method for addressing these estimation problems is to include student fixed effects

in the specification (use a within estimator). The individual fixed effect will absorb the time-

invariant individual specific error term (i) as well as any time-invariant observed and

unobserved student characteristics. Thus even if there are multiple endogenous inputs other than

homework in the education production function, the bias from time-invariant omitted variables -

such as student ability and education investments from before 8th grade - will be eliminated.

Moreover, if the other inputs, such as teacher experience, certification, wages, or class size, are

selected by students and parents, this selection is likely based on permanent student

characteristics. As such, any bias caused by self selection is likely to be eliminated by using

student fixed effects.

The fixed effects estimation may produce estimates of the impact of homework ( ) that

are biased downward, however, for two reasons. First, the time varying omitted education shock

10 For example, if a student became seriously ill for one semester, their test scores would likely suffer, but at thesame time, they might increase the amount of time spent on homework in an attempt to make up for missed school.This would create a spurious negative correlation between homework time and test scores. A similar process maybe likely for other types of time-varying unobserved heterogeneity, such as a poor teacher-student match. For alarger discussion of this effect, see Stinebrickner and Stinebrickner (2008).

11There is some evidence that students answers are not accurate: the sum of the reported hours spent on eachsubject is, on average, noticeably higher than the total amount of time students report spending on homework.


9/31

9

may be correlated with the transitory portion of the homework and test score variables. If this is

the case, the fixed effects will magnify the impact of transitory changes in homework due to this

shock. Second, the measurement error will now account for an even larger portion of the

variation left in the observed amount of homework. This means that the bias caused by

measurement error will be more serious in a fixed-effects specification than one without fixed

effects.12

In order to address all of these problems I use instrumental variables to estimate a

specification that includes student fixed effects, an approach recommended for education

production function estimation in Todd and Wolpin (2003, 2007). The required instruments

must affect student homework and be uncorrelated with the omitted variables. When used in a

fixed effects specification, the student fixed effect is included in both the first and second stage

equations, and the instruments used must vary over time. The use of instruments to predict

homework time isolates the variation in the homework variable that is not correlated with the

omitted variables and not due to measurement error. Estimating the effect of this exogenous

variation in homework on achievement gives a consistent estimate of the impact of homework.

V. Instrument Validity

The two variables used in this study as instruments are the amount of homework assigned

by the students teacher and the students locus of control. Exogenous changes in the amount of

homework assigned by the students instructor are unlikely to have any direct causal impact on

test scores, apart from their effect on the amount of homework completed. The second variable

12Certainty about the direction of the bias depends on some restrictive assumptions the measurement errorwill result in a bias towards zero if: i) the measurement error is uncorrelated with the regressor inquestion, and ii) the measurement error is independent of the measurement error associated with otherregressors (Cameron and Trivedi 2005).


10/31

10

that is used to predict students homework is the locus of control. This variable measures the

degree to which a student believes that she can impact her own future. A student with an internal

locus is more likely to believe that her future depends on choices that she makes, while a student

with an external locus is more likely to believe that external forces will dictate the events of her

life.13

The first requirement of a good instrument is that it is correlated with the endogenous

variable in question. Given the fixed effects specification used here, it must also be true in this

case that within-student variation in the instruments predicts within student variation in the

amount of homework that students complete. It is plausible that this condition holds for both

variables. As teachers assign more homework students will likely spend additional time studying

that particular subject. Also, previous studies have found that a strong internal locus of control is

associated with positive outcomes in situations that require independent decision-making, such

as positive health behaviors (Steptoe and Wardle 2001), lower dropout rates in distance

education (Parker 1999), and success in web-based coursework (Wang and Newlin 2000).

Thus it not surprising that students in this sample that have an internal locus, on average, spend

more time doing homework, since they are more likely to expect the time investment today to

pay off in the future. Moreover, the locus of control variables show significant variation within

students over time, as might be expected for students at this age.

[Table 2]

Table 2 shows the first stage equations for the instrumental variables estimates of the

education production function. The specification includes both homework and homework

squared in the second stage equation, and the square of each instrument in the two first stage

13The locus of control used here is a combination of students answers to three standard questions. For a detaileddescription of the construction of the locus of control variable, see appendix A.


11/31

11

equations for each specification. These results show that both hours of assigned homework and

locus of control are important determinants of student homework time in each specification.

Additionally, the last row in Table 3 shows the p-value for an F-test on the joint significance of

the excluded instruments. The null hypothesis, that both variables have no impact on students

homework, would be rejected with p-values below 0.01 for the first two specifications, and

below 0.1 for the third specification which uses a significantly smaller sample.

The second requirement for valid instruments is that they do not have an impact on

student achievement except through their impact on homework. There are two types of

traditional tests for instrument exogeneity, both of which indicate that these instruments are

valid. First, the Hansen J-statistic is a chi-squared test for the exogeneity of all the instruments

(or a test for overidentifying restrictions), and is appropriate for specifications that are estimated

with standard errors robust to heteroskedasticity. The null hypothesis is that the excluded

instruments are exogenous and this hypothesis cannot be rejected in either specification. The

associated p-values are reported in the second to last row of table 2.

As an additional test, I excluded only the assigned homework instruments from the main

equation, and then excluded only the locus of control instruments. When included in the main

equation, the homework instruments and locus of control instruments were jointly not

statistically different from zero (p-values 0.94 and 0.93 respectively). This test, as well as the

test of overidentifying restrictions collectively provide strong evidence that these excluded

instruments do not predict student achievement independent of their effect on homework

completion.

Nevertheless, there are three levels of unobserved heterogeneity that could nevertheless

result in a correlation between these instruments and student achievement: differences at the


12/31

12

student level, the classroom or teacher level, and the school or community level. I will address

each in turn.

[Table 3]

Student level heterogeneity

It is possible that the amount of homework assigned might reflect the students past

achievement or ability. Much of this effect will likely be eliminated by the inclusion of student

fixed effects. To test this, I estimate the impact of a lagged test score on the amount of

homework students were assigned, including the teacher and school covariates on the right hand

side. For example, if teachers assign more homework to students who have better academic

backgrounds or more innate ability, we could expect past test scores to positively predict

assigned homework. The results are shown in table 3. The estimates in the first column show

that lagged test scores are strong predictors of each instrument when student fixed effects are not

included. When the specification includes student fixed effects, however, the coefficient

associated with the lagged test scores change sign and cannot be distinguished from zero. This

indicates that neither past achievement or permanent ability are correlated with the instruments

when fixed effects are included.

It is also possible that both of the instruments are correlated with the same unobserved

student or school characteristics and contain the same bias. In the last row and last column of

table 4 the locus of control is regressed on the amount of assigned homework in specification

with student fixed effects and period intercepts. In this specification the students locus of

control has no significant effect on the amount of homework that the student is assigned. These

variables, therefore, provide independent exogenous variation in homework time.


13/31

13

School or community level heterogeneity

It is also possible that these instruments would end up being correlated with student

achievement because of omitted school characteristics or unobserved community factors. To

control for these, I create two school-specific peer variables equal to the average mathematics

test score of all of the other observed students in the school, and another equal to the average

amount of homework time that other students in the school complete. The averages calculated

for these variables are student-specific since they omit the own students achievement and

homework time. Also, they are created before limiting the sample for the purposes of the study.

Adding these peer variables into the first and second stage equations should control for any

unobserved school characteristics that are correlated with both the amount of homework assigned

or the locus of control and the students performance.

Classroom or teacher level heterogeneity

Another concern is that assigned homework might be biased by teacher-specific

unobserved characteristics, and that even within schools, the variation in teacher quality might

impact students studying habits and performance. To control for these influences I create peer

variables defined at the teacher level instead of the school level. Doing so greatly reduces the

usable observations because the amount of clustering within teachers is lower than the clustering

within schools in the sample design, especially in the second and third waves. The last column

of table 4 shows the results for the specification that includes both the school and teacher level

peer variables. Despite the smaller sample, the results are very similar.

A related concern is that teachers might assign more homework to some classes than

others, based on their appraisal of the students ability and motivation. This could be the case if


14/31

14

there was tracking in the students math classes or if some students took advanced placement or

remedial courses. To the extent that this homework assignment is based on student and peers

achievement, the fixed effects and peer effects should minimize this bias. As an additional

control, however, I include a set of indicator variables that tell what achievement level the

teacher considers the students in the class to be at, given the following categories: higher level,

average, lower level, or widely differing. This control is an important one in Eren and

Hendersons analyses (2008, 2009). I include these in every specification, though none of these

variables end up being significant in second stage equation, indicating that the fixed effects and

peer variables do a good job capturing these achievement level differences.

14

[Table 4]

VI. Results: The Impact of Mathematics Homework on Mathematics Achievement

Table 4 displays the pooled cross section OLS, fixed-effects, and fixed-effects-two-stage-

least-squares estimates of the education production function. The empirical model employed

includes a quadratic in homework time to capture decreasing returns to studying. The marginal

effect of an hour of homework (evaluated at the mean observed weekly mathematics homework

time) is reported at the bottom of table 4. Each specification also includes wave indicator

variables, the average observed class size and three indicators of teacher quality: whether the

students teachers are certified in their subject, an indicator that the teacher has less than 4 years

of experience, and the minimum teacher pay in the students school.

14 Because these class level indicators do a good job predicting student homework time, but do not seem to impactachievement in a fixed effects specification, I also estimated a specification where these were added to the set ofinstruments excluded from the second stage. The results were very similar to those reported in the otherspecifications in table 4.


15/31

15

The ordinary least squares pooled cross section estimate of the impact of an hour of

mathematics homework per week, shown in column one, is a 0.049 standard deviation increase

in the students mathematics test score, which corresponds to an improvement of about 2

percentile points.15 This estimate may be biased if the students unobserved prior education

inputs or innate abilities influence the amount of time spent on homework, or if students select

other inputs based on unobservable characteristics.

In order to control for these endogeneity issues, a fixed effects specification is employed.

If the selection is based on student characteristics that do not change over time, then the student-

fixed effects estimate will eliminate the bias not only in the impact of hours of homework, but

also in the other inputs. This estimator will also control for the linear effects of any differences

in past inputs that are not included in these specifications. The second column in table 4 shows

the estimates of the same specification as the one shown in the first column, but with student

fixed effects included. These estimates are much smaller. The impact of an hour of homework

with student fixed effects is almost 1/5th the size of the OLS estimates presented in the first

column.

As explained in section four, the fixed effects estimates will likely be biased by presence

of measurement error and time-varying unobserved heterogeneity. These problems can be

addressed by using instrumental variables in addition to the student fixed effects. The third

column in table 4 shows the results of the student fixed effects specification estimated using two-

stage least squares, where the excluded instruments are the amount of homework assigned by the

students teachers and the students locus of control, as well as the square of each instrument.

This estimate, while less precisely estimated than the fixed-effects estimate, is much higher, and

15 Starting from the 50th percentile in Mathematics test scores within the unrestricted NELS sample.


16/31

16

gives a quantitatively and qualitatively different result. A student doing an average amount of

mathematics homework who completes an extra hour of homework each week is estimated to

improve her mathematics test score by 0.182 standard deviations, an improvement of 7 percentile

points. Even if we accept the lower bound of the 95% confidence interval for this estimate,

which is 0.06 standard deviations, the return to homework is still 6 times higher than the fixed

effects estimate. This lower bound estimate indicates that an hour of homework per week would

move a student from the 50th percentile to the 52nd percentile in mathematics achievement.

If the instruments provide exogenous variation in student homework time, the estimates

in the third column should not change substantially when other inputs are included in the

specification. Column four tests this by adding variables to control for other school-wide inputs

and peer group influences. The specification shown in column four includes the average amount

of homework done by the students peers across the school,16 and the average mathematics test

score of the students peers.17 Though these variables are strong predictors both of students

homework time and test scores, the estimate of the impact of homework remains almost exactly

the same.

Other recent studies that have estimated the impact of homework time on student

achievement have not been able to account for both time-varying and invariant unobserved

characteristics. These studies found much smaller impacts.18 The results reported here indicate

16 The students peers, in this case, are any students with observed test scores from the same school in the same year.17

(Manski 1993) and (Hanushek et al. 2003) argue that peer effects of this type can introduce a bias due to thereintroduction of a students ability through the students influence on her peers. The estimated impact ofhomework does not change when these variables are included, indicating that if these variables are problematic, theydo not bias the estimate of the impact of homework. Specifications that instrument for these peer effects usinglagged values produce very low precision estimates that cannot be differentiated statistically from the estimatesreported here.

18 The estimates presented in this paper are larger in magnitude than any the (Cooper, Robinson, and Patall 2006)review. Aksoy and Link also used the NELS data, and employed student fixed effects. See (Aksoy and Link 2000)table three specification 1. If their estimate is divided by 13, which is the approximate standard deviation reported


17/31

17

that the use of student fixed effects to control for omitted inputs and characteristics has

consistently produced estimates that are too low. This is likely the case because of a downward

bias that results from measurement error, as well as downward bias caused by students

responding to negative or positive time varying unobserved heterogeneity. It is also possible that

the impact of unobserved past inputs and characteristics biased the result downward, although

the direction of bias from this source is uncertain. In light of this, the correction provided by the

instrumental variables estimate is especially valuable.

It is also worth considering the rate at which the return on homework diminishes. For

students currently not spending any time doing mathematics homework, the return on completing

one hour of homework per week would be about a 10 percentile point improvement in

mathematics test scores. This return decreases as students spend more time studying. These

results indicate that, on average, mathematics homework ceases to improve test scores if a

student does more than about 11 hours per week. By this standard, the vast majority of students

could increase their test scores if they spent more time on their mathematics homework.

VII. The Impact of Assigned Homework on Academic Achievement

This strong impact of homework time indicates that any policy that increases the amount

of homework students complete could be valuable. One such policy instrument is the amount of

homework that teachers assign. Teachers assignments, even if only completed a fraction of the

time, can have a strong influence on the amount of homework students complete. To illustrate

this, I estimate a reduced form version of the specification displayed in column 4 of table 4,

which is similar to the regression in column 2 of table 4, with assigned homework instead of

in table 1, they find that an hour of homework increases mathematics achievement by 0.051 standard deviations.The results of this study indicate that the true effect is much larger.


18/31

18

homework completed as the variable of interest. This specification includes student fixed

effects, school-wide peer variables, teacher characteristics, class size, class level dummy

variables and wave (grade) specific intercepts. The estimated marginal effect of being assigned

an additional hour of homework for students currently being assigned the average amount (2.9

hours per week) is 0.015 (standard error 0.004) test score standard deviations. This effect size is

notably smaller than the estimated impact of a student completing an hour of homework, as is

expected since homework assignments are an imperfect predictor of homework completion.

This effect size is also well within the range of similar estimates in other studies (Aksoy and

Link 2000; Betts 1996; Eren and Henderson 2008; 2009).

VIII. The Impact of Homework Time by Achievement Level and School Quality

In this section, I will try to relax the restriction that students at different levels of

achievement or in different types of schools receive the same return on homework time. To do

this I estimate the specification from the table 4 column 4 separately for high achieving and low

achieving students, as well as high and low performing schools.

[table 5]

First, students are separated into two groups of equal size based on their 8th grade science

test scores. This distinction yields a small difference that is not-statistically significant, but

indicates that students at the bottom half of the population in terms of academic performance

might see higher returns to homework completion. This evidence does not support the common

assumption that more able students receive a higher return to homework time because their time

is more productive (Eren and Henderson 2008; Neilson 2005). 19 Second, students were split

19 Eren and Henderson (2008) find higher returns to homework at the top and bottom of the achievementdistribution. When I split the sample into more than two groups, I find results very similar to those reported: low


19/31

19

into groups of equal size based on the test score performance of their peers, as a general measure

of school quality. This distinction yielded a much larger, but still not statistically significant

difference in the estimated effect. Students in the lower performing schools experienced a return

to homework time that was much larger than those in high performing schools. A student in a

low performing school, who currently studies one hour each week that wanted to move from the

50th percentile to the 75th percentile, would have to study about 2 additional hours per week,

whereas a student in a high performing school would have to study 4 additional hours to improve

the same amount. This may indicate that at low performing schools homework is a stronger

determinant of success. One explanation for such a result comes from Bishop (2006), who

reports that less material is presented in class time in low performing schools, which may

partially explain why studying at home would be more beneficial for these students. At the very

least, this evidence does not support the assumption by some education scholars (Kralovec and

Buell 2000; Kohn 2006) that homework will only widen achievement gaps.

[table 6]

Finally, it is worth exploring whether assigning more homework will have a differential

impact based on school quality and prior student achievement. Table 6 shows the impact of this

policy for the top and bottom halves of the achievement and school quality distributions. The

effect is estimated with similar precision for each subgroup, but the effect is much smaller for

high performing students and students in high performing schools, though once again, the

differences are not statistically significant. Here also there is a noticeable difference in the

impact between high and low achieving students, based on their science test scores when

entering the sample. The relative effectiveness of assigned homework to low achieving students

achieving students and those in low performing schools have higher returns, moreover, the instrumental variablesestimates are not precise enough for the small samples required to investigate a more detailed distributional impact.


20/31

20

relative to high achieving students is partially due to the fact that low achieving students are

somewhat more likely to increase their homework time if assigned additional homework.

IX. Conclusions

The results in this paper strongly support the argument that increasing the time that

students spend on their homework will have visible impacts on academic achievement in

mathematics. Furthermore, there is some evidence that increasing homework time could

improve the performance of low-performing students and those in low-performing schools.

Because the sample used is nationally representative, the results should be applicable to a

number of school settings, and thus as a policy tool, increasing the amount of homework that

students complete shows some promise. Additionally, because much of the benefit of assigning

additional homework seems to be realized by low-achieving students and students in low-

performing schools, this policy could be useful for lowering the achievement gap between high

achieving and low achieving students.


21/31

21

References

Aksoy, Tevfik, and Charles Link. 2000. A panel analysis of student mathematics achievement

in the US in the 1990s: does increasing the amount of time in learning activities affect math

achievement?.Economics of Education Review 19(3): 261-277.

Bennett, Sara, and Nancy Kalish. 2006. The Case Against Homework. Crown Publishers.

Betts, Julian. 1996. The role of homework in improving school quality. University of

California, San Diego, Department of Economics.

Bishop, John. 2006. Drinking from the Fountain of Knowledge: Student Incentive to Study and

LearnExternalities, Information Problems, and Peer Preasure. InHandbook of the

economics of education, eds. Eric Alan Hanushek and Finis Welch. Elsevier.

Cameron, A.C., & Trivedi, P. 2005.Microeconometrics: Methods and Applications. Cambridge

University Press.

Cooper, Harris, Jorgianne Civey Robinson, and Erika Patall. 2006. Does Homework Improve

Academic Achievement? A Synthesis of Research, 1987-2003.Review of Educational

Research 76(1): 1-62.

Eren, Ozkan, and Daniel Henderson. 2008. The Impact of Homework on Student

Achievement.Econometrics journal11(2): 326-348.

Eren, Ozkan, and Daniel Henderson. 2009. Are We Wasting Our Childrens time by giving

them more homework? Working Paper.


22/31

22

Hanushek, Eric. 1999. The Evidence on Class Size. InEarning and learning, eds. Susan E.

Mayer and Paul E. Peterson. Brookings Institution Press.

Hanushek, Eric, John Kain, Jacob Markman, and Steven Rivkin. 2003. Does Peer Ability Affect

Student Achievement?.Journal of Applied Econometrics 18(5): 527-544.

Hill, Lester, Jr. 1991. Effort and reward in college: a replication of some puzzling findings. In

Replication Research in the Social Science, ed. James W Neuliep. Newbury Park, CA: Sage,

p. 139-56.

Kohn, Alfie. 2006. The homework myth. Da Capo Press.

Kralovec, Etta, and John Buell. 2001. The End of Homework. Beacon Press.

Loveless, Tom. 2003. How Well are American Students Learning? Brookings Institution.

Manski, Charles F. 1993. Identification of Endogenous Social Effects: The Reflection

Problem. The Review of Economic Studies 60(3): 531-542.

Neilson, William. 2005. Homework and performance for time-constrained students.

Economics Bulletin 9(1): 1-6.

Parker, Angie. 1999. A Study of Variables that Predict Dropout from Distance Education..

International Journal of Educational Technology 1(2): 1-10.

Rau, William, and Ann Durand. 2000. The Academic Ethic and College Grades: Does Hard

Work Help Students to "Make the Grade"?. Sociology of Education 73(1): 19-38.


23/31

23

Schuman, Howard, Edward Walsh, Camille Olson and Barbara Etheridge. 1985. Effort and

Reward: The Assumption That College Grades are Affected by Quantity of Study. Social

Forces 63(4): 945-966.

Steptoe, Andrew, and Jane Wardle. 2001. Locus of control and health behaviour revisited: a

multivariate analysis of young adults from 18 countries.British Journal of Psychology 92(Pt

4): 659-672.

Stinebrickner, Ralph, and Todd Stinebrickner. 2008. The Causal Effect of Studying on

Academic Performance. The B.E. Journal of Economic Analysis & Policy 8(1).

Todd, Petra, and Kenneth Wolpin. 2003. On the Specification and Estimation of the Production

Function for Cognitive Achievement. The Economic Journal113(485): F3-F33.

Todd, Petra, and KennethWolpin. 2007. The Production of Cognitive Achievement in Children:

Home, School, and Racial Test Score Gaps.Journal of Human Capital1(1): 91-136.

Wang, Alvin, and Michael Newlin. 2000. Characteristics of Students Who Enroll and Succeed

in Psychology Web-based Classes.Journal of Educational Psychology 92(1): 137-43.


24/31

24

Tables

Table 1Selected Summary Statistics

Wave 1 Wave 2 Wave 3

Mean SD Mean SD Mean SD

Hours of mathematics homework per week 1.48 1.67 5.29 4.69 5.70 4.19

Hours of assigned homework per week 2.45 1.30 3.29 2.20 3.05 1.36

Mathematics test score 4.69 0.82 5.24 0.95 5.89 0.95

Class size 23.96 5.11 23.48 5.24 27.52 16.34

Teacher is certified in mathematics 0.85 0.36 0.99 0.07 0.99 0.07

Math teacher has 3 or fewer years of experience 0.09 0.29 0.11 0.31 0.04 0.18

Minimum annual teacher wage 28.77 5.13 29.49 5.38 31.28 6.94

Locus of control 0.09 0.57 0.06 0.61 0.14 0.62

Average of peers' test score 4.52 0.41 5.10 0.48 5.51 0.53

Average peers' homework time 5.35 1.54 7.28 2.32 13.01 3.32

Number of students 3704 3121 2052

Note. - This data is a sample of students from the National Education Longitudinal Study of

1988. Wave 1 corresponds to students in the 8th grade, wave 2 to the 10th grade, and wave 3 to

the 12th grade. All monetary variables are measured in thousands of dollars, and inflation

adjusted to 2005 dollars.


25/31

25

Table 2

First Stage Equation Estimation Results: Determinants of Homework Time

Corresponding column in table 4 3 4 5

Dependent variable HW HW2 HW HW2 HW HW2

Class size (in units of 10students)

-0.008 -0.166 -0.008 -0.178 -0.009 -0.198

(0.006) (0.110) (0.006) (0.109) (0.007) (0.140)

Certified teacher0.202 2.588 0.204 2.861 0.155 0.377

(0.207) (4.564) (0.207) (4.614) (0.264) (5.750)

Inexperienced teacher0.018 0.151 0.009 -0.108 0.043 0.858

(0.175) (3.451) (0.174) (3.436) (0.224) (4.667)

Minimum annual teachercompensation (in thousands)

-0.036 -0.623 -0.025 -0.465 -0.004 -0.153

(0.013) (0.266) (0.013) (0.268) (0.017) (0.347)

Class level - average 0.407 4.410 0.426 4.675 0.712 11.000(0.163) (3.190) (0.162) (3.170) (0.228) (4.523)

Class level - high0.900 12.970 0.928 13.430 1.258 21.110

(0.195) (3.957) (0.194) (3.944) (0.262) (5.362)

Class level - differs0.544 8.549 0.574 8.770 0.921 14.760

(0.209) (4.204) (0.209) (4.229) (0.288) (5.852)

Homework hours assigned bymath teacher*

0.163 2.377 0.157 2.252 0.082 0.918

(0.051) (0.981) (0.051) (0.982) (0.064) (1.268)

Locus of control*0.003 -3.513 -0.007 -3.649 -0.009 -4.840

(0.115) (2.743) (0.115) (2.748) (0.153) (3.831)

Assigned homework hourssquared*

-0.005 -0.070 -0.005 -0.063 -0.004 -0.055(0.003) (0.056) (0.003) (0.055) (0.003) (0.048)

Locus of control squared*0.226 5.839 0.242 6.076 0.346 9.642

(0.104) (2.261) (0.104) (2.273) (0.139) (3.195)

Observations 8770 8770 8770 8770 5453 5453

Student fixed effects yes yes yes yes yes yes

School level peer characteristics no no yes yes yes yes

Teacher level peer characteristics no no no no yes yes

Hansen J statistic (Chi-Sq p-value) 0.933 0.949 0.88

P-value for F-test of the jointimpact of excluded instruments 0.003 0.008 0.003 0.009 0.07 0.032

Note. - The asterisk denotes instruments excluded from the second stage regression. Standard

errors are shown in parentheses. All regressions include wave indicators. The sample includes

8877 observations on 3978 students. The first stage regressions are shown in table 4.


26/31

26

Table 3

Testing Instrument Validity

DependentVariable Independent Variable

Without StudentFixed Effects

With studentFixed Effects

Assigned HW Lagged Test Score0.223 0.052

(0.043) (0.194)

Locus of Control Lagged Test Score0.094 -0.028

(0.015) (0.050)

Assigned HW Locus of Control0.023

(0.049)

Note. - Standard errors are shown in parentheses. Each cell displays estimates from a separate

regression, each of which includes the full set of covariates included in the main specifications.

All standard errors are robust to the presence of arbitrary heteroskedasticity, and are clustered at

the level of the individual.


27/31

27

Table 4

Education Production Function Estimation Results

1 2 3 4 5

Estimation Method OLS OLS 2SLS 2SLS 2SLS

Hours of math homework perweek

0.069 0.014 0.259 0.260 0.237

(0.006) (0.003) (0.086) (0.088) (0.107)

Hours of math homework perweek squared and divided by 10

-0.003 -0.001 -0.011 -0.012 -0.009

(0.000) (0.000) (0.004) (0.005) (0.004)

Class size (in units of 10 students)-0.005 -0.001 -0.001 -0.001 -0.001

(0.001) (0.001) (0.001) (0.001) (0.001)

Certified teacher0.133 -0.026 -0.047 -0.031 -0.028

(0.032) (0.023) (0.033) (0.033) (0.041)

Inexperienced teacher-0.075 0.017 0.010 0.004 -0.007

(0.029) (0.018) (0.026) (0.025) (0.031)Minimum annual teachercompensation (in thousands)

0.003 0.002 0.005 0.003 0.000

(0.002) (0.001) (0.002) (0.002) (0.003)

Class level - average0.593 0.073 0.013 0.010 -0.031

(0.027) (0.018) (0.033) (0.033) (0.055)

Class level - high1.348 0.136 0.041 0.046 -0.012

(0.030) (0.021) (0.048) (0.048) (0.082)

Class level - differs0.557 0.095 0.046 0.031 -0.018

(0.035) (0.023) (0.038) (0.038) (0.067)

Marginal Effect of Homework -

evaluated at the mean

0.049 0.010 0.182 0.179 0.176

(0.004) (0.002) (0.059) (0.060) (0.082)

Observations 8770 8770 8770 8770 5453

Student fixed effects no yes yes yes yes

School level peer characteristics no no no yes yes

Teacher level peer characteristics no no no no yes

Note. - Standard errors are shown in parentheses. The dependent variable for each regression is

the mathematics achievement test score with a standard deviation of one. All regressions include

wave dummy variables. Column one is a pooled regression with the standard errors clustered at

the level of the individual. All standard errors are robust to the presence of arbitrary

heteroskedasticity. The first stage regressions are shown in table 2.


28/31

28

Table 5

Marginal Effect of Homework Time by Student Achievementand School Quality

Group 8th Grade ScienceTest Score

School Quality

Top Half0.148 0.119

(0.065) (0.046)

Bottom Half0.182 0.250

(0.023) (0.172)

Note. - Standard errors are shown in parentheses. The marginal effects are all evaluated at the

mean value of homework for the whole population. Each cell is the instrumental variables

estimate of the marginal effect of an hour of homework completed on mathematics test scores.

Teacher experience, certification, salary, class size, class level controls, peer effects, and wave

indicators were also included in each specification. Prior achievement is measured using the

students' 8th grade science test score. School quality is measured using the average mathematics

test score of other observed students in the same school. All standard errors are robust to the

presence of arbitrary heteroskedasticity.


29/31

29

Table 6

Marginal Effect of Assigned Homework on MathematicsAchievement by Prior Achievement and School Quality

Group 8th Grade ScienceTest Score

School Quality

Top Half0.007 0.008

(0.005) (0.006)

Bottom Half0.018 0.016

(0.006) (0.007)

Note. - Standard errors are shown in parentheses. For each result, a quadratic in assigned

homework is used to predict mathematics achievement in a student fixed effect regression

specification. The marginal effects are all evaluated at the mean value of assigned homework for

the whole population. Teacher experience, certification, salary, class size, class level controls,

peer effects, and wave indicators were also included in each specification. Prior achievement is

measured using the students' 8th grade science test score. School quality is measured using the

average mathematics test score of other observed students in the same school. All standard

errors are robust to the presence of arbitrary heteroskedasticity.


30/31

30

Appendix A: Variable Definitions

Mathematics test scores: The cognitive tests were given to students with each survey, raw scores

are not used, but instead the scores are normalized across wavesbefore sample restrictions

were imposed - with a standard deviation across all waves approximately equal to 1. The tests

were created to allow comparison across grades, and to accurately measure student achievement

even if students do not complete the entire exam.

Hours of mathematics homework: This variable measures the number of hours that the student

spends per week on mathematics homework. The NELS data records the hours of homework

completed in a series of categories. Each student is assigned the mean value for their category.

Class size: This is defined as the average size of the observed classes taken by the student in

each wave.

Teacher has certification in field: This variable equals one if the mathematics teachers surveyed

for this student to teach mathematics, and zero otherwise.

Inexperienced teacher: This indicator variable is equal to one if the teacher interviewed has three

or less years of experience. The source variables did not allow for separate indicators for one,

two, and three years of experience.

Minimum teachers wage:This variable comes from the school counselors survey, and records

the minimum annual wage for a teacher in the school that the student is attending. This variable

is adjusted so that it is measured in units of $1000 inflation-adjusted 2005 dollars.

Assigned homework: This is a continuous variable recording the average hours per week that the

students interviewed mathematics teacher assigned. The source variables were all continuous,

though the base year was a weekly variable and the second and third waves asked for daily

amounts. I multiplied the daily homework amounts by 5 to get the weekly amount.


31/31

Locus of control: This is a composite of three questions, which measures the degree to which the

student has an internal locus of control. Students with an internal locus believe that their

actions and choices can shape their future, where students with an external locus believe external

events will be the primary determinants of what their future is like. A higher number indicates

that the students locus is more internal. The questions ask the student to agree or disagree (five

point scale) with the following statements:

In my life, good luck is more important than hard work for success.

Every time I try to get ahead,something or somebody stops me.

My plans hardly ever work out, so planning only makes me unhappy.

Average of peersTest scores: These variables are the average mathematics test score of all other

observed students in the same school (or who have the same teacher) in the same year. This

variable was created prior to imposing sample restrictions.

Average of peers homework time: These variables are the average total hours of homework

reported by all other observed students in the same school (or who have the same teacher) in the

same year. This variable was created prior to imposing sample restrictions.

Class Level Indicator Variables: The three class level indicators come from a question of each

teacher that asks: Which of the following best describes the achievement level of the students in

this class compared with the average 8th (10th, 12th) grade student in this school? where the

options for response are Higher Levels, Average Levels, Lower Levels, or Widely Differing.

Documents

McMullen 2010 - The Impact of Homework Time on Academic Achievement