1
The voices of teachers are difficult to quantify in education policy discussions – thus they remain subject to policies with few opportunities to provide feedback. Policymakers have employed a suite of teacher policies centered on college-and-career readiness (CCR) standards. But evidence for these policies remains mixed, and school autonomy advocates point to the finding that district-level policy is responsible for only 1-2 percent of the total variation in student achievement (Chingos, Whitehurst & Gallaher, 2015). This paper matches teacher surveys of policies with the Stanford Education Data Archive (SEDA) to examine relationships between policy and student achievement (Reardon et. al, 2017). I measured these environments using a “policy attributes theory” (Porter et. al, 1988) in three states - Texas, Ohio and Kentucky – and found that the consistency of teacher policies predicts student achievement. When analyzing urban districts separately, stability was the only positive predictor. Number of districts = 165. Includes state fixed effects, percentage of ELLs and per pupil expenditures. Policy Attribute Base Coefficient Interaction with Poverty Specificity -0.774 3.303 Consistency 2.260** -8.299** Authority -0.166 1.1927 Power -0.569 1.498 Stability -0.432 r2 0.143 Consistency Matters: Teacher Policy, Poverty, and Student Achievement Adam Kirk Edgerton, Ph.D. Student, Education Policy, University of Pennsylvania GSE § When teachers feel that policies and materials are working in concert with each other, student achievement appears to benefit, compared to specific curriculum, whether the standards are appropriate, and rewards or sanctions. § Stability may be more important than the policies themselves in urban districts. § Strong accountability policies or specific curricula do not show relationships to district-level achievement. To what extent do teacher policy environments moderate the relationship between district poverty and student achievement? This research was supported in part by Grant R305C150007 from the Institute of Education Sciences in the U.S. Department of Education to the University of Pennsylvania. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Institute of Education Sciences or the U.S. Department of Education. Chingos, M.M., Whitehurst, G.J. & Gallaher, M.R. (2015). Porter, A. C., Floden, R., Freeman, D., Schmidt, W., & Schwille, J. (1988). Porter, A. C. (1994). Reardon, S., Ho, A., Shear, B., Fahle, E., Kalogrides, D., & DiSalvo, R. (2017). § Specificity = How specific is the guidance that teachers receive on the CCR standards? (pacing, curriculum, etc.) § Consistency = How aligned are the standards-based policies that teachers experience? § Authority = How much do teachers believe that the standards are appropriate, rigorous and flexible enough for all students? § Power = To what extent do teachers experience rewards and sanctions for standards implementation? § Stability = How long do teachers think the policies will last? Please indicate your opinion on the degree to which the following were aligned to the CCR standards for (ELA or math): a) The (ELA or math) sections of the test b) District-mandated summative assessments c) Formative or diagnostic assessments developed by school d) Formative or diagnostic assessments used district-wide e) Textbooks used in your school f) Curriculum selected or developed by your district g) State-developed or organized professional development h) District-developed or organized professional development i) Administrator feedback provided to you from classroom observations (1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, 4 = strongly agree) § Poverty is a larger magnitude predictor for achievement than FRL is this more evidence for decreasing the use of the FRL measure? § There is noise/measurement error In the survey metric as the sample design is stratified by state, not district rural districts have as few as 2 teacher respondents. § Metaregression corrects for some of the measurement error in the output, but r2 is low. Districts are the unit of analysis (not weighted by enrollment). -10 -8 -6 -4 -2 0 2 4 POV/FRL Consistency Interaction Term Predictive Coefficient for Achievement Poverty FRL Poverty Coefficient Stability Coefficient -12.779* (5.542) 3.177* (1.339) FRL Coefficient Stability Coefficient -6.095* (2.192) 3.891* (1.327) 39 districts are included in the sample. Teacher responses in these districts are greater than 30.

Consistency Matters: Teacher Policy, Poverty, and Student ...€¦ · and Student Achievement Adam Kirk Edgerton, Ph.D. Student, Education Policy, University of Pennsylvania GSE §

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Consistency Matters: Teacher Policy, Poverty, and Student ...€¦ · and Student Achievement Adam Kirk Edgerton, Ph.D. Student, Education Policy, University of Pennsylvania GSE §

Thevoicesofteachersaredifficulttoquantifyineducationpolicydiscussions–thustheyremainsubjecttopolicieswithfewopportunitiestoprovidefeedback.Policymakershaveemployedasuiteofteacherpoliciescenteredoncollege-and-careerreadiness(CCR)standards.Butevidenceforthesepoliciesremainsmixed,andschoolautonomyadvocatespointtothefindingthatdistrict-levelpolicyisresponsibleforonly1-2percentofthetotalvariationinstudentachievement(Chingos,Whitehurst&Gallaher,2015).ThispapermatchesteachersurveysofpolicieswiththeStanfordEducationDataArchive(SEDA)toexaminerelationshipsbetweenpolicyandstudentachievement(Reardonet.al,2017).Imeasuredtheseenvironmentsusinga“policyattributestheory”(Porteret.al,1988)inthreestates-Texas,OhioandKentucky–andfoundthattheconsistencyofteacherpoliciespredictsstudentachievement.Whenanalyzingurbandistrictsseparately,stabilitywastheonlypositivepredictor.

Numberofdistricts=165.Includesstatefixedeffects,percentageofELLsandperpupilexpenditures.

Policy Attribute Base Coefficient Interaction with

Poverty

Specificity -0.774 3.303

Consistency 2.260** -8.299**

Authority -0.166 1.1927

Power -0.569 1.498

Stability -0.432

r2 0.143

ConsistencyMatters:TeacherPolicy,Poverty,andStudentAchievementAdamKirkEdgerton,Ph.D.Student,EducationPolicy,UniversityofPennsylvaniaGSE

§  Whenteachersfeelthatpoliciesandmaterialsare

workinginconcertwitheachother,studentachievementappearstobenefit,comparedtospecificcurriculum,whetherthestandardsareappropriate,andrewardsorsanctions.

§  Stabilitymaybemoreimportantthanthepoliciesthemselvesinurbandistricts.

§  Strongaccountabilitypoliciesorspecificcurriculadonotshowrelationshipstodistrict-levelachievement.

Towhatextentdoteacherpolicyenvironmentsmoderatetherelationshipbetweendistrictpovertyandstudentachievement?ThisresearchwassupportedinpartbyGrantR305C150007fromtheInstituteofEducationSciencesintheU.S.DepartmentofEducationtotheUniversityofPennsylvania.ThecontentissolelytheresponsibilityoftheauthorsanddoesnotnecessarilyrepresenttheofficialviewsoftheInstituteofEducationSciencesortheU.S.DepartmentofEducation.

Chingos,M.M.,Whitehurst,G.J.&Gallaher,M.R.(2015).Porter,A.C.,Floden,R.,Freeman,D.,Schmidt,W.,&Schwille,J.(1988).Porter,A.C.(1994).Reardon,S.,Ho,A.,Shear,B.,Fahle,E.,Kalogrides,D.,&DiSalvo,R.(2017).

§  Specificity=Howspecificistheguidancethatteachers

receiveontheCCRstandards?(pacing,curriculum,etc.)§  Consistency=Howalignedarethestandards-basedpolicies

thatteachersexperience?§  Authority=Howmuchdoteachersbelievethatthestandards

areappropriate,rigorousandflexibleenoughforallstudents?

§  Power=Towhatextentdoteachersexperiencerewardsandsanctionsforstandardsimplementation?

§  Stability=Howlongdoteachersthinkthepolicieswilllast?

PleaseindicateyouropiniononthedegreetowhichthefollowingwerealignedtotheCCRstandardsfor(ELAormath):a)The(ELAormath)sectionsofthetestb)District-mandatedsummativeassessmentsc)Formativeordiagnosticassessmentsdevelopedbyschoold)Formativeordiagnosticassessmentsuseddistrict-widee)Textbooksusedinyourschoolf)Curriculumselectedordevelopedbyyourdistrictg)State-developedororganizedprofessionaldevelopmenth)District-developedororganizedprofessionaldevelopmenti)  Administratorfeedbackprovidedtoyoufromclassroom

observations(1=stronglydisagree,2=somewhatdisagree,3=somewhatagree,4=stronglyagree)

§  Povertyisalargermagnitudepredictorforachievementthan

FRL–isthismoreevidencefordecreasingtheuseoftheFRLmeasure?

§  Thereisnoise/measurementerrorInthesurveymetricasthesampledesignisstratifiedbystate,notdistrict–ruraldistrictshaveasfewas2teacherrespondents.

§  Metaregressioncorrectsforsomeofthemeasurementerrorintheoutput,butr2islow.Districtsaretheunitofanalysis(notweightedbyenrollment).

-10

-8

-6

-4

-2

0

2

4

POV/FRL Consistency InteractionTerm

Pred

ictiv

eCo

efficientfo

rAc

hievem

ent

PovertyFRL

Poverty Coefficient Stability Coefficient

-12.779* (5.542) 3.177* (1.339)

FRL Coefficient Stability Coefficient

-6.095* (2.192) 3.891* (1.327)

39 districts are included in the sample. Teacher responses

in these districts are greater than 30.