Upload
anissa-sparks
View
212
Download
0
Embed Size (px)
Citation preview
Evaluation of the DC Opportunity Scholarship Program: Final Report
Conducted by Westat, University of Arkansas,
Chesapeake Research AssociatesPresented by
Patrick J. Wolf, Principal InvestigatorIES Research Conference
National Harbor, MarylandJune 30, 2010
Overview of Program Part of 3-sector strategy of urban education reform Student Eligibility
Grade K-12, residents of DC Family income ≤ 185% Federal poverty line Priority to students at schools in need of improvement (SINI)
Scholarship Up to $7,500 to cover tuition, school fees, and transportation Renewable for up to 5 years
Participation (2004-2009) 68 of the 88 general purpose private schools in DC 5,547 eligible applicants; 3,738 scholarship recipients; 2,881
initial scholarship users Closed to new participants in spring of 2009
Overview of Study Design
Lottery-based randomized control trial Sample: 2,308 eligible applicants in 2004 and 2005 Data:
Evaluation-administered SAT-9 Surveys: parents, students, principals
Analysis: impact of offer, use of scholarship, attendance at private school
Reports: 2 describing program applicants and our sample 4 estimating impacts (after 1, 2, 3, and 4+ years)
Scholarship Use Over 4+ Years
Just over one-fourth of students offered a scholarship consistently used it throughout the study
Participation drop-off (net): averaged 22% per year Top reasons for dropping out of OSP
Child got into a charter school (22%)
Lack of space (for students transitioning from K-8) (19%)
Moved out of DC (15%)
Transportation problems (14%)
Students more likely to drop out of OSP: lower initial test scores, older grade levels, male, with special needs, more siblings
Treatment Group Scholarship Use by Final Year
Partially used51%
Consistently used27%
Never used22%
In 2008-09
Not in 2008-09
Key Findings
OSP significantly raised graduation rate
No conclusive evidence OSP affected achievement over longer-run In final year, no statistically significant impacts for
students overall, SINI, lower performing, males Third year overall reading gains didn’t persist Impacts on reading test scores for some subgroups, but
results may be due to chance
Positive impacts on ratings of safety and satisfaction from parents but not students
High School Graduation Rates, Overall Sample & SINI Subgroup, Parent Reports, 2008-09
ITT
IOT
ITT Impacts on Reading Achievement Overall, by Years After Application
-2.41
-0.53
0.89
-0.13
1.03
3.17
4.46*3.90
4.46
6.88
8.03 7.94
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5After one year After two years After three years After at least four years
* Statistically significant at the 95 percent confidence interval.
ITT Impacts on Math Achievement Overall, by Years After Application
- 0.26
- 3.07
- 2.42
- 3.01
2.74
0.23
0.81 0.70
5.74
3.53
4.034.41
-4
-3
-2
-1
0
1
2
3
4
5
6
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5After one year After two years After three years After at least four years
Parent and Student Reports of Safety and an Orderly School Climate, 2008-09
**Statistically significant at the 99 percent confidence level.
Parent and Student Reports of School Satisfaction, 2008-09
**Statistically significant at the 99 percent confidence level.
Context DC proficiency and NAEP scores up 20 percent 2005-2009 Only 3 percent of DC schoolchildren participated and little
evidence of competitive response from DC schools Reliable studies indicate graduation yields key benefits:
ETS study – increases earnings by $8,500/year, decreases unemployment by one-third
Census study – all else equal, graduates live 7 years longer than dropouts Belfield and Levin – each graduate reduces the cost of crime by $112,000 Cecelia Rouse – each additional graduate saves the nation $260,000
Study limitations: One study, one city, specific program design Both achievement and attainment analyses in final year based on
subgroups Some key outcomes based on parent and student reports