Wisconsin Scholars Longitudinal Study: An Introduction SARA GOLDRICK-RAB & DOUGLAS N. HARRIS Co-Directors University of Wisconsin-Madison

Wisconsin Scholars Longitudinal Study:An Introduction

SARA GOLDRICK-RAB & DOUGLAS N. HARRISCo-DirectorsUniversity of Wisconsin-Madison

WWW.FINAIDSTUDY.ORG

Thanks for the Extensive Support!

• Funders: Bill and Melinda Gates Foundation, William T. Grant Foundation, Spencer Foundation• Partners: University of Wisconsin System, Wisconsin

Technical College System• Board of directors of the Fund for Wisconsin Scholars• Our staff, past, present, and future• And especially our project manager, Alison Bowman

Policy Context

• The nation faces a college completion problem• Completion rates are especially poor among

students from low-income families• Governments spend $155B+ on need-based

financial aid each year (inc. grants & loans)• To what degree is aid exerting an independent

contribution to degree completion rates?• Is it a cost-effective contribution?• Through what mechanisms do effects arise?• For which students is aid most effective?

How Financial Aid Could Help Students

• The conditions required to get the aid could provide motivation• “I will continue to enroll in college in order to get this

money.”• “I will attend full-time rather than part-time in order to get

this money.”

• The income could be used in positive or negative ways• “This money helps me work a little less and study a little

more.”• “This money makes me less worried that I can get through

college.”• “With this money, I’ll stay in a dorm where I can hang out

with my friends.”

Prior Research

• Studies suggest modest positive impacts of financial aid on college attendance and persistence for the average student• The effects seem to vary depending on the conditions

associated with the program• They also depend on who the program is serving• Isolating a causal effect of financial aid is particularly

difficult

Grant vs. No Grant Comparison

Ways to Evaluate the Effects of a Grant Program

ddf 100 students sign up to receive a grant

Grant vs. No GrantGrant

100 students sign up

No Grant 1000s of students don’t sign up


College Completion Rate= 70%

No Grant College Completion Rate= 60%



No Grant College Completion Rate= 60%

Students receiving the grants have higher college completion rates… Can we attribute that difference

to the grants?

Scholarship vs. No ScholarshipScholarship


No Scholarship College Completion Rate= 60%

The observed differences in completion rates could be due to unobserved differences among

students—which led them to get the grants.

Grant vs. No Grant

• This method compares apples to oranges• The reason for not getting the grant could be

associated with the reason for not finishing college (“selection bias”)• If we know what the reason is, we can

“control” for it• If we don’t know what it is, or can’t observe it

in the data we have, we can’t control for it• The estimated impact of the grant mixes up

these factors

Fund for Wisconsin Scholars

• A new, generous grant program• Allocates grants with a method that facilitates

rigorous estimation of impacts• Willing to participate in an evaluation

• An uncommon opportunity to help large numbers of Wisconsin college students—while generating research knowledge needed to help students nationwide


• Grants to students enrolled in UW System & the Wisconsin Technical College System• Began making awards in fall 2008• Eligible students graduated from Wisconsin

public high schools, enrolled in college within 3 years of graduation, for the first-time, and received a Pell Grant• Had to have at least $1 of unmet need after all non-

repayable sources of aid were accounted for• Had to enroll full-time (12 credits) by the date of

record


• Grant amounts:• $1,800 per year for students at 2-year colleges• $3,500 per year for students at universities

• Terms of renewal:• Continued receipt for up to 5 years• Transferrable among publics• Must register for 12 credits by the start of each new

term• Maintain ‘satisfactory academic progress’ (~C

average)• Terms are very comparable to the federal Pell Grant


• Identification of eligible students:• Fall of first year of college (after enrollment)• FFWS works with Wisconsin’s financial aid officers• Students don’t have to sign up for a “chance” to get the grants;

they have already done the hard work of completing the FAFSA• Selection process:• Students are chosen (2-year and 4-year separately)• Potential recipients are notified• Students respond and verify their eligibility

• Award process:• After verification, payments are disbursed via financial aid

offices each term• First payment arrives by 2nd semester, year 1

Random Assignment of FFWS Grants

All eligible students have an equal chance of receiving a grant

A coin toss helps ensure recipients and non-recipients are equivalent groups

FFWS Grant vs. No FFWS Grant

• This method compares apples to apples• The reason for not getting the grant could NOT be associated

with the reason for not finishing college• The estimated impact of the FFWS grant is clean– if we

observe differences in students’ outcomes they are due to the grant

How Students Get the FFWS Grant

• They are enrolled and already have financial aid• The FFWS grant must go to the financial aid

office & by law the student’s package must be adjusted if receiving the grant means their financial aid exceeds the institution’s cost of attendance• Usually this means reducing loans and/work-study

money; schools agreed not to reduce institutional aid

• So they get the grant as loan reduction, cash refund, or a combination, depending on the size and composition of their initial aid package

Wisconsin Scholars Longitudinal Study

• What are the average effects of the FFWS grant on college attainment?• For which students is the grant most effective?• Under what conditions is the grant most

effective? (e.g. loan vs. cash, type of college)• How does the FFWS grant affect students?• How does it change how they spend their time?• In what ways does it alter their relationships

with other people?• How does it affect how they think and feel?


• Includes 3,000 students who were eligible for the FFWS grant in Fall 2008• 1,200 students randomly chosen to receive the

grant• 1,800 students not chosen (at random) to

receive the grant• The sample is split between universities and 2-

year colleges• We also plan to include students who will be

eligible for the FFWS grant in Fall 2012


• We observe students’ outcomes with the following kinds of data:• National Student Clearinghouse (enrollment

anywhere)• College transcripts• Surveys (administered yearly, by mail)• Interviews (conducted in-person, every

semester, with a stratified sample of 36 students at 4 universities)

• We also plan to consider employment and earnings outcomes

Questions?

• About the study’s research questions?• Design?• Data?

Education Impacts of the Wisconsin Scholars Grant on University Students:2008-2011

SARA GOLDRICK-RAB, DOUGLAS N. HARRIS,JAMES BENSON, & ROBERT KELCHENUniversity of Wisconsin-Madison

WWW.FINAIDSTUDY.ORG

http://WWW.FINAIDSTUDY.ORG/

Sample

• 1,500 students who enrolled at one of Wisconsin’s 13 public universities in Fall 2008• All students in this analysis received a Pell Grant

in that term• We compare the outcomes of 600 students

randomly selected to be offered the FFWS grant and 900 eligible students who were not chosen

Data

• Enrollment: National Student Clearinghouse (2008-2011)• Obtained for all students • Captures enrollment anywhere in the U.S., if NSC can

find a “match.” Our “match” rate ~98%.

• Credits & GPA (2008-2010)• Financial aid packages (2008-2010)• Student surveys (2008, 2009)• In-depth interviews (50 students, every 6 months, 2008-

11)

Were Students Offered and Not Offered the Grant Similar Before the Grant Was Awarded?

Full Sample Non-Recipient

Mean

Treatment Difference

P-Value

% Female 57.3 56.7 1.0 .711

% Minority 24.6 24.0 1.1 .681

Average Age 18.2 18.2 0.0 .942

% First Gen 53.4 53.5 -0.2 .958

% Dependent 97.3 97.0 0.5 .572

EFC ($) 1,633 1,603 53 .669

% $0 EFC 30.6 31.9 -2.3 .362

Parent AGI ($) 29,963 29,403 1,014 .314

% 1st time in college 95.9 95.7 0.5 .681

Terms prior enroll 1.8 1.8 -0.1 .856

N 1500 900 600

Question 1: How did the FFWS grant affect students’ financial resources?

Impact on Aid Package: Year 1 (2008-2009)

CONTROL TREATMENT IMPACT ($3,500)

Total Aid $11,426 $1,665***

Pell (%) 99.8 - 0.1

State Grant (%) 99.0 - 3.7 ***

SEOG (%) 63.6 -9.8***

ACG (%) 80.4 0.3

Institutional Aid (%) 54.7 1.6

Work Study (%) 18.3 - 5.0 *

Sub. Staff (%) 77.9 - 11.1***

Unsub Staff (%) 38.9 - 3.8

Total loans ($) 3428.3 -909.5

93% of treatment group received FFWS this year.Aid amounts are unconditional on receipt—impacts reflect supplanting + student decisions

Impact on Aid Package: Year 2 (2009-2010)

CONTROL TREATMENT IMPACT ($3,500)

Total Aid $10,082 $814.3 **

Pell (%) 74.3 - 0.6

State Grant (%) 67.2 0.5

SEOG (%) 39.2 1.2

ACG (%) 24.1 1.5

Institutional Aid (%) 48.1 -5.4

Work Study (%) 16.3 -3.6

Sub. Staff (%) 66.7 -7.2*

Unsub Staff (%) 45.6 -9.8**

Total loans ($) 3581.2 -830.9

67% of treatment group received FFWS this year. Aid amounts are unconditional on receipt.

Summary

• About 1/3 of the amount of the FFWS grant was used to reduce students’ loans and work-study (as well as replace some state grant aid)• After two years, students offered the FFWS grant had

about $1,800 less debt than students not offered the grant

Question 2: How did the FFWS grant affect academic outcomes?

Average Impacts of the Grant on Enrollment:2008-2011

Control Mean Treatment Effect

Total # terms enrolled (f/s, %) 5.192 0.05 (.09)

Ever enrolled summer (%) 21.4 0.8 (2.6)

Ever enrolled winter (%) 5.8 3.9 (1.7) **

Ever transferred (%) 23.7 -0.4 (2.7)

Ever attended 2-year college (%)

14.4 -0.4 (2.2)

Completed associate’s degree (%)

3.1 0.4 (1.1)

These are average effects, comparing FFWS recipients vs. control group

Notes: *** p<.01, **p<.05, * p<.10

Average Impacts of the Grant on Credits:2008-2010

Control Mean Treatment Effect

Credit Accumulation

Average credits completed 46.9 0.9 (1.0)

Earned 1-29 credits (%) 18.2 0.4 (2.4)

Earned 30-47 credits (%) 16.0 2.6 (2.4)

Earned 48-59 credits (%) 42.3 - 8.2 (3.1) ***

Earned 60+ credits (%) 22.2 6.3 (2.8) **

Progress toward 4-year Degree

60+ credits, 2-2.5 GPA 2.2 0.0 (0.9)

60+ credits, 2.5-3.0 GPA 6.8 -0.5 (1.6)

60+ credits, 3.0-3.5 GPA 8.5 4.7 (2.0) **

60+ credits, 3.5+ GPA 4.3 2.4 (1.4) *

Notes: *** p<.01, **p<.05, * p<.10

Effects of the Grant Varied

• We compared the effects based on a student’s propensity to persist, using pre-FFWS factors:• Age, parental education, race/ethnicity, gender, number of

siblings• Family income, assets, expected family contribution• Years of high school math and science, GPA, ACT score,

hours worked in high school, receipt of the Academic Competitive Grant

• Help from parents completing FAFSA• # credits registered for 2nd week of freshman year• Receipt of the Supplemental Educational Opportunity Grant• College attended

Propensity to Persist

• Overlapping disadvantages means that no single factor describes the category but parental education and ACT scores are some of the defining characteristics• High (94%): Parents typically have at least a bachelor’s

degree, students have higher than average test scores• Middle (82%): Parents typically have a high school

education, students have moderate test scores• Bottom (55%): Parents typically have attended some

college but not necessarily a university, students have lower test scores

Impact of the Grant on Enrollment by Propensity to Persist

2008-2009 2009-2010 2010-2011

Top: Control 99 94 94

Top: FFWS 94 83 79

Middle: Control

96 85 82

Middle: FFWS

93 84 81

Bottom: Control

94 62 55

Bottom: FFWS

97 83 72

-15

-1

+ 17

Impact of Grant on Total Number of Enrolled Terms: 2008-2011

Bottom tercile Middle tercile Top tercile0

1

2

3

4

5

6

TreatmentControl

p-=.001


Impact of Grant on Number of Attempted Credits: 2009-2010


5

10

15

20

25

30

TreatmentControl

p-=.034


Note: NoImpacts on AttemptsIn ‘08-09

Impact of Grant on Number of Completed Credits: 2008-2010


10

20

30

40

50

60

TreatmentControl

p-=.331


Impact of Grant on Percent Completing 1-29 Credits: 2008-2010


100

Treatment

Control

p-=.025


Impact of Grant on Percent Completing 60+ Credits: 2008-2010


100

Treatment

Control

p-=.424


Summary

• On average, the grant appeared to accelerate time-to-degree for some students. This is promising since less than 30% of Pell recipients in Wisconsin complete bachelor’s degrees in 4 years.

• The grant appeared most effective for the students who were the least likely to persist in college

Are FFWS Requirements Driving the Effects?• We think this is unlikely

• The requirements are not unusual• Students think of the FFWS grant as simply as part of their

aid package – it requires exactly what Pell does (12 credits, satisfactory academic progress)

• Many think the FFWS grant requires a B average– but they think the Wisconsin Higher Education Grant does too

• Awareness of the grant and its requirements is limited:• In Fall 2008 survey just 37% of those awarded the grant

reported getting it– a year later, just 49% of those awarded it seemed to know they had it

Decisions About Full-Time Enrollment

• Interviews do not indicate that students took specific grants into account when making decisions about the number of credits to take• But they did consider overall financial constraints and the

needs of their family members• More important for determining the # credits was their

perception of how well they were doing in school

• There was an overall lack of awareness of how number of credits relates to time-to-degree• Lots of shifting from “4-year track” (15+ credits) to “5-

year track” (12-14) credits, and sometimes to <12 too

Are FFWS Resources Driving the Effects?

• Resources could have many different types of effects

• For example, increased resources could allow students to substitute study time for work time

• The increased resources could also help students meet their other obligations, for example to family members, helping them stay enrolled

• The resources could prove necessary for continued enrollment, but insufficient to help them manage a higher credit load

Who Helped Students Make Decisions?

• Students who were unlikely to persist in college were first-generation students with close family ties:• Their families may have worked extra hard to help them

succeed in college, and they may have made decisions about college (including how to spend $) with their parents (Minikel-Lacocque & Goldrick-Rab, 2011)

• Students who were very likely to persist in college came from families where attending college was more normative, these were “emerging adults”:• Students may have been afforded more

independence about how to use the grant

Policy Implications

• Targeting of social programs can maximize their cost-effectiveness• The challenge lies in identifying the best ways to target• It can be difficult to find politically feasible approaches

Policy Implications

• Costs of the Pell Grant are substantial and growing (~$20-40B)• The Obama administration wants to keep the grant maximum

high while finding other places to cut• One proposal is to require students to take 15 credits per term

(instead of 12) to get the maximum Pell ($5,500)• Only 31% of Pell recipients today take 15+ credits per term—most

are juniors or seniors.• This change means the Pell would be cut by $1000 or more for

the 41% of Pell recipients currently taking 12-14 credits per term• Assuming some of these succeeded in registering for 15+ credits,

savings would be approximately $1-2 billion per year– more, if those at 12-14 credits drop down—or dropout

• Cost-effectiveness of the approach is unclear

Questions?

Visit www.finaidstudy.org for a copy of the paper

http://www.finaidstudy.org/

Documents

Wisconsin Scholars Longitudinal Study: An Introduction SARA GOLDRICK-RAB & DOUGLAS N. HARRIS Co-Directors University of Wisconsin-Madison