Square Peg in a Round Hole: Developing Student Retention Models in Programs Designed for Adult Learners

Square Peg in a Round Hole:Developing Student Retention Models in Programs Designed for Adult Learners

Rachael Denison

Northeastern University

2014 CALEM, All Rights Reserved.1

*Evaluation Keyword: rdenison

Agenda

• Intro• Context• Measuring Retention• Application of Retention Data• Further Analysis of Indicators• Actions/Interventions• Next Steps

2014 CALEM, All Rights Reserved.2

Traditional retention measures: Our square peg/round hole predicament...

Need• Mission-driven• Predict & stabilize

enrollments• Financial impact• Focused recruitment

strategies• Focused advising &

student support services

Challenges• Standard measures don’t

fit• Stop outs vs. drop outs• Work, family impacts• Application predictors

limited (no standard test scores)

4

There are many aspects to the retention conversation.

Measurement

Application of Retention

Rates• Forecasting• Budgeting• Internal

Benchmarking

Understanding Indicators

Action / Intervention

2014 CALEM, All Rights Reserved.

Square peg in a round hole: Measuring retention

6

Enrollment cycle – 8 points of entry

Fall 1Fall 2

Winter 1

Winter 2Spring 1

Spring 2

Summer 1

Summer 2


Fiscal Year

7

Flexible enrollment patterns of attendance

Fall 1

Fall 2

Spring 1

Spring 2


Fiscal Year

8

Retention Defined


Fiscal Year Cohort

Enrolled (or Graduated)

in Year 2

Retention Rate

9

Sample Report (masked data)


Retention & Graduation Rates

Initial Year 2nd Year 3rd Year

Cohort Starting Size Returning Size Returned % Total

GraduatingTotal %

GraduatingTotal % Returning

or GraduatingReturning

Size Returned % Total Graduating

Total % Graduating

Total % Returning or Graduating

FY05 115 50 43% 3 3% 46% 35 30% 7 6% 37%

FY06 110 65 59% 2 2% 61% 36 33% 9 8% 41%

FY07 285 175 61% 12 4% 66% 112 39% 42 15% 54%

FY08 440 290 66% 18 4% 70% 172 39% 82 19% 58%

FY09 500 345 69% 32 6% 75% 214 43% 98 20% 62%

FY10 850 565 66% 96 11% 78% 290 34% 288 34% 68%

FY11 830 475 57% 149 18% 75% 241 29% 302 36% 65%

FY12 720 450 63% 155 22% 84% 219 30% 314 44% 74%

FY13 800 540 68% 122 15% 83%

FY14 795

10

Weekly Monitoring – Year 1 to Year 2


ProgramFY13 Starting

Cohort3 Yr Avg

Retention RateReturning

Enrolled CountReturning

Retention RateProgram 1 60 86.2% 45 75.0%Program 2 180 85.4% 160 88.9%Program 3 40 91.9% 30 75.0%Program 4 15 84.4% 14 93.3%Program 5 40 75.9% 30 75.0%Program 6 55 86.1% 45 81.8%

Grand Total 390 85.0% 324 83.1%

Year 1 - Year 2 Retention by Program

Fiscal Year 2013 Cohort Tracking - Progress to Fiscal Year 2014Progress is tracked by starting program in FY13 to enrollment in ANY graduate course in FY14 (including

certificates and non-degree)

Data current as of December 16, 2013

11

Weekly Monitoring – Term 1 to Term 2

ProgramFL13 Starting

Cohort

3 Yr Avg Retention Rate

(FL-WN)WN14 Current Enrolled Count

Current FL-WN Retention

RateProgram 1 10 -- 8 80.0%Program 2 25 93.3% 22 88.0%Program 3 60 89.1% 53 88.3%Program 4 35 94.8% 33 94.3%Program 5 10 80.0% 6 60.0%Program 6 20 81.4% 14 70.0%

Grand Total 160 87.6% 136 85.0%

Term 1 - Term 2 Retention by ProgramFall Cohort Tracking - Progress to Winter

Progress is tracked by starting program in Fall 2013 continuing to take a course within the same degree program in Winter 2014


12

Time to Completion


13

Foundation Year – Measuring progress to degree completion


• One year program, encourages students to “transfer out” after year 1

• Goal is degree completion, system persistence (not institutional retention)

14

The retention conversation…

Measurement

Application of Data:

Forecasting, Budgeting,

Internal Benchmarking

Understanding indicator



15

Retention Data Used to Inform Budget Planning


Cohort Size

Retention

Rate

Avg. Credit Hour Load

Credit Hour

Projection

Annual Retention Rate for continuing student impact

Quarterly Retention Rates for new student impact

FY15 Continuing Student Credit Hour

Generation

Students in Year 3

Students in Year 4 or more

FY12 Cohort

(or before)

FY13 Cohort

Students In Year 2

FY14 Cohort

Retention Data Used to Inform Budget Planning

Time to Completion

17

Retention Data Used for Internal Benchmarking

ProgramLeads/ Year

Avg Apps/ Year

App Growth

Conversion Rate

Yield Rate

Retention Rate (Y1 to Y2)

3-Year Completion

RateActive

Students

Program 2 10,433 743 -28% 2.1% 68.2% 85% 76% 688

Program 3 3,992 441 35% 5.1% 78.9% 93% 75% 671

Program 4 1,010 401 23% 7.1% 85.9% 84% 59% 549

Program 5 1,369 313 9% 11.0% 81.5% 86% 62% 453

Program 6 1,398 167 21% 6.7% 85.7% 89% 81% 273

Program 7 1,142 117 14% 7.0% 85.7% 89% 65% 212

Program 8 1,052 100 -8% 4.6% 82.2% 87% 70% 156

Program 9 346 123 23% 13.8% 66.3% 89% 77% 153


Retention and the enrollment lifecycle: it’s about more than new enrollments.

ENROLLMENT LIFECYCLE REPORT – SAMPLE

LEADS APPLICATIONS NEW STUDENTS CONTINUING STUDENTS QUARTER HOURS

Fall Term Leads Submitted Apps

Projected Submitted

Apps Completed Apps

Projected Completed

Apps New Enrolled

Projected New

Students

Avg. QH Load -

New Enr. Stu.

Continuing Enrolled

Avg. QH Load - Cont.

Enr. Stu.Quarter Hours -

FallQuarter Hours YTD

Annual

Eng MQEC Acc

EC Qual Act Goal

% of Goal

Proj Sub

% of Goal Act Goal

% of Goal Proj

% of Goal Act Goal

% of Goal Proj

% of Goal Act Exp Act Exp Act Exp Act Goal

% of Goal Act Goal

% of Goal

Program 1 179 13 4 4 12 13 92% 12 94% 7 7 100% 10 143% 2 5 40% 7 147% 8.0 7.5 10 3 6.0 7.6

12,021 13,148 91% 15,522 42,456 37% Program 2 1,000 42 26 28 140 122 115% 142 117% 121 104 116% 117 112% 78 81 96% 93 115% 8.8 8.6 169 107 7.6 7.3

Program 3 831 22 20 29 59 78 76% 60 77% 55 70 79% 49 70% 46 56 82% 40 72% 7.8 7.5 120 85 7.1 7.0

Program4 900 141 73 58 218 329 66% 221 67% 182 266 68% 182 68% 139 193 72% 135 70% 8.3 9.4 328 254 8.4 8.1

• Four statuses (levels of engagement/score)Workable Leads

• Submitted, completed, acceptedApplications

• Average course loadNew Students

• Average course loadContinuing Students

• Term and YTDCredit Hours

Com

paris

on to

Goa

l and

Pr

ojec

tion

19


Measurement




Understanding indicators



20

Understanding Retention Indicators – Predictive Analytics

Application & Student Record Data

• Demographics• First Term Grades• Employment Status• Military Affiliation• Financial Aid Application


21

Analysis – Decision Tree

• Enters each variable by importance and provides statistically significant splits.

• Example: 84% baseline -> English

• Surprisingly, where English is not the primary language, the students do better. This is a possible overlap with International students.


22

Those without a W or an I had a success rate of 85%. However, those who earned a W or an I had a success rate of 48%.


Findings - W or I Grade in First Term

23


Measurement




Understanding indicator



24

Action & Intervention

Targeted Outreach

Early Alert/Early Warning

Informing & Empowering Faculty

Global Student Success

Tutoring

Student Appreciation Week

Passport to Success Program


Next Steps

Further build out of predictive analytics• Overlay

admissions, LMS, early alert, call center data

Monitor other measures of student progress to completion• Within term

(grades)• Withdrawal

rates

Implement informed retention strategies• Measure

impact of strategies

26

Questions & Discussion


27

Rachael Denison

[email protected]

Northeastern University


Evaluation Keyword: rdenison

Documents

Square Peg in a Round Hole: Developing Student Retention Models in Programs Designed for Adult Learners