Proprietary - Not for duplication

American Public University System

Comprehensive Assessment of Student Retention in Online

Learning ProgramsWally Boston, Ed.D. and Phil Ice,

Ed.D.

October 19, 2010

The State of Online Learning

Approximately 4 million students taking online courses in Fall 2008 with projected 12.9% growth rate.

Outpacing face-to-face growth by 6 to 1.

US Department of Education meta-analysis – online is more effective than face-to-face (http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf)

% Growth in U.S. Online Enrollments

Retention

A concern in higher education since the late 1800’s. High School GPA, SAT and/or ACT scores, etc.

(background characteristics) most reliable predictors of retention; key reason why most selective colleges have highest graduation rates.

Tinto, Astin, Braxton and others have demonstrated the importance of academic and social integration for student persistence.

Measures of academic and social integration well defined in the face-to-face setting – not in online.

Online dropout rates much higher than face-to-face – some studies show up to 7 times higher.

APUS Study

American Public University System operates through American Military University and American Public University

Founded in 1991 Accredited by HLC and DETC 79 degree programs encompassing associates’,

bachelor’s, and master’s Approximately 75,000 students as of 6/30/10 100% online, asynchronous courses Monthly course starts

APUS student growth

Strategic Management Problem

• Student enrollment growth beginning in 2006 increased dramatically over previous three years.

• Senior management wanted to measure student retention and see if there factors that influenced persistence.

• APUS maintains comprehensive data warehouse that integrates student demographic data and academic records.

• Selected 2007 as base year.• Selected all undergrad students pursuing degree or

certificate programs.• Determined 2007 student status as of 12/31/2009 as either

active, graduated, or disenrolled (note: APUS automatically disenrolls students after 12 months of not enrolling in at least one class).

APUS Student Body Characteristics

• 95% of students were part-time in 2007.• # of first time, full time freshmen equaled 26.• 83% bachelor’s, 16% associates, 1% certificates• 87% of students transferred credits from previously attended

colleges or from ACE-evaluated military training.• Mean credit hours received in transfer request were 39 credit

hours. 90% of students received less than 60 credit hours.• Active-duty military students represented 69% of

undergraduate enrollment in 2007.• 24% of new students in 2007 dropped out after attending only

two classes.• Gender – Males = 66%, Females = 34%.• Age – Mean = 29.8 and Median = 28.

Method

• 21,521 undergraduates completed at least one courses at APUS in 2007. 20,569 records selected.

• 10,064 active (49%) at 12/31/2009.• 6,858 disenrolled (33%) at 12/31/2009.• 3,647 graduated (18%) at 12/31/2009.• First pass analysis used regression with forward entry.• Independent variables selected of Transfer Credits Received,

Age, Gender, Ethnicity, Cumulative GPA, Last Course Grade Received, Military / Civilian Status, Degree Program, Course Duration, Time Since Last Course

Categorical variables reduced to binary dummy variables and some variables collapsed into buckets

Non relevant data removed from model and re-run.

Sample Groupings

Age bucketed into IPEDs classification Plus and minus grades collapsed into single variable 15 hour transfer credit blocks defined as binary dummy

variables with no transfer credit a separate entry

Results

45 variables were found to be significant predictors of retention

32.8% of variance accounted for by the model No transfer credits – 15.8% No of Courses completed in 2007 – 4.5% Last Grade Received of F – 3.8% Last Grade Received of W (Course Withdrawal) – 2.7% Cumulative 4.00 GPA – 1.4% No other variable over 0.6% No difference in regression outcomes in segregating active

duty military students from civilian students. Race and gender were insignificant variables in this analysis.

Potential Implications

Lack of transfer credit may indicate inadequate academic preparation or part-time students may be overwhelmed with requirements for graduation (mean # of courses completed in 2007 was 3.6 which would not be sufficient to graduate in 10 years IF no transfer credits were received. 74% of no transfer credit students were disenrolled at 12/31/2009.

Registrations are likely a proxy for activity. Graduated students at 12/31/2009 averaged 6.1 courses taken in 2007. Disenrolled students averaged 2.5 courses completed in 2007.

F or W may have significant psychological impact. Department of Defense requires service members to reimburse Department for tuition if grade received is F or W. For disenrolled students, last grade of F = 31% and last grade of W = 15% of that population.

4.0 GPA may be related to students transferring to other institutions. Represents 9% of disenrolled students.

Actionable Intelligence

Students who lack transfer credit may need additional student services.

Given early dropout decision of students, special focus on retention in first three classes may improve overall persistence.

Mandating introductory skills assessment may reduce the number of F’s and W’s received by students and should be a strategic objective.

More research needed on perceived institutional value for 4.0 GPA students.

How representative is data from IPEDS regarding online institutions’ persistence rates?

Selected 10 institutions with large or totally online student populations.

8 institutions were for-profit, 2 were non-profit. Unduplicated ugrad headcount for Fall 2007 ranged

from 6,233 to 334,851 Total unduplicated headcount for Fall 2007 was

624,239. Total FTE headcount for Fall 2007 was 303,206 (high

part-time %’s). Total # of graduates within 150% of normal time for Fall

2007 was 522. Total completions for academic year 2008 was 50,869.

Measuring persistence – IPEDS alternative

Matching completions in a year against unduplicated or FTE headcount is not representative of persistence rate because of high growth rate in online attendance during this period.

Completions/graduates represent an average time to complete of 6.7 years at APUS, thus completions reported to IPEDS represent multiple years when the students started.

Some type of cohort tracking by year would be more appropriate to measure persistence.

Measuring persistence – IPEDS alternative

Averaging Method of Retention Calculation

APUS Undergraduate Students

2007 Calendar Year

Undup Count

Dropped

Graduated

Active Grad + Active

Undup Grad %

Grad + Active

%

20,569 6,858 3,647 10,064 13,711 17.7% 66.7%

APUS Undergraduate Students

2005 Calendar Year

Undup Count

Dropped

Graduated

Active Grad + Active

Undup Grad %

Grad + Active

%

8,921 3,869 2,564 2,488 5,052 28.7% 56.6%

Global Implications

Findings related to Transfer Credit and Courses taken per year are relevant for measuring part-time student participation in higher education. Mean of 6.7 years for APUS bachelor’s student to graduate exceeds current IPEDS measurements in addition to only first time, full time students are tracked.

IPEDS collects data on annual number of completions (graduations of non first time, full time students). Modifying system to measure transfer students would simply require tracking new students at institution by cohort year who were not first time, full time.

Directions for Future Research

Introduction of LMS and student survey data into the model

Model refinement using techniques such as HLM, Decision Trees and Neural Network Analysis

Qualitative inquiry into predictors to develop an explanatory model through triangulation

Multi-Institutional data aggregation and analysis with an eye toward detecting potential differences in models.

Findings on no difference between race/gender should be pursued in studies with other online programs to see if military orientation is explanation or if online/color blind instruction is a factor.