Comprehensive Assessment of Student Retention in Online Learning

  • Published on

  • View

  • Download


Comprehensive analysis of SIS based data sources related to student retention in online environments.


<ul><li> 1. Proprietary - Not for duplication American Public University System Comprehensive Assessment of Student Retention in Online Learning Programs Wally Boston, Ed.D. and Phil Ice, Ed.D. October 19, 2010 </li> <li> 2. The State of Online Learning <ul><li>Approximately 4 million students taking online courses in Fall 2008 with projected 12.9% growth rate. </li></ul><ul><li>Outpacing face-to-face growth by 6 to 1. </li></ul><ul><li>US Department of Education meta-analysis online is more effective than face-to-face ( </li></ul></li> <li> 3. % Growth in U.S. Online Enrollments </li> <li> 4. Retention <ul><li>A concern in higher education since the late 1800s. </li></ul><ul><li>High School GPA, SAT and/or ACT scores, etc. (background characteristics) most reliable predictors of retention; key reason why most selective colleges have highest graduation rates. </li></ul><ul><li>Tinto, Astin, Braxton and others have demonstrated the importance of academic and social integration for student persistence. </li></ul><ul><li>Measures of academic and social integration well defined in the face-to-face setting not in online. </li></ul><ul><li>Online dropout rates much higher than face-to-face some studies show up to 7 times higher. </li></ul></li> <li> 5. APUS Study <ul><li>American Public University System operates through American Military University and American Public University </li></ul><ul><li>Founded in 1991 </li></ul><ul><li>Accredited by HLC and DETC </li></ul><ul><li>79 degree programs encompassing associates, bachelors, and masters </li></ul><ul><li>Approximately 75,000 students as of 6/30/10 </li></ul><ul><li>100% online, asynchronous courses </li></ul><ul><li>Monthly course starts </li></ul></li> <li> 6. APUS student growth </li> <li> 7. Strategic Management Problem <ul><li>Student enrollment growth beginning in 2006 increased dramatically over previous three years. </li></ul><ul><li>Senior management wanted to measure student retention and see if there factors that influenced persistence. </li></ul><ul><li>APUS maintains comprehensive data warehouse that integrates student demographic data and academic records. </li></ul><ul><li>Selected 2007 as base year. </li></ul><ul><li>Selected all undergrad students pursuing degree or certificate programs. </li></ul><ul><li>Determined 2007 student status as of 12/31/2009 as either active, graduated, or disenrolled (note: APUS automatically disenrolls students after 12 months of not enrolling in at least one class). </li></ul></li> <li> 8. APUS Student Body Characteristics <ul><li>95% of students were part-time in 2007. </li></ul><ul><li># of first time, full time freshmen equaled 26. </li></ul><ul><li>83% bachelors, 16% associates, 1% certificates </li></ul><ul><li>87% of students transferred credits from previously attended colleges or from ACE-evaluated military training. </li></ul><ul><li>Mean credit hours received in transfer request were 39 credit hours. 90% of students received less than 60 credit hours. </li></ul><ul><li>Active-duty military students represented 69% of undergraduate enrollment in 2007. </li></ul><ul><li>24% of new students in 2007 dropped out after attending only two classes. </li></ul><ul><li>Gender Males = 66%, Females = 34%. </li></ul><ul><li>Age Mean = 29.8 and Median = 28. </li></ul></li> <li> 9. Method <ul><li>21,521 undergraduates completed at least one courses at APUS in 2007. 20,569 records selected. </li></ul><ul><li>10,064 active (49%) at 12/31/2009. </li></ul><ul><li>6,858 disenrolled (33%) at 12/31/2009. </li></ul><ul><li>3,647 graduated (18%) at 12/31/2009. </li></ul><ul><li>First pass analysis used regression with forward entry. </li></ul><ul><li>Independent variables selected of Transfer Credits Received, Age, Gender, Ethnicity, Cumulative GPA, Last Course Grade Received, Military / Civilian Status, Degree Program, Course Duration, Time Since Last Course </li></ul><ul><li>Categorical variables reduced to binary dummy variables and some variables collapsed into buckets </li></ul><ul><li>Non relevant data removed from model and re-run. </li></ul></li> <li> 10. Sample Groupings <ul><li>Age bucketed into IPEDs classification </li></ul><ul><li>Plus and minus grades collapsed into single variable </li></ul><ul><li>15 hour transfer credit blocks defined as binary dummy variables with no transfer credit a separate entry </li></ul></li> <li> 11. Results <ul><li>45 variables were found to be significant predictors of retention </li></ul><ul><li>32.8% of variance accounted for by the model </li></ul><ul><li>No transfer credits 15.8% </li></ul><ul><li>No of Courses completed in 2007 4.5% </li></ul><ul><li>Last Grade Received of F 3.8% </li></ul><ul><li>Last Grade Received of W (Course Withdrawal) 2.7% </li></ul><ul><li>Cumulative 4.00 GPA 1.4% </li></ul><ul><li>No other variable over 0.6% </li></ul><ul><li>No difference in regression outcomes in segregating active duty military students from civilian students. </li></ul><ul><li>Race and gender were insignificant variables in this analysis. </li></ul></li> <li> 12. Potential Implications <ul><li>Lack of transfer credit may indicate inadequate academic preparation or part-time students may be overwhelmed with requirements for graduation (mean # of courses completed in 2007 was 3.6 which would not be sufficient to graduate in 10 years IF no transfer credits were received. 74% of no transfer credit students were disenrolled at 12/31/2009. </li></ul><ul><li>Registrations are likely a proxy for activity. Graduated students at 12/31/2009 averaged 6.1 courses taken in 2007. Disenrolled students averaged 2.5 courses completed in 2007. </li></ul><ul><li>F or W may have significant psychological impact. Department of Defense requires service members to reimburse Department for tuition if grade received is F or W. For disenrolled students, last grade of F = 31% and last grade of W = 15% of that population. </li></ul><ul><li>4.0 GPA may be related to students transferring to other institutions. Represents 9% of disenrolled students. </li></ul></li> <li> 13. Actionable Intelligence <ul><li>Students who lack transfer credit may need additional student services. </li></ul><ul><li>Given early dropout decision of students, special focus on retention in first three classes may improve overall persistence. </li></ul><ul><li>Mandating introductory skills assessment may reduce the number of Fs and Ws received by students and should be a strategic objective. </li></ul><ul><li>More research needed on perceived institutional value for 4.0 GPA students. </li></ul></li> <li> 14. How representative is data from IPEDS regarding online institutions persistence rates? <ul><li>Selected 10 institutions with large or totally online student populations. </li></ul><ul><li>8 institutions were for-profit, 2 were non-profit. </li></ul><ul><li>Unduplicated ugrad headcount for Fall 2007 ranged from 6,233 to 334,851 </li></ul><ul><li>Total unduplicated headcount for Fall 2007 was 624,239. </li></ul><ul><li>Total FTE headcount for Fall 2007 was 303,206 (high part-time %s). </li></ul><ul><li>Total # of graduates within 150% of normal time for Fall 2007 was 522. </li></ul><ul><li>Total completions for academic year 2008 was 50,869. </li></ul></li> <li> 15. Measuring persistence IPEDS alternative <ul><li>Matching completions in a year against unduplicated or FTE headcount is not representative of persistence rate because of high growth rate in online attendance during this period. </li></ul><ul><li>Completions/graduates represent an average time to complete of 6.7 years at APUS, thus completions reported to IPEDS represent multiple years when the students started. </li></ul><ul><li>Some type of cohort tracking by year would be more appropriate to measure persistence. </li></ul></li> <li> 16. Measuring persistence IPEDS alternative Averaging Method of Retention Calculation APUS Undergraduate Students 2007 Calendar Year Undup Count Dropped Graduated Active Grad + Active Undup Grad % Grad + Active % 20,569 6,858 3,647 10,064 13,711 17.7% 66.7% APUS Undergraduate Students 2005 Calendar Year Undup Count Dropped Graduated Active Grad + Active Undup Grad % Grad + Active % 8,921 3,869 2,564 2,488 5,052 28.7% 56.6% </li> <li> 17. Global Implications <ul><li>Findings related to Transfer Credit and Courses taken per year are relevant for measuring part-time student participation in higher education. Mean of 6.7 years for APUS bachelors student to graduate exceeds current IPEDS measurements in addition to only first time, full time students are tracked. </li></ul><ul><li>IPEDS collects data on annual number of completions (graduations of non first time, full time students). Modifying system to measure transfer students would simply require tracking new students at institution by cohort year who were not first time, full time. </li></ul></li> <li> 18. Directions for Future Research <ul><li>Introduction of LMS and student survey data into the model </li></ul><ul><li>Model refinement using techniques such as HLM, Decision Trees and Neural Network Analysis </li></ul><ul><li>Qualitative inquiry into predictors to develop an explanatory model through triangulation </li></ul><ul><li>Multi-Institutional data aggregation and analysis with an eye toward detecting potential differences in models. </li></ul><ul><li>Findings on no difference between race/gender should be pursued in studies with other online programs to see if military orientation is explanation or if online/color blind instruction is a factor. </li></ul></li> </ul>


View more >