Embed Size (px)
DESCRIPTION
Improving student persistence, especially among under-represented minority students, is a driving goal at many colleges and universities. Academic technologies, such as the Learning Management System (LMS), are frequently used to deliver innovative pedagogical strategies to increase engagement and improve persistence. This study presents research on a redesigned hybrid high-enrollment undergraduate course exploring the relationship between LMS activity, student background characteristics, current enrollment information, and student achievement.
Citation preview
Logging On To Improve AchievementEvaluating the relationship between use of the Learning Management System, student characteristics, and academic achievement in a hybrid large enrollment undergraduate course
Research Highlights: Presentation to SoLAR Storm
November 15, 2012John C Whitmer ([email protected])
Committee Chair: Dr. Paul Porter, Sonoma State University
Slides: http://slidesha.re/sFKjcm
Introduction
• Educational Doctorate Degree (EdD) candidate (University of California Davis & Sonoma State University)
• Advanced to candidacy, defending ~ January 14
• Associate Director, California State University LMSS Project, Chancellor’s Office
Me
Presentation Outline
1. Study Case & Context
2. Results for Instructional Practices
3. Results for LMS Data Analysis
4. Conclusions
STUDY CASE & CONTEXT
Problem: Student Graduation• Less than 50% of college/university students graduate
within 6 years• California State University: 52.4%
(first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)
• Students from under-represented minority racial/ethnic groups graduate at much lower rates • California State University: 38.3%
(African American students, first-time freshman, 2000 cohort) (CSU Analytic Studies, 2011)
• Contributing factor: mega-enrollment intro courses• Infrequent interaction, prevent faculty/student relationships
Case: Introduction to Religious Studies
• Redesigned to hybrid delivery through Academy eLearning
• Highest LMS usage entire campus Fall 2010 (>250k hits)
• 373 students (54% increase)
• Bimodal results• 10% increased SLO mastery• 7-11% increase in DWF
54 F’s
Research Questions
1) Is there a relationship between student LMS usage and academic performance? Does this relationship vary by the pedagogical purpose underlying LMS usage? (correlation)
2) Is there a relationship between student background characteristics or current enrollment information and academic performance? (correlation)
3) Does analyzing combined student characteristics and current enrollment information increase the predictive relationship between combined LMS usage data and student success? (multivariate regression)
4) Does a student’s economic status and student of color status vary the predictive relationship between combined LMS usage, combined background characteristics and current enrollment information? (multivariate regression, restricted model)
Independent Variables: Student Characteristics
Independent Variables: LMS Usage
Research Methods (Cliff’s notes version)
1. Extract data, validate with appropriate “owner”
2. Transform variables • measures of interest (e.g. “URM”, not race/ethnicity)• analysis methods (categorical into numeric)
3. Examine data for • outliers, missing data, data distributions, etc.• colinearity between variables (e.g. independence)
4. Join data into single data file, collapse to one record/student
5. Run analysis
Results for Instructional Practices
Correlation: LMS Usage w/Final Grade
Scatterplot of Assessment Activity Hits v.
Course Grade
Correlation: Student Char. w/Final Grade
Most interesting finding (so far):
Smallest LMS Use Variable
(Administrative Activities)
r=0.3459
Largest Student Characteristic
(HS GPA)
r=0.3055>
Regression R2 Results Comparison
RESULTS FOR LMS DATA ANALYSIS
Lms Logfiles: “Data Exhaust”
1. Logfile tracks server actions (not educationally relevant activity)
2. Duplicate logfile hits for single student action
3. To remedy, filtered logfiles by:• Time (> 5sec, <3600 sec)• Actions (no “index views”, more)
Logfile Data Filtering Results
Discus
sion
Activi
ty H
its
Conte
nt A
ctivi
ty H
its
Asses
smen
t Act
ivity
Hits
Mai
l Act
ivity
Hits
Admin
istra
tive
Activi
ty ..
.0
50
100
150
200
250
300
350
400
450
382
151
58 4926
54 5123 36
16
Final data set: 72,000 records (from 250K+)
LMS Use Consistent across Categories
Factor Analysis of LMS Use Categories
Missing Data On Critical Indicators
Conclusions
1. At the course level, LMS use better predictor of academic achievement than any student characteristic variable. Behavioral data appears to supercede demographic information (what do, not who are).
2. Moderate strength magnitude of complete model demonstrates relevance of data, but suggests that refinement of methods could produce stronger results.
3. LMS data requires extensive filtering to be useful; student variables need pre-screening for missing data.
Ideas & Feedback
Potential for improved LMS analysis methods:• social learning • activity patterns • discourse content analysis• time series analysis
Group students by broader identity, with unique variables:• Continuing student (Current college GPA, URM, etc.• First-time freshman (HS GPA, SAT/Act, etc)
Contact Info
John Whitmer
Skype: john.whitmer
USA Phone: 530.554.1528
By WingedWolfDamián Navas
