Embed Size (px)
Citation preview
Predictive Analytics Reporting
(PAR) Framework
Ellen Wagner WICHE Cooperative for Educational Technologies
The Predictive Analytics Reporting (PAR) Framework
• A “big data” analysis effort iden*fy drivers related to loss and momentum and to inform student loss preven*on
• WCET member ins*tu*ons voluntarily contribute de-‐iden6fied student records to create a single federated database which is then analysed using descrip*ve, inferen*al and predic*ve methods.
PAR Objectives • Iden*fy common variables likely to influence student reten*on and progression, and measure the degree of influence
• Determine if measures and defini*ons of reten*on, progression, and comple*on differ materially among various types of postsecondary ins*tu*ons
• Begin to iden6fy the highest impact interven6ons with targeted pools of at risk-‐students
Costs and Completion Rates
Source: New York Times; NCES
0
10
20
30
40
50
60
70
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2-‐yr colleges 4-‐yr colleges
Gradua6on rates at 150% of 6me
Cohort year
Performance Based Funding
hTp://www.ncsl.org/issues-‐research/educ/performance-‐funding.aspx
Are You “Data-Ready”?
hTp://www.whitehouse.gov/issues/educa*on/higher-‐educa*on/college-‐score-‐card
PAR Framework background • Funded by Bill & Melinda Gates Founda*on in 2011, 2012, 2013 • Managed by WICHE Coopera*ve for Educa*onal Technologies, operated by WCET core project team • 16 grant-‐funded ins*tu*onal partners ▫ 7 4-‐year schools ▫ 5 community colleges ▫ 4 for-‐profit ins*tu*ons
• New self-‐funding members • Grant extensions for new work
• In-‐kind dona*ons to date ▫ Blackboard ▫ iData ▫ Starfish
PAR FRAMEWORK GRANT PHASES 1. Iden6fy poten6al for mul6-‐ins6tu6onal data mining
2 Expand sample, variables and measurement period
3 Deliver models, frameworks, benchmarks and alerts to individual campuses
4. Perform field tests to analyze interven6ons across ins)tu)ons
5. Scale and self-‐sufficiency
8
Institutional Partners Founding Partners (since 2011): American Public University System* Colorado Community College System* Rio Salado College* University of Hawaii System* University of Illinois Springfield* University of Phoenix*
New Members (as of Oct 2013): Northern Arizona University Kaplan University Excelsior College University of North Dakota
Implementa6on Partners (since 2012): Ashford University Broward College Capella University Lone Star College System Penn State World Campus Sinclair Community College Troy University University of Central Florida University of Maryland University College Western Governors University
DATA STATISTICS
• Total Counts Time Frame • August 2009 – May 2013
– 13,090,351 course records – 1,842,917 student records
Reusable predic*ve models
Common Defini*ons of terms
Student level watch lists for
targeted interven*ons
Mul*-‐Ins*tu*onal collabora*on
Measurable results
PREDICT
ACT
RESULTS
Common Defini*ons of interven*ons
Scalable cross-‐ins6tu6onal improvements
The PAR Framework Value Proposition
Structured, Readily Available Data • Common data
defini*ons = reusable predic*ve models and meaningful comparisons.
• Openly published via a cc license @ hTps://public.datacookbook.com/public/ins*tu*ons/par
PAR Data Inputs Student
Demographics & Descrip*ve
Gender Race
Prior Credits Perm Res Zip Code HS Informa*on Transfer GPA Student Type
Student Course Informa*on Course Loca*on
Subject Course Number
Sec*on Start/End Dates Ini*al/Final Grade Delivery Mode Instructor Status Course Credit
Student Academic Progress
Curent Major/CIP Earned Creden*al/CIP
Student Financial
Informa*on FAFSA on File – Date
Pell Received/Awarded – Date
Course Catalog Subject
Course Number Subject Long Course Title
Course Descrip*on Credit Range
** Future
Lookup Tables Creden*al Types Offered Course Enrollment Periods
Student Types Instructor Status Delivery Modes Grade Codes
Ins*tu*on Characteris*cs
Possible Addi*onal **
Placement Tests NSC Informa*on SES Informa*on
Sa*sfac*on Surveys College Readiness Surveys Interven*on Measures
Some of PAR’s “Outputs” Va
lidated
Mul*-‐
Ins*tu*o
nal
Dataset
Reflec*ve Ins*tu*onal Reports
Cross-‐Ins*tu*onal Benchmarks
Aggregate Models Policy
Ins*tu*onal Models Local Interven*on
Student Watch List Compara*ve Interven*ons
Actionable Predictive Models
Sample benchmark framework
Once we know what we are talking about its easier to deal with the “Now, What?” problem.
PAR Student Success Matrix (SSMx)
Literature-‐based tool for benchmarking student services
and interven*ons § 600+ total interventions submitted
§ Ability to compare among all 16 PAR institutions
§ Basis for institutional intervention field tests
§ Publically available, over 1,000 downloads since June 2013
hTps://par.datacookbook.com/public/ins*tu*ons/par
Testing Intervention Effectiveness
• 4 Year Ins6tu6ons: – Does Peer Mentoring (social integra*on focus) improve success as
measured by % of students earning a C or beTer?
• Community Colleges: – Do the same or similar developmental educa*on approaches offered
at mul*ple ins*tu*ons yield the same or similar results at all each ins*tu*on?
• Without common diagnoses, it is difficult to agree on treatments to address diagnosed problems.
• Without agreed-‐upon treatments, it is difficult to measure the efficacy of treatments.
• If one cannon measure efficacy it is impossible to scale.
• Perhaps worse of all, we con*nue to guess about what really works in student success.
Helping Students Succeed.
Scaling Student Success
Delivering on the Promise of Data
Fin
