Slug Success at University of California, Santa Cruz · Slug Success at University of California, Santa Cruz Understanding Predictive Analytics Campus ©2015 The Advisory Board Company

©2015 The Advisory Board Company • eab.com

Student Success Collaborative TM

Slug Success at University of California, Santa Cruz Understanding Predictive Analytics

Campus


ROAD MAP

1

2

3

2

Promises and Perils

More Than Just Predictive Models

Why Predictive Analytics?


3

Using Data Analytics to Spot Struggling Students Before It’s Too Late

Informed Outreach

Start with a large behavioral data set

Identify traits correlated with needs

Group individuals by predictive traits

Precisely target resources and services

How a Predictive Model Focuses Efforts

Obvious Risk Cases

Most do not return for a second year

All-Stars Students

Vast majority will ultimately graduate

FIRST YEAR GPA 2.0

Murky Middle Outcome difficult to predict

without advanced data

FIRST YEAR GPA 3.0


4

Cross-Industry Rush to Leverage Data, Higher Education Lags Behind

A New Data-Driven Economy

Includes teaching and research positions

Big Data Job Postings by Industry Predictive Models Already Commonplace

Advertising Predicting products we might want to buy

Sports Predicting the highest-value players

Social Networking Predicting relationship compatibility

Healthcare Predicting patient re-admissions

Politics Predicting swing voter behavior

Entertainment Predicting the media we might enjoy

3,248

4,474

4,873

5,011

5,594

6,290

6,476

6,874

8,698

8,992

16,716

Hospitals and HealthSystems

Colleges and Universities

Retail

Management Consulting

Computer Systems Design

Employment Services

R&D Technical Services

Internet InformationServices

Manufacturing

National Security and Int'lAffairs

Banking, Credit, Insurance


5

Faced with an Aging Population, Hospitals Using Risk Segmentation to Deliver Care More Efficiently

How Is Healthcare Dealing With Its Demographic Crisis?

Source: Advisory Board Company Interviews and Analysis

5%

Complex

illnesses

25%

Chronic

conditions

70%

Healthy or well-

managed conditions

Risk Segmentation Enables Scalable Care Reported Results

Low-Risk Patients Reduce demand on the system by shifting patients to e-medicine and promoting healthy lifestyles

High-Risk Patients Minimize hospital readmissions by surrounding the patient with an in-home “care team”

Rising-Risk Patients Prevent costly escalations by using analytics to monitor risk factors and intervene quickly

Lower cost of care per patient

Fewer avoidable hospital visits

Fewer patient re-admissions

Reduced traffic through the ED

Advanced Risk Stratification

Scalable Support

Four Pillars of Population Health Management

Differentiated Care

Ownership and Accountability


6

Population Health Management for Higher Education

The New Blueprint for Student Success

Source: EAB Interviews and Analysis

Moderate Risk High Risk Low Risk

Enable Effective Self-Direction Provide easy access to information to leverage students themselves

Coordinate Efficient High-Touch Care Work closely with students and manage their interactions with support offices

Proactively Monitor and Intervene Create an analytics “safety net” to catch common problems before they escalate

Differentiated Care Strategies

High-Touch Care

Proactive Intervention

Preventative Measures



Proactive Intervention

Time and Cost Savings

How do we responsibly deploy differential care across our population?

How do we use process and technology to scale our efforts?

Core Considerations


ROAD MAP

1

2

3

7

Promises and Perils




8

Different Types of Analytics to Answer Different Questions

Defining (and Demystifying) Analytics – The EAB Approach

Source: Gartner (October 2014)

Descriptive What happened?

Diagnostic Why did it happen?

Predictive What will happen?

Prescriptive What should I do?

Decisions Actions Data

1

2

3

4

Human/Interpretive Input

Analytics

The Big

4 Types of Business Analytics

Advising Reports

Descriptive

Institution Reports

Diagnostic

Major Explorer

Prescriptive

Predicted Support Scores

Predictive

Examples from Slug Success


9

Four Types of Business Analytics

Descriptive Analytics

Descriptive analytics look backwards in time and they try to answer the question what happened? They help identify patterns by looking at or comparing inputs and outcomes across different time periods. While simple, these powerful analytics are an essential starting point because they help us identify problems that need to be solved, sometimes using process and sometimes by building more advanced analytics.


10


Diagnostic Analytics

Diagnostic analytics also look backwards, but they don’t just describe what happened, they attempt to uncover why something happened. They look at the relationships between certain things to find correlations and even causality, which means they can help organizations take corrective action.


11


Predictive Analytics

Predictive analytics are forward looking. They rely on historical analysis, present conditions, and computational modeling to answer the question: what will happen? Predictive analytics try to actually compute the most likely outcome. Slug Success predictive risk models do exactly that for students: they look at historical patterns to create a model for success and then compare each individual student to that model to estimate how likely it is that that individual will be successful.

Support Level


12


Prescriptive Analytics

Prescriptive analytics which answer the question “what should I do?”. You can think of these as “what if” versions of predictive analyses that help an organization see how predictions change if they adjust certain inputs. In student success, prescriptive analytics are more of an art than a science. However, prescriptive analytics have their place in helping us select between possible choices, and when coupled with good interpretation and contextual knowledge can be incredibly impactful.


13

Keeping Students On Track Toward Graduation in Their Major

Success Markers and Notifications

Anatomy of a Success Marker

Required milestone course for the major (e.g., Chemistry 101)

Minimum recommended grade (e.g., B-)

Appropriate timing (e.g., 0 – 30 units)

Chemistry Major

Success Marker #1

Success Marker #2

Success Marker #3

Success Marker #4

Success Marker #5

Platform Notifications

Success markers already completed

Χ Success markers missed due to grade or timing

Success markers that are upcoming


14

Not All Courses Equal

Reviewing Historical Records to Identify Predictive Courses and Grades

Two Required Courses for a Chemistry Major

Predictive Not Predictive

64% 58%

25%

13% 10%

30%

0%

10%

20%

30%

40%

50%

60%

70%

A B C D F W

Gra

du

atio

n R

ate

in

th

e M

ajo

r

Grade

CHEM101

✓ ✘

52% 55%

48% 43%

12%

27%

0%

10%

20%

30%

40%

50%

60%

70%

A B C D F W

Gra

du

atio

n R

ate

in

Ma

jor

Grade

BIOL305


15

Components in Slug Success that Form the Entire Risk Assessment

Summary of Available Student-Level Analytics

30-Second Gut Check and Success Markers

Reports/Notes Tab

Alerts, Cases, Progress Reports (Future)

Trend Graphs


ROAD MAP

1

2

3

16

Promises and Perils




17

What We’ve Heard

Challenges of Leveraging Analytics

Inaction

Difficult to Know What Worked

Not Enough Experience or Support to Make Good Decisions

Typical BI tools often don’t make as much of an impact as they could because users either don’t know what to do with them or feel paralyzed by the sheer amount of information. Analytics users need actionable insights, concrete next steps, and the support to take action on their insights.

The best data-enabled decision-making doesn’t happen in a vacuum. This is particularly true for predictive and prescriptive analytics. Overreliance on analytics without contextualization can lead to poor decision-making. Users need to be supported in how to leverage context, experience, and best practices to take the right actions.

Analytics that help you identify opportunities for action are great. But without closing the loop, how do you know which actions worked? How can you replicate success and stop wasting time on interventions that don’t work?

Lots of Data, Little Insight

Just because it exists doesn’t make it useful. There is a lot of bad data out there, and data that just doesn’t yield much understanding. But it takes a lot of time and effort to figure out which data to use and how to deliver it to users in a user-friendly and actionable format.


18

How Advisors Have Used SSC to Support Their Students

A Framework for Leveraging Data

Preempt Identify students before a difficult conversation becomes necessary

• Identify and reach out to students with moderate or high predicted support levels in their current or desired major

o Layer in additional data points into your outreach, to ensure your intervention is actionable

• Leverage the Population Health Management concept to strategically target resources based on support level

Persuade Build urgency, convince the student to act or change

Reframe Get the student excited about a tailored support or parallel plan

• Use support predictions in the Major Explorer to build a student’s confidence about success in an alternative major

• Start a conversation using SSC career data

• Use missed/upcoming success markers to show a student the hill they will have to climb

• Turn on “Student View” and discuss their declining GPA or consistently low credit completion

• Use the analytics to inform your conversations, providing evidence to your recommendations

• Don’t overly rely on the predicted support level in one-on-one conversations


19

Tips from Our SSC Advisors on How to Use the Prediction Effectively

“How do I Use a Student’s Support Prediction?”

How to Use Predicted Support Effectively

…And How Not To

Use support levels to strategize which students to monitor more heavily

Use support levels as a way to frame conversations with students, without telling them they are “moderate” or “high”

Use support predictions to explore major options and strengthen long-term academic planning

Consider success markers to help students select and prepare for upcoming courses

Contextualize a student’s support level by remembering the group against which that student is being compared

Don’t only meet with “red” students

Don’t ignore your intuition and experience in evaluating whether a student might be at risk

Don’t make all your “red” students switch to a “green” major

Don’t ignore a student’s passion if it’s in a “red” or “yellow” area

Don’t assume a student won’t or can’t graduate because they are “red”; “red” is a probability, not a fate, and will never be 100% accurate


20

It’s a Tool. You’re the Advisor.

The Power of Language

High Risk

Red

No chance

I’m just telling you what the data says

Take a look at this screen. You should be concerned.

Look at these majors and pick something green.

Negative Interpretation Positive Guidance

Let’s look at where you have struggled and discuss ways I can support you.

While some areas have been tough for you, I see some clear strengths here!

I understand that this is something you want. And I want you to succeed. Let’s have an honest conversation about how students have succeeded in this program and where you are on that path.

Let’s look at a couple different paths to success…I would like to set up another appointment to see how things are going and which path you feel best about.


21

Southern Illinois University Improved Retention, Increased Tuition Revenue

Results from a Risk-Focused Campaign

Oct 2013 Apr 2014

Additional first time, full time students retained

85+ Additional tuition revenue from retention increase

Increase in fall to spring retention

3.6%

Moderate Risk Interventions

% of Total Interventions

20%

26%

NoIntervention

SomeIntervention

Moderate Risk Retention Rate

Fall to Spring Retention

92%

98%

$500K


22

Georgia State’s Use of Success Markers Benefited the School and Students

Results from a Success Marker Focused Initiative

141

138

Fall 2010 Fall 2013

Decreased Time to Degree…

Average Credits at Time of Graduation

All Students African American STEM Majors

150

140

Fall 2010 Fall 2013

Total savings by students in the graduating class of 2014 compared to the class of 2013

$4M

…And Reduced Overall Cost

Georgia State used success markers to drive course redesign, identify courses needing supplemental instruction, and create freshmen learning communities. This led to great results, including:


23

Analytics Are at the Heart of Slug Success

In Summary

Research

Consulting

Technology

Analytics

Three Key Takeaways

Data can help strategically target resources to students based on support levels

Analytics are more than just predictive models

Advisor interpretation and communication is critical to ensure positive impact

1

2

3

Education Advisory Board

2445 M Street NW, Washington DC 20037

P 202.266.6400 | F 202.266.5700 | eab.com

Documents

Slug Success at University of California, Santa Cruz · Slug Success at University of California, Santa Cruz Understanding Predictive Analytics Campus ©2015 The Advisory Board Company