Strengthening Multiple Measures of Assessment: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda - 2013 Strengthening Student Success Conference

8/13/2019 Strengthening Multiple Measures of Assessment: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda - 2013 Strengthening Student Success Conference

1/31

Harnessing the Power of

Alternative Assessment to Advancethe Completion Agenda

Eloy Ortiz Oakley

Superintendent-President, Long Beach City College

Terrence Willett

Director of Planning and Research, Cabrillo College/RP Group Researcher

Patrick Perry

Vice Chancellor, CCCCO

Strengthening Student Success Conference Ocotber9, 2013

1


2/31


3/31

LBCCs Research

Examined five cohorts of more than 7,000

LBUSD grads who attend LBCC directly

after high school

Determined strongest predictors ofsuccess in college courses

Improved alignment of assessment and

placement with evidence based

predictors of success

3


4/31

Key Findings

1. Standardized tests predict standardized

tests very well but

2. Classroom performance predicts

classroom performance3. More information tells us more about our

students likelihood to succeed

Findings presentedreal opportunitiestosignificantly improve student placement and

achievement, and their college experience

4


5/31

Reimagining the Transition to

College: Fall 2012 Promise Pathways

Predictive placement model using multi-

method, evidence-based assessment

Prescriptive, full-time course load via first-

semester success plan

Emphasis on Foundational Skills

English, Reading, Math

5


6/31

11% 7%

13%

9%

60%

31%

0%

10%

20%

30%

40%

50%

60%

70%

Transfer Level English Transfer Level Math

F11 First time students F11 LBUSD F12 Promise Pathways

Alternative placement:Transfer-level Placement Rates

6


7/31

F2012 Promise Pathways 1-year vs. F20066-year rates of achievement

5.1%

11.9% 12.9%15.0%

41.4%

35.9%

9.9%

27.3%

31.2%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Successfully Completed

Transfer Math

Successfully Completed

Transfer English

Behavioral Intent to

Transfer

F2011 LBUSD (N=1660) F2012 Promise Pathways (N=976) F2006 LBUSD 6 Year Rate 7


8/31

64%55%

62%

51%

0%

10%

20%

30%

40%

50%

60%

70%

Transfer Level English Transfer Level Math

Non-Pathways (n=2095 English, 1803 Math)

Promise Pathways (n=516 English, 156 Math)

Success rates in transfer-level coursesFall 2012

Neither of these differences approach significance, p >.30 8


9/31

Equity Impact of Promise Pathways:

F2011 Baseline Equity Gaps

1%7% 5%

18%

11% 11%

21%

36%

4%

11% 11%

29%

10%

22% 22%

31%

0%

10%

20%

30%

40%

50%

60%

70%

Successfully

Completed Transfer

Math

Successfully

Completed Transfer

English


Transfer

Full-time, full year

F2011 Black F2011 Asian F2011 Hispanic F2011 White


10/31

Equity Impact: F2012 Students of Color

Compared to F2011 White Baseline

6%

28% 28%

53%

15%

44%39%

65%

13%

40%

33%

62%

10%

22% 22%

31%

0%

10%

20%

30%

40%

50%

60%

70%

Successfully

Completed Transfer

Math

Successfully

Completed Transfer

English


Transfer


F2012 Black F2012 Asian F2012 Hispanic **F2011 White


11/31

Equity gaps remain but significant gains for all

F2012 Students

6%

28% 28%

53%

15%

44%39%

65%

13%

40%

33%

62%

29%

57%52%

60%

0%

10%

20%

30%

40%

50%

60%

70%

Successfully

Completed Transfer

Math

Successfully

Completed Transfer

English


Transfer


F2012 Black F2012 Asian F2012 Hispanic **F2011 White


12/31

Cohort 1 and 2 PromisePathways Demographics

15% 16%

53%

15%

2%

10% 12%

66%

10%

2%

0%

10%

20%

30%

40%

50%

60%

70%

African American Asian/PI/Filipino Hispanic White Unknown

F2012 (N=976) F2013 (N=1321)


13/31

A BRIEF OVERVIEW

The Student Transcript-

Enhanced PlacementProject (STEPS)

13Harnessing the Power of Alternative Assessment to Advance the Completion Agenda


14/31

What Are We Seeking to Do?

Examine the value of using transcripts as part of

the assessment process

Create predictive models using intersegmental

data to study students who had already takencommunity college courses

Use models to analyze how well transcript data

predicts the first English & math courses students

take and how well they do in them

Recruit colleges to do local analyses to generate

campus-specific insights and trigger CCC/K-12

conversations



15/31

In English

tests

predict tests,grades and courses

matter butvary by college

15


16/31

16

Cox & Snell pseudo R-square ~ 0.35

Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

01234

5678910

11

CST's A-G

Courses

HS

CourseLevel

HS

CourseGrade

HS GPA*

Coun

tofColleges

Showin

gSignificance

Predictor Variable Category

Predicting College English Level

Weakest

Intermediate

Strongest


17/31

In math

tests

predict tests,but high school

level is alsoimportant

17


18/31

18



01234

5678910

11

CST's A-G

Courses

HS

CourseLevel

HS

CourseGrade

HS GPA*

Coun

tofColleges

Showin

gSignificanc

e


Predicting College Math Level

Weakest

Intermediate

Strongest


19/31

In English,

gradespredict

grades

19


20/31

20



0123456789

1011

CST's A-G

Courses

HS

CourseLevel

HS

CourseGrade

HS

GPA*

College

CourseLevel

Coun

tofColleges

Showin

gSignificance


Predicting College English Success

Weakest

Intermediate

Strongest


21/31

In math,success

predictors

vary by

college

21


22/31

22



0123456789

1011

CST's A-G

Courses

HS

CourseLevel

HS

CourseGrade

HS

GPA*

College

CourseLevel

Coun

tofColleges

Showin

gSignificance


Predicting College Math Success

Weakest

Intermediate

Strongest


23/31

What Does This Mean?

There is a disconnect between test scoresand performance in the first course taken

at many institutions

Using transcript elements would help torefine assessment

The relative weight of variables might be

influenced by local factors, due toconsiderations such as articulation and

curriculum



24/31

What Happens Next?

Updating the statewide analysis

Encouraging more colleges to replicate the study

so they understand the value of transcript data for

their own students

Starting conversations about how the study relates

to other aspects of developmental education

reformsuch as alignment, course offerings, and

curriculum Helping to develop a tool that enables colleges to

access high school data to inform multiple

measures assessment



25/31

Chancellors Office Multiple

Measure Warehouse ProjectPatrick Perry

Vice Chancellor of Technology,Research & Information Systems


26/31

Common Assessment Project

AB743 passed in 2011 (AB743, Block): BOG shall establish a common assessment system

to be used as one of multiple measures for

placement and advisement

Ongoing funding provided in 2013-14 budget for

project

RFA to move $$ to district currently underway


27/31

Common Assessment Project

Project intends to:

Identify a common, systemwide assessment for

math, English, ESL

Provide the assessment at no cost to districts forunlimited usage

Provide for score portability statewide

Create a multiple measures data warehouse toenhance placement activities


28/31

Multiple Measures Warehouse

Will include:

Outcomes of the common assessments

CCC statewide MIS Enrollment data

EAP scores

CAHSEE scores

All HS math and English enrollment records, GPAs

(likely through 11thgrade)


29/31

Multiple Measures Warehouse

Application will be written allowing a student

placement profile to be viewed by student

services personnel

big data predictive analytics applied to

multiple measures variables to assist in

placement effectiveness


30/31

Contact Information

Eloy Ortiz Oakley

[email protected]

Terrence Willett [email protected]

Patrick Perry

[email protected]

30


31/31

Additional Resources More information about Long Beach City College's research

http://bit.ly/PathwaysResearch

More information about the RP Groups Student Transcript-

Enhanced Placement (STEPS) Project

http://bit.ly/RPSTEPS

Step by Step process for replicated the research andplacement:

http://bit.ly/RPSTEPS2

Jobs for the Future Report: Where to Begin?: The Evolving Role

of Placement Exams for Students Starting College http://bit.ly/JFFReport

CCRC research on Assessment, Placement, and Progression in

Developmental Education

http://bit ly/CCRCAssess 31
http://bit.ly/PathwaysResearchhttp://bit.ly/RPSTEPShttp://bit.ly/RPSTEPS2http://bit.ly/JFFReporthttp://bit.ly/CCRCAssesshttp://bit.ly/CCRCAssesshttp://bit.ly/JFFReporthttp://bit.ly/JFFReporthttp://bit.ly/RPSTEPS2http://bit.ly/RPSTEPS2http://bit.ly/RPSTEPS2http://bit.ly/RPSTEPShttp://bit.ly/RPSTEPShttp://bit.ly/PathwaysResearchhttp://bit.ly/PathwaysResearchhttp://bit.ly/PathwaysResearch

Documents

Strengthening Multiple Measures of Assessment: Harnessing the Power of Alternative Assessment to Advance the Completion Agenda - 2013 Strengthening Student Success Conference