Promising pathwaysUnderstanding and predicting preparation and success

and its potential as a model for change for the California Community Colleges and beyond

Donald Berz, Executive Vice President, Academic Affairs John J. Hetts, Director of Institutional Research

Long Beach City CollegeCritical aspects of this research were completed by our Office of Institutional Effectiveness, including our Associate Dean of Institutional Effectiveness , Eva Bagg and our two research analysts, Andrew Fuenmayor and Karen Rothstein, with key additional support from Terrence Willett, Nick Wade, and the research team at Cal-PASS

2

Overview1) Background on the research2) Placement in English at Long Beach City

College3) Predicting placement in English at LBCC4) Predicting performance in English at LBCC5) Alignment of placement and performance

a) Building a better pathwayb) Effects of misalignment

6) Recommendations

3

1) Background: Data developmentWorked with Cal-PASS to generate 5 year cohort (05-06

to 09-10)All courses taken at Long Beach Unified School District

(LBUSD) and Long Beach City College (LBCC), California Standards Tests (CSTs), more…

Key aspects of cohortStudents coming directly from LBUSDExamining first English course taken at LBCC

Key variablesJunior & senior year English courses taken in high school

and spring semester grades in those courses11th grade CST-ELA scores at LBUSDEnglish courses at LBCC and grades in those courses

4

2) Fall 2010 LBUSD Cohort

English 1

One level below

Two levels below

Three Levels Below

No Assessment

Total

Total 171 581 0 480 442 1674

Percent 10.2% 34.7% N/A 28.7% 26.4% 100%

What does LBUSD’s pattern of English placement look like?

5

3) Predicting Preparation/PlacementPrimary method of assessing level of preparation

has been:student’s performance on standardized assessment

instrument (Descriptive Test of Language Skills (DTLS) and, more recently, Accuplacer)

combination of assessment & essay

What predicts assessed preparation?Predicting how students will score on the assessmentI.e., where they will be placed in our English course

sequence.

6

Predicting Placement via DTLS/AccuplacerOutcome: Whether or not students are placed into

Basic Skills

Key predictors:11th Grade – Grade in 11th grade English12th Grade – Grade in 12th grade EnglishCST - 11th Grade California Standards Test score in

English (CST-ELA)Course taken in 12th grade

Using LBUSD’s 12th grade English course Rhetoric & Composition as baseline for comparison Comparing 5 major groups of courses to that baseline

AP English, Multicultural Literature, Film Analysis, Remedial English, Other Electives

7

Predicting Placement in English 1 (vs. any remedial English): (relative size of influence - standardized beta; i.e., how big, relatively, the impact of the variable is on placement)

LBU

SD

Cours

e

(com

pare

d t

o R

heto

ric

and C

om

p)

Grade, 12th

Grade, 11th

AP Lit

Remedial

Film Analysis

Multicult Lit

Other Elective

LBUSD Course

11th Grade CST

-5 0 5 10 15 20

2.2*

2.5*

4.2***

-0.4

1.1

1.9

0.9

15.3†

* p<.05, *** p <.0001, † = p <1 x 10-52

8

11th Grade CST Accuplacer What matters most for predicting

placement/Accuplacer is: 11th grade CST scores

I.e., performance on and placement using Accuplacer is almost entirely predicted by CST score in 11th grade AP increases placement in English 1

Students 11th grade CST score could be used instead of Accuplacer to place students effectively disregarding last 3 semesters of high school

9

4) Predicting PerformancePeople jump prematurely to conclusions about quality of the

instruction high schools are providingBut that data just speaks to how students are placed into

Basic SkillsNeed to verify which factors best predict performance

Same data but using successful completion (C or better) of English courses

Rhetoric & Composition again used as comparison for courses at LBUSD

English course taken at LBCCEnglish 1 used as baseline for comparison against

One level below: Fundamentals of Writing Three levels below: College English Skills I

10

Predicting success in English at LBCC(relative size of influence - standardized beta; i.e., how big, relatively, the impact of the variable is on placement

3 levels below transfer

1 level below transfer

LBCC Course

Grade, 12th

Grade, 11th

AP Lit

Remedial

Film Analysis

Multicultural Lit

Other Elective

LBUSD Course

11th grade CST

-5.0 0.0 5.0 10.0 15.0 20.0 25.0

6.9******

3.7***

11.1††5.2******

4.0****

-3.0**1.1

2.3*

2.2*

4.0***

LBU

SD

Cours

e

(vs.

Rh

eto

ric

and

Com

p)

LBC

C c

ours

e(v

s. E

nglis

h 1

)

* p<.05, **, p< .01, *** p <.0001, **** p<.0001, ******, p<.000001 † † = p <1 x 10-27

11

Summary: PerformanceWhat most strongly predicts performance in

our English courses is: grade in 12th grade EnglishGrade in 11th grade also mattersCST/Accuplacer matters less than either gradeTaking AP at least as important as

CST/Accuplacer Taking remedial English predicts poorer performance

12

5) Misaligned Assessment and Performance11th grade CST strongly predicts Accuplacer-based placement

weakly predicts performance in courses.

Grade in 12th grade English strongly predicts performance Only weakly related to placement in those courses

Implication: current method of placement is not well aligned with what leads to success in our classrooms

Placing students based on 12th grade course performance holds potential to more strongly align placement with likelihood of success Place students currently likely to fail in more appropriate courses Place students likely to succeed in courses more suitable to

preparedness

13

Building a better pathwayBased on research done for on California Leadership Alliance

for Student Success (CLASS) initiative and other best practicesPrescriptive course loadEmphasis on Basic Skills coursesPriority enrollmentEmphasis on full-time course workStudent Success Course

Modeled on Counseling 1: Orientation for College SuccessResults: Higher persistence, better course success rates,

higher achievement rates, smaller equity gapsDirect placement in Basic Skills

A’s and B’s more likely to successfully complete in English 1 than current rate, C’s to 1 level below.

14

What this will mean at LBCC: Promise Pathways, Semester 1

0%

10%

20%

30%

40%

50%

60%

11%

21%17%

52%52%

29%

19%

0%

Current 1st semester enrollment

Promise Pathways model

15

More broadly what this means…

0%

10%

20%

30%

40%

50%

60%

70%

8%

21%19%

53% 54%

28%

18%

8% 13%

19%

59%

40%

34%

26%

9%

26%

20%

45%49%

32%

19%20%

14%

8%

58%62%

27%

11%

Asian Black Hispanic White

Current Placement Model Promise Pathways Model

16

Why this matters

Total Asian Black Hispanic White

Current 101 16 9 45 22

Current + P2 289 44.998 24.865 128.49 63.123

Direct Placement 366 67 38 154 70

Direct Placement + P2 913.84 167.507 94.075 384.356 175.286

50

150

250

350

450

550

650

750

850

950

Projected completion of Transfer Level English in One Year (cohort of ~1700)

Current Current + P2 Direct Placement Direct Placement + P2

17

Why this will workPlacement usingcurrent method

LBUSD Grade

Transfer Level

1 level below

3 levels below

A 36% 23% 18%

B 30% 35% 33%

C 27% 29% 33%

D 6% 11% 15%

F 1% 1% 1%

Promise Pathways built on emerging new best practices

Data not previously available now is, thanks to Cal-PASS

Fits principles that most educators know well based on decades of experience

Solves two problems at once: Students placed in courses

where they are likely to fail (red in table)

Students placed in courses far below that they are capable of (green in table)

18

Additional evidenceWorking model nearby at Grossmont-

Cuyamaca95% of their A & B students in one of their

large high schools were placed into Basic Skills in English Our rate for LBUSD seniors is ~81%

A & B students now placed into English 186% of A & B students successfully complete

English 1 on their first try Only 5% would have been allowed to even try.

19

Additional implicationsHelps explain why acceleration and alignment work

so well whenever and wherever they’re tried*Many students are prepared but are being misclassified.Acceleration works well because it places students

closer to where they are fully prepared to succeed

Why instructors complain about students being unable to succeed in English 1Students placed there based on mechanism that

doesn’t relate to performance30% of them are not actually prepared despite

Accuplacer judging them so and are likely to fail

20

6) Recommendations: There is nothing you could do that would do more to increase all

your student progress and achievement rates than: 1) Determining the actual predictors of performance in your

Basic Skills sequences. 2) Aligning placement with those predictors*

If you have one, call your IE/IR office right after this meeting Start building your local database with the help of Cal-PASS**

Support the efforts of Cal-PASS, the Institute for Evidence-Based Change, and the Data Quality Campaign to make K-16 data available to all community colleges in California and nationally.

Might we suggest that you consider John Lennon’s advice from one of his later, albeit more controversial, songs…. All we are saying… is give B’s a chance