Promising pathways Understanding and predicting preparation and success and its potential as a model for change for the California Community Colleges and beyond Donald Berz, Executive Vice President, Academic Affairs John J. Hetts, Director of Institutional Research Long Beach City College Critical aspects of this research were completed by our Office of Institutional Effectiveness, including our Associate Dean of Institutional Effectiveness, Eva Bagg and our two research analysts, Andrew Fuenmayor and Karen Rothstein, with key additional support from Terrence Willett, Nick Wade, and the research team at Cal-PASS

Overview 1) Background on the research 2) Placement in English at Long Beach City College 3) Predicting placement in English at LBCC 4) Predicting performance in English at LBCC 5) Alignment of placement and performance a) Building a better pathway b) Effects of misalignment 6) Recommendations 2

1) Background: Data development Worked with Cal-PASS to generate 5 year cohort (05-06 to 09-10) All courses taken at Long Beach Unified School District (LBUSD) and Long Beach City College (LBCC), California Standards Tests (CSTs), more… Key aspects of cohort Students coming directly from LBUSD Examining first English course taken at LBCC Key variables Junior & senior year English courses taken in high school and spring semester grades in those courses 11 th grade CST-ELA scores at LBUSD English courses at LBCC and grades in those courses 3

2) Fall 2010 LBUSD Cohort English 1 One level below Two levels below Three Levels Below No Assessment Total Percent10.2%34.7%N/A28.7%26.4%100% What does LBUSD’s pattern of English placement look like? 4

3) Predicting Preparation/Placement Primary method of assessing level of preparation has been: student’s performance on standardized assessment instrument (Descriptive Test of Language Skills (DTLS) and, more recently, Accuplacer) combination of assessment & essay What predicts assessed preparation? Predicting how students will score on the assessment I.e., where they will be placed in our English course sequence. 5

Predicting Placement via DTLS/Accuplacer Outcome: Whether or not students are placed into Basic Skills Key predictors: 11 th Grade – Grade in 11 th grade English 12 th Grade – Grade in 12 th grade English CST - 11 th Grade California Standards Test score in English (CST-ELA) Course taken in 12 th grade Using LBUSD’s 12 th grade English course Rhetoric & Composition as baseline for comparison Comparing 5 major groups of courses to that baseline AP English, Multicultural Literature, Film Analysis, Remedial English, Other Electives 6

Predicting Placement in English 1 (vs. any remedial English): (relative size of influence - standardized beta; i.e., how big, relatively, the impact of the variable is on placement) LBUSD Course (compared to Rhetoric and Comp) * p<.05, *** p <.0001, † = p <1 x

11 th Grade CST Accuplacer What matters most for predicting placement/Accuplacer is: 11 th grade CST scores I.e., performance on and placement using Accuplacer is almost entirely predicted by CST score in 11 th grade AP increases placement in English 1 Students 11 th grade CST score could be used instead of Accuplacer to place students effectively disregarding last 3 semesters of high school 8

4) Predicting Performance People jump prematurely to conclusions about quality of the instruction high schools are providing But that data just speaks to how students are placed into Basic Skills Need to verify which factors best predict performance Same data but using successful completion (C or better) of English courses Rhetoric & Composition again used as comparison for courses at LBUSD English course taken at LBCC English 1 used as baseline for comparison against One level below: Fundamentals of Writing Three levels below: College English Skills I 9

Predicting success in English at LBCC ( relative size of influence - standardized beta; i.e., how big, relatively, the impact of the variable is on placement LBUSD Course (vs. Rhetoric and Comp) LBCC course (vs. English 1) * p<.05, **, p<.01, *** p <.0001, **** p<.0001, ******, p< † † = p <1 x

Summary: Performance What most strongly predicts performance in our English courses is: grade in 12 th grade English Grade in 11 th grade also matters CST/Accuplacer matters less than either grade Taking AP at least as important as CST/Accuplacer Taking remedial English predicts poorer performance 11

5) Misaligned Assessment and Performance 11 th grade CST strongly predicts Accuplacer-based placement weakly predicts performance in courses. Grade in 12 th grade English strongly predicts performance Only weakly related to placement in those courses Implication: current method of placement is not well aligned with what leads to success in our classrooms Placing students based on 12 th grade course performance holds potential to more strongly align placement with likelihood of success Place students currently likely to fail in more appropriate courses Place students likely to succeed in courses more suitable to preparedness 12

Building a better pathway Based on research done for on California Leadership Alliance for Student Success (CLASS) initiative and other best practices Prescriptive course load Emphasis on Basic Skills courses Priority enrollment Emphasis on full-time course work Student Success Course Modeled on Counseling 1: Orientation for College Success Results: Higher persistence, better course success rates, higher achievement rates, smaller equity gaps Direct placement in Basic Skills A’s and B’s more likely to successfully complete in English 1 than current rate, C’s to 1 level below. 13

What this will mean at LBCC: Promise Pathways, Semester 1 14

More broadly what this means… 15

Why this matters 16

Why this will work Placement using current method LBUSD Grade Transfer Level 1 level below 3 levels below A36% 23%18% B30% 35%33% C 27% 29% 33% D 6%11% 15% F 1% Promise Pathways built on emerging new best practices Data not previously available now is, thanks to Cal-PASS Fits principles that most educators know well based on decades of experience Solves two problems at once: Students placed in courses where they are likely to fail (red in table) Students placed in courses far below that they are capable of (green in table) 17

Additional evidence Working model nearby at Grossmont- Cuyamaca 95% of their A & B students in one of their large high schools were placed into Basic Skills in English Our rate for LBUSD seniors is ~81% A & B students now placed into English 1 86% of A & B students successfully complete English 1 on their first try Only 5% would have been allowed to even try. 18

Additional implications Helps explain why acceleration and alignment work so well whenever and wherever they’re tried* Many students are prepared but are being misclassified. Acceleration works well because it places students closer to where they are fully prepared to succeed Why instructors complain about students being unable to succeed in English 1 Students placed there based on mechanism that doesn’t relate to performance 30% of them are not actually prepared despite Accuplacer judging them so and are likely to fail 19

6) Recommendations: There is nothing you could do that would do more to increase all your student progress and achievement rates than: 1) Determining the actual predictors of performance in your Basic Skills sequences. 2) Aligning placement with those predictors* If you have one, call your IE/IR office right after this meeting Start building your local database with the help of Cal-PASS** Support the efforts of Cal-PASS, the Institute for Evidence-Based Change, and the Data Quality Campaign to make K-16 data available to all community colleges in California and nationally. Might we suggest that you consider John Lennon’s advice from one of his later, albeit more controversial, songs…. All we are saying… is give B’s a chance 20