California Standards Test and CAHSEE Correlation Use of Student Data for Targeted Preemptive Intervention November 1, 2006 Dr. Janis Fries-Martinez, Principal Gerardo Loera, Assistant Principal Polytechnic High School Los Angeles Unified School District 2

In This Presentation You Will Learn: How to accurately identify 10 th grade students who need intervention for the CAHSEE before they take it in the spring How to identify a target group of students to strategically raise the percentage of students scoring Proficient or Above on the CAHSEE in order to meet AYP requirements How to predict, with significant amount of certainty, which 10 th grade students will pass and fail the CAHSEE on their initial try

No Child Left Behind and State Accountability Criteria - CAHSEE AYP and API are dependent in large part to CAHSEE scores  AYP: Annual Measurable Objectives are measured mainly by success of 10 th grade students taking the CAHSEE the 1 st time Problem: High failure rate on CAHSEE Problem: Not enough students scoring Proficient or Above on CAHSEE CAHSEE GraduationAPIAYP

Guiding Questions How do we effectively target students for pre- emptive intervention for success on the CAHSEE before they initially take it? How do we proactively identify students that need additional support? Do we know which students are likely to pass the test or get a proficient score ?

A passing score on the CAHSEE is:  350 or greater on the English Language Arts portion of the test AND  350 or greater on the Mathematics portion of the test A proficient score is:  380 or greater on the English portion of the test  380 or greater on the Mathematic portion of the test What are Passing and Proficient Scores?

Some Possible Attributes Contributing To Success On The CAHSEE? Grades/Marks Demographics  Socioeconomic status  Learning environment CELDT Scores Language Classification Special Education Status CST Scores Periodic Assessments Etc.

Studied CST/CAHSEE Correlation at Different High Schools Test Groups: Poly HS: Class of 2006 Poly HS: Class of 2007 Santee HS: Class of 2007 LAUSD Class of 2008 (37,000+ scores) LAUSD Class of 2007 (36,000+ scores)

Polytechnic High School Class of th Grade ELA CST vs. 10 th Grade ELA CAHSEE – Matched Scores Note: 800+ scores included Source: Secondary Student Information System

South LA High School #1 - Class of th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores

LAUSD Class of th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores Note: 37,000+ scores included Source: School Information Branch

LAUSD Class of th Grade Algebra I CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 23,282 scores included Source: School Information Branch

LAUSD Class of th Grade Geometry CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 8,410 scores included Source: School Information Branch

LAUSD Class of th Grade MATH CST vs. 10th Grade Math CAHSEE – Matched Scores Note: 36,190 scores included Source: School Information Branch

Math CAHSEE Predictors Mathematics – Algebra I Model 9th Grade CST score of 271 or higher on Algebra I CST predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 334 or higher predicts a Proficient score of 380 or higher on the CAHSEE Mathematics – Geometry Model 9 th Grade CST score of 214 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 280 or higher predicts Proficient score of 380 or higher on the CAHSEE (AYP)

Summary of Predictors English Language Arts Model Based on LAUSD Class of 2008 (37,000+ scores) 9 th Grade English CST score of 292 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 346 or higher predicts Proficient score of 380 or higher on the CAHSEE (To meet AYP)

Data from the LAUSD School Information Branch – Class of 2008 Model: 292 ELA CST = 350 ELA CAHSEE Predicted Results PassFail Actual Results Pass Fail This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the English portion of the CAHSEE based on the 9 th grade CST is correct 83.4% of the time. ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA score – 83.4% accurate Total Accuracy

ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA Proficient or Above score – 84.6% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 346 ELA CST = 380 ELA CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above Below Prof This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore this model is correct 84.6% of the time.

Algebra I - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 72.9% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 271 Alg. I CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass Fail This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade CST is correct 72.9% of the time.

Algebra I – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score – 85.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 334 Alg. I CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above Below Prof This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a score on the Mathematics portion of the CAHSEE based on the 9 th grade CST is correct 85.1% of the time.

Geometry - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 88.7% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 214 Geom. CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass Fail88682 This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade Geometry CST is correct 88.7% of the time.

Geometry – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 82.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 334 Geom. I CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above Fail This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9 th grade Geometry CST is correct 82.1% of the time.

Math (ALL) - “Reliability” Matrix for predicting Passing score on CAHSEE Math – 72.7% accurate Data from the LAUSD School Information Branch – Class of 2007 Model: 268 Math CST = 350 MATH CAHSEE Predicted Results PassFail Actual Results Pass Fail This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9 th grade Math CST is correct 72.7% of the time.

Math (ALL) – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 81.5% accurate Data from the LAUSD School Information Branch – Class of 2008 Model: 326 Math CST = 380 MATH CAHSEE Predicted Results Prof/AboveBelow Prof Actual Results Prof/Above Below Prof This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9 th grade Math CST (regardless of which CST test was taken such as Algebra I, Geometry, General Math etc.) is correct 81.5% of the time.

Summary of Predictors Mathematics Model Based on LAUSD Class of 2007 (36,190 scores) 9 th Grade CST score in Math of 268 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score in Math of 326 or higher predicts a 380 or higher on the CAHSEE (AYP) English Language Arts Model Based on LAUSD Class of 2008 (37,000+ scores) 9 th Grade English CST score of 292 or higher predicts a passing score of 350 or higher on the CAHSEE 9 th Grade CST score of 346 or higher predicts Proficient score of 380 or higher on the CAHSEE (AYP)

For Program Improvement Schools:  A Review of the 2006 AYP Progress Report needs to be made in order determine which subgroups, if any, fell short of meeting the Annual Measurable Objectives (AMO) Recall that the AMOs are directly associated with the percentages of 10 th grade students that score Proficient or Above on the ELA and Math portions of the CAHSEE Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

For Program Improvement Schools (cont.):  Once a subgroup has been identified, an SSIS extract can be performed to create a list of students needing pre-intervention The extract should produce a list of 10 th grade students that scored less than a 346 on their English 9 th Grade CST. This list should be sorted in decreasing order by their English 9 th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

For Program Improvement Schools (cont.):  Once a subgroup has been identified, an SSIS extract can be performed to create of list of students needing pre-intervention The extract should produce a list of 10 th grade students that scored less than a 326 on their Math 10 th Grade CST. This list should be sorted in decreasing order by their Math 9 th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the Math Model

For Program Improvement Schools (cont.):  The resulting lists are prioritized lists of students that will benefit from pre-intervention and would most likely not score a Proficient or Above on the CAHSEE without additional support. Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model

How to identify 10 th grade students that will probably fail one or both parts of the CAHSEE?  Perform an SSIS extract for 10 th Grade students that scored less than a 268 on their 9 th Grade Math CST  Perform an SSIS extract for 10 th grade students that scored less than a 292 on their 9 th Grade English CST Deciding Which Students To Target – Possible Action Plans For Applying the Predictive Model

Cynthia Lim in the LAUSD School Information Branch has offered to provide schools with technical support on identifying students at- risk of not passing the CAHSEE or not getting a Proficient or Above score. Deciding Which Students To Target – Technical Assistance

CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 Master schedule permitted for approximately 170 seats for CAHSEE preemptive intervention in Spring of 2006 Selected a window of scores predicted to be Based on student performance this resulted in a target range as follows: For ELA: CSTs of For Math: CSTs of

On average, non prepped 10 th Graders matched the predicted values. Math predicted mean score: Math actual mean score: ELA predicted mean score: ELA actual mean score: CAHSEE Preemptive Intervention Case Study – Polytechnic High School Results

ELAMath # In preemptive preparation # Predicted to pass # Actual did pass Increase % Increase11.1%+21.2 # Predicted proficient4816 # Actual Proficient6167 Increase % Increase27.1%319% CAHSEE Preemptive Intervention Case Study – Polytechnic High School Results

CAHSEE Preemptive intervention works! Preparation was a cost effective means of positively impacting the study group. CAHSEE Preemptive Intervention Case Study – Polytechnic High School Results