Lessons Learned Over 15 Years: The California High School Exit Examination 1999 - 2014 National Conference on Student Assessment June 2016 Eric E. Zilbert Education Research and Evaluation Administrator California Department of Education

Presentation Overview A short history of the CAHSEE. Effects of high stakes on test development, administration, and reporting. Role of the Independent Evaluator Lessons Learned

A gleam in his fathers eye High School Competency Exams developed by each district pursuant to California law. 1999, CAHSEE - state exam linked to the state’s new academic content standards Bill to create the CAHSEE was championed by Jack O’Connell Intended as a requirement for the class of 2004; the SBE later revised the deadline and it was officially imposed first on the class of 2006.

A new test is born. Two main sections: English-language arts (ELA) and mathematics. ELA 80 multiple-choice questions two multi-paragraph essays*. Math 90 multiple choice questions.[6] ELA section based on10th-grade standards, score of 60% to pass; Math section based on grade 8 standards with partial coverage of algebra, and requires a score of 55% to pass.

Postpartum Depression (Grade Nine Tryout)

Postpartum Depression (Grade Nine Tryout)

The Independent Evaluation Mandated as part of the law establishing the CAHSEE Role of the Independent Evaluator Monitor testing sessions Replicate scaling, scoring and equating Track overall passing rates for classes of students Conduct studies of impacts on graduation and dropout rates Collect data from students teachers, and administrators Make recommendations

Purposes of the Evaluation Quality: Does the exam provide an accurate and unbiased measure of the knowledge and skills specified in the curriculum standards to be tested by the exam? Fairness: Do all students have adequate notice of the new requirement, opportunity to learn the material covered by the exam, and opportunity to demonstrate what they have learned? Consequences: Do benefits from the exam, in terms of improved student achievement and other positive consequences, outweigh any negative consequences?

First Recommendation “The State Board of Education, Legislature, and Governor should give serious consideration to postponing full implementation of the HSEE requirement by 1 or 2 years.”

Emphasis on Remediation The Early Years 2002 - 2006 Emphasis on Remediation

Impacts of the Independent Evaluation Greater public trust in the examination and the examination process Additional source of information on how to judge the quality of assessments. Provided capacity building assistance to the Assessment Division. Additional advisors through the Evaluation Advisory Committee Responsive policy related research through special studies

Stereotype Threat!

Students With Disabilities Beginning with 2009–10 school year, students with an IEP or a Section 504 plan exempted from the CASEE Requirement. Must satisfy all other state and local requirements for high school graduation, on or after July 1, 2009. Until the SBE determines that alternative means to the CAHSEE are not feasible or such alternative means have been implemented.

2010 Evaluation Report Recs Recommendation 1: A pilot study is needed to try out specific criteria for meeting the CAHSEE requirement using an approach similar to that recommended by the AB 2040 Panel. The study should address the feasibility of collecting and scoring the required work samples. Recommendation 2: The CDE should work with its CAHSEE contractor to improve the system used by districts for ordering regular and special needs versions of the CAHSEE. Recommendation 3: California should ensure that statewide student data systems are as accurate and up-to-date as possible.

Recommendation 4: Collect post-high school outcome information for students who have taken the CAHSEE and use this information in reviewing the content and rigor of the CAHSEE requirements. Recommendation 5: California education leaders and educators should encourage students who do not pass in four years to continue to master CAHSEE skills and work to improve effectiveness of fifth-year programs. Recommendation 6: New interventions should be targeted at earlier grades, using test scores to identify students who have fallen behind their classmates and are at risk of failing to meet the CAHSEE requirement.

Alternate Means

Third-Party Observations of Students with Disabilities 2012 recommendation, in the event the exemption went away: Emphasize to LEAs the importance of having students’ Individualized Education Program (IEP) decision-making teams ensure that appropriate testing variations, accommodations, and modifications (in terms of test materials, facilities, and proctoring) are offered.

2013 recommendations based on third-party observations: Provide a standard protocol or suggested policy for ensuring that students do not have access to cell phones during the exam or during breaks (e.g., have students place all electronic devices in their backpacks and take the backpacks to the front of the room). Include this information in the test administration manuals. The goal of test variations identified specifically for ELLs is to reduce construct-irrelevance variance that is due to language. Because the creation and availability of local glossaries is variable with LEA resources, and because clearer guidance on what should be in the glossaries is not provided, variations can be a source of unfairness to students and a threat to standardization

Something Special 2006 Review of the Appropriateness of the California High School Exit Exam Content Standards for High School Accountability 2008 California High School Exit Exam (CAHSEE) Special Populations Study 2012 Post High School Outcomes Study 2013 Middle School English Learner Study 2015 Comparing Student Performance on CAHSEE and Smarter Balanced

CST CMA CAHSEE People (x=10) Theta (p=.5)

CAHSEE % Passing 2001-2013

Denouement

Denouement

Questions? R I P CAHSEE 1999-2016