1. Lessons Learned Over 15 Years: The California High School Exit Examination 1999 - 2014
National Conference on Student Assessment June Eric E. Zilbert Education Research and Evaluation Administrator California Department of Education

2. Presentation Overview
A short history of the CAHSEE.
Effects of high stakes on test development, administration, and reporting.
Role of the Independent Evaluator
Lessons Learned

3. A gleam in his fathers eye
High School Competency Exams developed by each district pursuant to California law.
1999, CAHSEE - state exam linked to the state’s new academic content standards
Bill to create the CAHSEE was championed by Jack O’Connell
Intended as a requirement for the class of 2004; the SBE later revised the deadline and it was officially imposed first on the class of 2006.

4. A new test is born.
Two main sections: English-language arts (ELA) and mathematics.
ELA 80 multiple-choice questions two multi-paragraph essays*.
Math 90 multiple choice questions.[6]
ELA section based on10th-grade standards, score of 60% to pass;
Math section based on grade 8 standards with partial coverage of algebra, and requires a score of 55% to pass.

5. Postpartum Depression (Grade Nine Tryout)

6. Postpartum Depression (Grade Nine Tryout)

7. The Independent Evaluation
Mandated as part of the law establishing the CAHSEE
Role of the Independent Evaluator
Monitor testing sessions
Replicate scaling, scoring and equating
Track overall passing rates for classes of students
Conduct studies of impacts on graduation and dropout rates
Collect data from students teachers, and administrators
Make recommendations

8. Purposes of the Evaluation
Quality: Does the exam provide an accurate and unbiased measure of the knowledge and skills specified in the curriculum standards to be tested by the exam?
Fairness: Do all students have adequate notice of the new requirement, opportunity to learn the material covered by the exam, and opportunity to demonstrate what they have learned?
Consequences: Do benefits from the exam, in terms of improved student achievement and other positive consequences, outweigh any negative consequences?

11. First Recommendation
“The State Board of Education, Legislature, and Governor should give serious consideration to postponing full implementation of the HSEE requirement by 1 or 2 years.”

12. Emphasis on Remediation
The Early Years
Emphasis on Remediation

13. Impacts of the Independent Evaluation
Greater public trust in the examination and the examination process
Additional source of information on how to judge the quality of assessments.
Provided capacity building assistance to the Assessment Division.
Additional advisors through the Evaluation Advisory Committee
Responsive policy related research through special studies

14. Stereotype Threat!

15. Students With Disabilities
Beginning with 2009–10 school year, students with an IEP or a Section 504 plan exempted from the CASEE Requirement.
Must satisfy all other state and local requirements for high school graduation, on or after July 1, 2009.
Until the SBE determines that alternative means to the CAHSEE are not feasible or such alternative means have been implemented.

16. 2010 Evaluation Report Recs
Recommendation 1: A pilot study is needed to try out specific criteria for meeting the CAHSEE requirement using an approach similar to that recommended by the AB 2040 Panel. The study should address the feasibility of collecting and scoring the required work samples.
Recommendation 2: The CDE should work with its CAHSEE contractor to improve the system used by districts for ordering regular and special needs versions of the CAHSEE.
Recommendation 3: California should ensure that statewide student data systems are as accurate and up-to-date as possible.

17. Recommendation 4: Collect post-high school outcome information for students who have taken the CAHSEE and use this information in reviewing the content and rigor of the CAHSEE requirements. Recommendation 5: California education leaders and educators should encourage students who do not pass in four years to continue to master CAHSEE skills and work to improve effectiveness of fifth-year programs. Recommendation 6: New interventions should be targeted at earlier grades, using test scores to identify students who have fallen behind their classmates and are at risk of failing to meet the CAHSEE requirement.

18. Alternate Means

19. Third-Party Observations of Students with Disabilities
2012 recommendation, in the event the exemption went away:
Emphasize to LEAs the importance of having students’ Individualized Education Program (IEP) decision-making teams ensure that appropriate testing variations, accommodations, and modifications (in terms of test materials, facilities, and proctoring) are offered.

20. 2013 recommendations based on third-party observations:
Provide a standard protocol or suggested policy for ensuring that students do not have access to cell phones during the exam or during breaks (e.g., have students place all electronic devices in their backpacks and take the backpacks to the front of the room). Include this information in the test administration manuals.
The goal of test variations identified specifically for ELLs is to reduce construct-irrelevance variance that is due to language. Because the creation and availability of local glossaries is variable with LEA resources, and because clearer guidance on what should be in the glossaries is not provided, variations can be a source of unfairness to students and a threat to standardization

21. Something Special
2006 Review of the Appropriateness of the California High School Exit Exam Content Standards for High School Accountability
2008 California High School Exit Exam (CAHSEE) Special Populations Study
2012 Post High School Outcomes Study
2013 Middle School English Learner Study
2015 Comparing Student Performance on CAHSEE and Smarter Balanced

22. CST
CMA
CAHSEE
People (x=10)
Theta (p=.5)

23. CAHSEE % Passing

24. Denouement

25. Denouement

26. Questions?
R I P
CAHSEE