1. California Assessment of Student Performance and Progress (CAASPP)
California Next Generation Science Standards
(CA NGSS) Roll out Phase 3
2015–16 Assessment Literacy and Update
Don Killmer, Education Administrator I Assessment Development Office
xx minutes Running Time: xx minutes
WELCOME AND INTRODUCTIONS
Introduce session and delivery team members

2. Purpose of the Session
Provide an in-depth opportunity for CA NGSS statewide teams to learn about the purpose and features of California’s comprehensive assessment system, the CAASPP System.
Provide updates regarding the development of the statewide CA NGSS-aligned summative assessments.
xx minutes Running Time: xx minutes
Slide(s) Intent: Introduce the purpose of the Assessment Literacy Sessions
Presenter says:
It is so wonderful to have so many people in the room today representing so many schools and school districts that are ready to take their implementation of the California Next Generation Science Standards (CA NGSS) to the next level. And we are so excited to share all the information with you to help you get there, particularly as it relates to CAASPP System and science assessments.
Not only are these Sessions a chance to learn more about the CAASPP System, but they are a chance to network with other schools and school districts within the region, regarding our way forward as we put systems in place to integrate CA NGSS assessments into our statewide assessment system. Let’s take a few minutes to talk about the purpose of the CAASPP Literacy Sessions.
There are two main purposes of the CAASPP Literacy Sessions:
Provide an in-depth opportunity for CA NGSS statewide teams to learn about the purpose and features of California’s balanced and comprehensive assessment system, the CAASPP System.
Provide updates regarding the development of the statewide CA NGSS-aligned summative assessments.
We encourage you all to consider taking the information presented today and share it with your schools, districts, or counties and encourage teachers to learn more about our statewide comprehensive assessment system as they plan and work together in their local teams during this transition phase.

3. California’s Comprehensive Assessment System: An Overview
xx minutes Running Time: xx minutes
Slide(s) Intent:
Presenter says:
We are going to spend the next few minutes talking about what a comprehensive assessment looks like, including the CAASPP System to make sure that everyone in the room has a common understanding of the system, including its purpose, what it includes now and how it will expand to include more assessments in the future.

4. Rethinking the Role of Assessment
Assembly Bills (AB) 484 amended California education law to reflect the feedback from the field and policy makers:
Assessment should be more than just measurement.
Accountability alone was no longer enough.
Educators need more than a single end-of-year assessment.
Policymakers wanted more than alignment with new standards and use of technology.
“It is the intent of the Legislature…to provide a system of assessments that has the primary purpose of assisting teachers, administrators, and pupils and their parents to improve teaching and learning.”─AB 484.
xx minutes Running Time: xx minutes
Slide(s) Intent: The purpose of this slide is to introduce the roll of assessment.
Presenter says:
Assembly Bills (AB) 484 amended California education law to reflect the feedback from the field and policy makers.
A system of assessments that has the primary purpose of assisting teachers, administrators, and pupils and their parents to improve teaching and learning.

5. What is a Comprehensive Assessment System?
Provides useful information to students, parents, educators, local educational agencies (LEAs), and policymakers.
Includes multiple measures and tools, each with its own purpose:
Designed to support teaching and learning.
Facilitated by the use of a variety of delivery modes.
That are of high quality and coherently integrated.
That produce valid and reliable information used to inform instruction, program improvement, and accountability.

6. Assessment Cycles by Purpose
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the concept of assessment cycles and focus on the purpose and frequency of the different types of assessments.
Presenter says:
This graphic (Assessment Cycles by Purpose) is one that is used to describe the relationship between the two assessments. This figure is from the ELA/ELD Framework.
It demonstrates one way to think about assessment for different purposes by conceptualizing assessment as operating in different cycles: short, medium, and long.
Those assessments that are more proximate to student learning (i.e., minute-by-minute, daily, weekly) operate in a short cycle because they address a short period of teaching and learning. Short-cycle assessment serves a formative purpose because its intended use is to inform immediate teaching and learning. Assessments administered at the end of the year are long-cycle because they cover a much longer period of learning. They are primarily used for summative purposes.
Adapted from Herman and Heritage (2007)
The ELA/ELD Framework was adopted by the California State Board of Education on July 9, The ELA/ELD Framework has not been edited for publication. © 2014 by the California Department of Education.

7. What is the State’s Role?
State Administered Assessments
Used for accountability
Used for public reporting of data/Statewide “snapshot” of performance
Example: CAASPP summative assessments
State Supported Assessments
Model high-quality assessments
Means of professional development/learning
Examples: Grade 2 diagnostic, Smarter Balanced interims, and digital library
xx minutes Running Time: xx minutes
Slide(s) Intent: To provide participants an overview of the features of a comprehensive assessment system.
Presenter says:
The State Administers Assessments
Used for accountability
Used for public reporting of data/Statewide “snapshot” of performance
Example: CAASPP summative assessments
The State Supports Assessments
Model high-quality assessments
Means of professional development/learning
Examples: Grade 2 diagnostic, Smarter Balanced interims, and digital library

8. Assessment Development Update
CA NGSS-aligned Summative Assessments, including the high level test design plan
Formative tools and resources
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the assessment development update.
Presenter says:
We are going to take a few minutes to talk about where we are in terms of assessments aligned with the CA NGSS
CA NGSS-aligned Summative Assessments, including the high level test design plan
Formative tools and resources

9. CA NGSS Assessment Development
On March 9, 2016, the State Board of Education (SBE) approved the following regarding science assessments.
Grades for which the operational CA NGSS-aligned summative assessments will be administered.
Test Design Plan.
Approval allows the CDE to initiate the development of the CA NGSS assessments.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the CA NGSS assessment development update.
Presenter says:
On March 9, 2016, the State Board of Education approved the following regarding science assessments.
Grades for which the operational CA NGSS-aligned summative assessments will be administered.
Test Design Plan.
Approval allows the CDE to initiate the development of the CA NGSS assessments.

10. Approved CA NGSS Summative Assessments
The SBE approved the development of California Science Tests (CASTs); 3 online summative assessments:
Grade five assessment, consisting of grade five performance expectations (PEs) and matrix sampling of PEs from kindergarten through grade four;
Grade eight assessment, consisting of middle school (grades six through eight) PEs;
Grade ten, eleven, or twelve assessment, consisting of high school PEs.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Approved CA NGSS Summative Assessments.
Presenter says:
The SBE recommended testing at grade five and grade eight because they are the culminating grades for the elementary and middle school levels, respectively.
At the high school level, the SBE recommended allowing schools the option to choose between testing at grades ten, eleven, or twelve for individual students.

11. CAST Design Plan
Educational Testing Service (ETS) is the contractor charged with the development of the assessment design and implementation of the new CA NGSS-aligned tests.
The CAST Design Plan was developed with input from a team of state and national content experts, researchers, and psychometricians.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Assessment Design Plan.
Presenter says:
We are going to take a few minutes to talk about the test design plan that was developed by ETS, the current testing contractor, and approved by the SBE.
The proposed design was developed based on extensive feedback from California educators and other stakeholders and created with the guidance of CDE leadership and ETS’s Assessment Design Team.
The Assessment Design Team was made up of some of the country’s leading experts on NGSS, assessment design, innovative item types, psychometrics, accessibility, and computer-based assessment delivery.

12. Role of Educators and Other Stakeholders in Assessment Design Process
Design was informed by feedback provided by:
Hundreds of CA science teachers, including representatives of the California Science Teachers Association (CSTA)
Higher education officials, including representatives of Stanford University’s NGSS Assessment Program (SNAP)
STEM reform experts, including representatives from the National Research Council’s (NRC’s) Framework for K-12 Science Education
Reps from various other advocacy groups
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Assessment Design Plan.
Presenter says:
Stakeholders included:
1. hundreds of California science teachers, including representatives of the California Science Teachers Association;
2. higher education officials, including representatives of Stanford’s NGSS Assessment Program (SNAP);
3. science, technology, engineering, and mathematics (STEM) reform experts, including representatives from the National Academy of Science’s National Research Council Framework for K–12 Science Education;
4. leaders from California’s high-tech industry, and other national science education reform leaders; and
5.representatives of various advocacy organizations.

13. Stakeholder Feedback Priorities for CA NGSS Assessments
Focus on providing information to support the improvement of teaching and learning.
Promote the dramatic shift in science instruction across all grades.
Reflect fidelity to the CA NGSS.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Stakeholder Feedback Priorities for CA NGSS Assessments
Presenter says:
In addition to collecting the advice provided by stakeholders in the Next Generation Science Standards Systems Implementation Plan for California, the CDE and ETS consulted with a large and diverse group of experts to gather feedback on priorities for the state’s new CA NGSS assessments.
During these events, stakeholders overwhelmingly expressed a strong desire that the CA NGSS summative assessments:
focus on providing information to support the improvement of teaching and learning; and
promote the dramatic shift in science curriculum and instruction called for by CA NGSS across all grades, not just at the grades where California elects to administer a CA NGSS summative assessment.
Note: Important to emphasize goals for #1 are about SUPPORTING teachers and schools in delivering high-quality science instruction, and to clarify that emphasis on group-level scores does not equate to emphasis on accountability

14. CAST Design Goals Goals for the design are to:
Emphasize importance of group-level results to promote improvements to teaching and learning.
Provide models of high quality, CA NGSS-aligned assessment items.
Create incentives for schools to provide science instruction in every grade, not just in tested grades.
Measure the range and depth of CA NGSS PEs by leveraging the state’s distinctly large student population.
Minimize testing time and costs.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Assessment Design Goals.
Presenter says:
Goals for the design are to:
Emphasize importance of group-level results to promote improvements to teaching and learning.
Provide models of high quality, CA NGSS-aligned assessment items.
Create incentives for schools to provide science instruction in every grade, not just in tested grades.
Measure the range and depth of CA NGSS PEs by leveraging the state’s distinctly large student population.
Minimize testing time and costs.

15. CAST Design Features
Assessment design measures the range and depth of CA NGSS PEs over a three-year cycle.
Assessment items, generated by evidence- centered design (ECD) based task models, each integrate a minimum of two CA NGSS dimensions:
Disciplinary core ideas (DCIs)
Science and engineering practices (SEPs)
Crosscutting concepts (CCCs)
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Assessment Design Features.
Presenter says:
The assessment program will be designed initially to measure, over a three-year cycle, all CA NGSS PEs from K–12. However, the process of developing items will inform the extent to which all CA NGSS PEs can be validly and reliably assessed.
Assessment items, will be generated based on evidence-centered design (ECD) based task models, each item integrate a minimum of two CA NGSS dimensions:
Disciplinary core ideas (DCIs)
Science and engineering practices (SEPs)
Crosscutting concepts (CCCs)

16. CAST Design Features (cont.)
Design makes use of a diverse range of item types.
Both independent items as well as item sets are used.
This is a two-stage adaptive assessment.
Uses partial matrix sampling of content (PEs)
Group level feedback while ensuring individual student performance is measured fairly and comparably
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the assessment development update.
Presenter says:
The proposed design calls for two-stage adaptive assessments that measure the full range and depth of the CA NGSS content standards or performance expectations (PEs), by leveraging the state’s very large student population.
This design, known as “partial matrix sampling,” will provide the robust and broadly-based group-level feedback needed to support teaching and promote curriculum improvement, and while at the same time, ensuring that each student is measured fairly and comparably.
Furthermore, it does so while keeping the testing time required of each student within reasonable limits.

17. CAST Design Features (cont.)
Administered at grades five, eight, and ten, eleven, or twelve.
The assessment is designed to be administered in two hours or less.
The design requires that all students in the tested grades (5, 8, and 10, 11, or 12) participate in three segments of the test: Segment A, Segment B, and Segment C.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce the Assessment Design Features.
Presenter says:
The design requires that all students in the tested grades (5, 8, and 10, 11, or 12) participate in three segments of the test, each of which serves a specific purpose to further the goals outlined in the next few slides.

18. Summative Segment A
The design requires that all students in the tested grades (5, 8, and 10, 11 or 12) participate in three segments of the test:
Segment A is a two-stage adaptive segment
Machine-scorable short answer and selected response items that cover a very broad range of the CA NGSS PEs
Contributes to student and group scores
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce Segment A.
Presenter says:
Segment A is a two-stage adaptive session that presents items that are substantively equivalent in content but which may differ in difficulty so as to most appropriately match each student’s level of performance.
The session includes machine-scorable short answer and selected response items that cover a very broad range of the CA NGSS PEs.
This segment contributes substantially to the measurement precision of student scores.

19. Transition from Segment A to Segment B
Performance in Segment A guides selection of DCIs presented in Segment B.
The assignment of the item content (DCIs) in Segment B will be random unless DCI performance in Segment A is particularly weak.
xx minutes Running Time: xx minutes
Slide(s) Intent: To transition from Segment A to Segment B.
Presenter says:
Note:
Please do not use the word “Screener”, instead refer to the function of Segment A as “selecting” DCIs to for Segment B to promote a student’s best possible performance there.
A screens out some DCIs if performance is especially weak.
The content presented in segment Bis adapted to the student’s performance in Segment A.
Where a student demonstrates very weak performance in a particular CA NGSS DCI in Segment A, the student will not be presented with items based on the same CA NGSS DCI(s) since to do so would yield little or no additional measure of student performance.
In this way, Segment A’s adaptive function is as a “screener” rather than as a “router.” Where a student performs relatively equally in all CA NGSS DCIs in Segment A, the assignment of items presented in Segment B will be random. Performance on this segment contributes to the student score.

20. Summative Segment B
Segment B includes item sets which require students to solve a series of complex problems set in discipline-specific contexts which deeply measure a student’s command of selected CA NGSS PEs.
Contributes to student and group scores.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce Summative Segment B.
Presenter says:
Segment B includes item sets which require students to solve a series of complex problems set in discipline-specific contexts which deeply measure a student’s command of selected CA NGSS PEs.

21. Summative Segment C
Segment C includes a range of items that, collectively, broadly and deeply measure the CA NGSS PEs associated with the tested grade-span.
All CA NGSS PEs (minus those identified as not suitable for summative assessments) will be collectively assessed at the group-level.
Encourages the teaching of science at all grade levels
Contributes to group-level scores.
xx minutes Running Time: xx minutes
Slide(s) Intent: To introduce Summative Segment C.
Presenter says:
Segment C presents students with a range of items that, collectively, broadly, and deeply measure the full universe of CA NGSS PEs associated with the tested grade-span.
Because all CA NGSS PEs will be collectively assessed at the group level, California science educators will be responsible for ensuring that all students are instructed in the full range of the standards.
This mitigates against teachers/schools electing to focus instruction only on particular portions of the CA NGSS. Performance on this segment contributes only to group-level scores.

22. CAST Design Summary Segment A Segment B Segment C Reporting Level
Contributes to Student and Group Score
Contributes to Group Score
Scope and Depth of Measurement
Wide breadth—measures a broad sample of CA NGSS PEs
Deep measurement of targeted sample of 1-2 item sets (multiple CA NGSS SEPs and CCCs per item set, and one or more DCIs per item set as specified by task models)
Broad and deep-full range of measurement of PEs for each grade span
Type of Items
Selected-response, machine-scorable items. Items primarily independent but some organized in sets.
Item sets which require students to solve a series of complex problems set in discipline-specific contexts.
Will emulate item types presented in Segment A or B.
CA NGSS PEs measured by grade/grade span
Gr. 5: Grade 5-specific PEs
Gr. 8: Grade-span PEs
Gr. 10/11/12: Grade-span PEs
Gr. 5: Grade-span PEs
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Design Summary.
Presenter says:
This table presents an overview or summary of the proposed design for the CA NGSS summative assessments.

23. State Board of Education Approved Recommendation
The CDE recommends the use of a collection of embedded performance tasks as the basis for the CAA for Science based on:
its measurement features;
its alignment with our guiding principles; and
stakeholder feedback.
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Assessment Implementation Timeline.
Presenter says:
The proposed timeline for the development of the new CA NGSS summative assessments:
Pilot test/item tryout: spring 2017
Field test (census): spring 2018
Operational: spring 2019

24. Summative Assessment based on Embedded Performance Tasks
An assessment based on student performance on a collection of instructionally-embedded performance tasks/assignments. For the CAA pilot tests these will be state-developed classroom tasks/assignments. Moving forward the CDE continues to investigate the use of locally-developed classroom tasks/assignments (state-approved) or a combination of state- and locally-developed tasks/assignments.
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Assessment Implementation Timeline.
Presenter says:
The proposed timeline for the development of the new CA NGSS summative assessments:
Pilot test/item tryout: spring 2017
Field test (census): spring 2018
Operational: spring 2019

25. Summative Assessment based on Embedded Performance Tasks (cont.)
Tasks and assignments:
administered by the student’s primary teacher at state-specified intervals
student responses scored according to state-defined criteria and protocols
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Assessment Implementation Timeline.
Presenter says:
The proposed timeline for the development of the new CA NGSS summative assessments:
Pilot test/item tryout: spring 2017
Field test (census): spring 2018
Operational: spring 2019

26. CA NGSS Assessment Implementation Timeline
Year
California Science Test (CAST)
Available Testing Window
California Alternate Assessment for Science
2016–17
Pilot test
March 20, 2017 to the LEA’s last day of instruction
2017–18
Field test
TBD
2018–19
Operational test
*Second Tuesday in January to the LEA’s last day of instruction
2019–20
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Assessment Implementation Timeline.
Presenter says:
The proposed timeline for the development of the new CA NGSS summative assessments:
Pilot test/item tryout: spring 2017
Field test (census): spring 2018
Operational: spring 2019

27. CA NGSS Assessment Implementation Timeline (cont.)
Beginning in March 2017, all students in grades five and eight and grades 10, 11, or 12 will take either the CAST or eligible students may take the California Alternate Assessment for Science.
The CDE continues to await a response from the U.S. Department of Education on our waiver request.
xx minutes Running Time: xx minutes
Slide(s) Intent: To present the Assessment Implementation Timeline.
Presenter says:
The proposed timeline for the development of the new CA NGSS summative assessments:
Pilot test/item tryout: spring 2017
Field test (census): spring 2018
Operational: spring 2019

28. How to Get Involved in the Development of the CA NGSS
As California moves through its transitional phase toward the implementation of the CA NGSS summative assessments (including CA NGSS alternate assessment) there are many opportunities for professional development.
Opportunities for CA educator involvement include:
Item writing
Item review
Data review
Form review
To get involved complete the content
reviewer application at

29. Reflection and Questions
As a team, identify where your district/school falls in terms of transitioning to new science assessments and share any ideas to improve the teaching and learning of the CA NGSS.
xx minutes Running Time: xx minutes
Slide(s) Intent: The purpose of slide is to assess your systems readiness to implement a CA NGSS assessments, and to invite questions from the participants.
Presenter says:
Based on where your district falls along the CA NGSS implementation process, please identify where your district/school falls in terms of transitioning to new science assessments and to improve the teaching and learning of the CA NGSS.
It is important to note that your district or school may be in different places along the process.

30. Contact Information
Assessment Development and
Administration Division
(916)
California Assessment of Student
Performance and Progress
Assessment Development Office
(916)
xx minutes Running Time: xx minutes
Slide(s) Intent: To provide CDE assessment contact information.
Presenter says:
Presenter thanks the participants and invites them to contact the CDE for additional information regarding assessments.