1. LASER Convening Spokane, WA November 17, 2016
Delivered by Dawn Cope - OSPI Science Assessment
Spokane, WA
November 17, 2016
November 2016

2. Topic 1
Transitioning from the Measurements of Student Progress(MSP) and the Biology End-of-Course to the NGSS assessments
State of the current tests
Accountability years
November 2016

3. Why do we have state assessments?
The No Child Left Behind Act (NCLB) and state law require that we give a state science assessment once each in elementary, middle, and high school for the purpose of school and district accountability.
The Every Student Succeeds Act (ESSA) continues those same requirements.
State law also requires that students in the class of and beyond must pass the high school science assessment to be eligible to graduate.
November 2016
November 2016

4. What are the current state assessments?
Measurements of Student Progress (MSP) in grades 5 and 8
Biology End of Course Exam in grade 10
These assessments are based on the Washington State 2009 Science Learning Standards.
November 2016
November 2016

5. Assessment Timeline Year Grade 5 Grade 8 High School Spring 2017 MSP
Accountability
2009 Science Standards
Biology EOC
Accountability and Graduation
(Grade 10)
Spring 2018
NGSA*
3-5 band of NGSS
Middle school band of NGSS
(Grade 11)
High school band of NGSS
if needed to meet assessment graduation requirement
Biology EOC: Current 9 & 10 graders take it for accountability & graduation in 2017 (if 9th graders are taking Biology)
NGSA: Taken by 11th graders in 2018 for accountability (even though they took the Bio EOC for accountability in 2017, this happened with Math, too)
Collection of Evidence: Will be an alternative for either Bio EOC or NGSA
November 2016
*Next Generation Science Assessment
November 2016

6. Science Assessment Graduation Requirements
Current
Grade in 16-17
Class of…
Science Assessment
Science Assessment (and beyond) 12
2017
Biology EOC exam retake, as needed for graduation 11
2018 10
2019
Biology EOC for graduation and accountability unless previously passed
Next Gen Science for accountability in 11th grade (2018) 9
2020
Biology EOC for graduation if enrolled in a Biology course
Next Gen Science for accountability in 11th grade (2019)
Biology EOC: Current 9 & 10 graders take it for accountability & graduation in 2017 (if 9th graders are taking Biology)
NGSA: Taken by 11th graders in 2018 for accountability (even though they took the Bio EOC for accountability in 2017, this happened with Math, too)
Collection of Evidence: Will be an alternative for either Bio EOC or NGSA
November 2016
November 2016

7. Topic 2 Piloting Plan for grades 5, 8, and high school November 2016

8. Grades 5 and 8 Spring 2016 A limited pilot occurred
Pilot items were embedded in online operational MSPs
Approximately 8000 students saw pilot items
Spring 2017
A larger pilot will occur
Pilot items will be embedded in online operational MSPs
Approximately 56,000 students will see pilot items
Pilot forms are randomly “spiraled” to students to get a sample that represents the demographics of WA state.
Districts will be selected randomly from volunteers
Need a valid sample size
November 2016
November 2016

9. High School Spring 2017 Important Dates and Times Participation
A separate, voluntary, comprehensive, online field test will occur
Important Dates and Times
Test Window: May 1 through June 15, 2017
Test Length: 50 minutes; one class period
Registration Window: December 1, 2016 through January 31, 2017
Participation
Target population: students in Grade 11
Can also participate: students in Grades 10 and 12
Not able to participate: students in Grade 9
Other considerations
Scores will not be provided
No science course prerequisites
Science teachers will be sent the information though the science assessment listserv, and directed to tell their principals and District Coordinators if they would like to participate.
Approximately 30,000 students are needed
CTE classes (or any class) can participate in the NGSA pilot
Test length shorter than a normal NGSA test will be
There will be a “Lessons Learned” document released about the Pilot
November 2016
November 2016

10. Topic 3
Organization of the NGSS Assessments
November 2016

11. Draft Blueprints Test Organization 4 Clusters:
1-2 PEs, bundled together.
Common for all students
Clusters 1-2 are Anchor Items (Items that have been used in the past)
Clusters 3-4 are Operational Items (Items that have been piloted)
Matrix Cluster:
5th cluster, different for kids sitting in the same room
Worth fewer points
Also contains 2-4 stand alone items
Adds items
Helps with reporting at district and state levels
Field Test Items
Embedded Pilot Items – (Online only) don’t count towards student scores
Dawn Cope shared a draft diagram, it’s too “drafty” to share starewide
November 2016
November 2016

12. Topic 4
Prioritizing the Standards
November 2016

13. Test Planning Meetings-Nov 2016
The objectives for this committee were to make recommendations to OSPI about:
The NGSS Performance Expectations that should be assessed on the state science assessment and those that should be assessed in the classroom only
Content of the PE makes writing items that adequately address the PE very difficult
Requires a task or performance that cannot be simulated in an online test environment
How often each assessable Performance Expectation should be included on the state science assessments.
1 = critical, important, and necessary (assessed in approximately % of the tests)
2 = important and necessary (assessed in approximately 50% of the tests)
3 = necessary (assessed in approximately 25% of the tests)
The committees selected approximately one-third of the PEs in a domain for each priority level.
November 2016
November 2016

14. Test Planning Meetings, continued
Results will not be made public, except…
“Classroom only” designations will be included in item specifications documents
NGSS Item Specifications documents will be made available Fall 2017
November 2016
November 2016

15. Topic 5
NGSS Assessment Item Description
November 2016

16. NGSS-Aligned Item Cluster
Set of interrelated items linked to a common stimulus
Stimulus is developed around a science phenomenon
What does this look like?
November 2016

17. Cluster Map November 2016 November 2016
SEPS and CCCs may be mixed from different PEs to provide richer items in the cluster
PEs from ETSC standards are paired with other PEs . ETSC PEs need another PE as a context.
Possible reports:
“Strands” need to 3D claims, not just isolated to one dimension or one part of a dimension
“Use of SEP & CCC in Life Science”
“Use of SEP & CCC in Physical Science”
“Use of SEP & CCC in Earth Science”
ETSC PEs will be bundled with other PE’s. Need other PEs to provide a context for ETSC PEs
November 2016
November 2016

18. NGSS Standalones 1-2 points
Short stimulus that could include graphics: tables, diagrams
One part-or multiple parts
All machine scored item types are eligible except simulations
November 2016

19. Item Types Machine-scored Multiple Choice (MC) Multiple Select (MS)
Editing Task with Choice (ETC)
Matching Item (MI)
Table Item (TI)
Grid Item (GI)
Equation Builder
Simulation
Hand-scored
Short answer
We are using a variety of item types to write single-part and multi-part 2-dimensional and 3-dimensional items aligned to the NGSS.
Similar to Smarter Balanced items

20. Standalones November 2016 This is not a WA State item
This is not an item written to the NGSS
OSPI will have sample items to share after the next pilot
Take a look at Utah’s site for what items might look like on NGSA
A WA item might include more (Include an explanation?)
Utah Testing Portal “SAGE”
November 2016
November 2016

21. Stimulus and Item November 2016 This is not a WA State item
This is not an item written to the NGSS
Similar to a WA design for Cluster Items
Split Screen: Stimulus/data on Left, assessment prompt on right
Utah Testing Portal “SAGE”
November 2016
November 2016

22. Locking and Updating November 2016
Locking & updating may be used to keep students from carrying forward a mistake, i.e., student graphed data from previous item.
Not all questions will not have the lock out feature
November 2016
November 2016

23. Topic 6 NGSS Assessment Sample Item Availability November 2016
No WA Sample items available until piloting
November 2016
November 2016

24. NGSS Training Tests Available Fall 2017 Grade 5 Grade 8 High School
For all stakeholders
Will be based on piloted items from spring 2017 pilot
November 2016
November 2016

25. SAIC Grade 5 Interactive Prototype (5PS1-1 and 5PS1-2)
Center on Standards and assessment implementation
Useful for:
Cluster
Bundling PE’s
Phenomena
Multi-part items
2-D, 3-D items
Item-types
Do’s and don'ts
Used in item writing training by OSPI Assessment
Not completely ideal, meant to start the conversation
Most useful thing to do with this info is to identify flaws
November 2016
November 2016

26. Other Possible Resources…
Concord Consortium
Measured Progress reprinted sample items created by Rodger Bybee for his book, Translating the NGSS for Classroom Instruction.
Bethel School District Sample Assessments
STEM Teaching Tools
Utah’s Student Assessment of Growth & Excellence (SAGE)
November 2016

27. Topic 7 ACT/NGSS Crosswalk OSPI has not conducted an alignment study
ACT would be the best source to answer this question
November 2016

28. Topic 8
Working together…OSPI Science Assessment and OSPI Science Learning and Teaching
November 2016

29. Meetings Monthly Committee Assessment Development Meetings
National Technical Advisory Meetings
State Leadership Meetings
National Organizations and Committees
Future assessment system endeavors
Formative Assessment Tools/Digital Library
Classroom based assessments for non-tested grades
OSPI T&L & Assessment Depts working together (not typical arrangement in most states)
WA/Dawn Cope is seen as a leader at national assessment meetings
Won’t be a national consortium to develop a test
Small pockets of states collaborating
WA working with states w/similar context as us (WV, Hawaii, California)
November 2016
November 2016

30. Professional Development Opportunities
Event
Timing
Cluster Writing*
Oct 2015
March 2016
Oct 2016
March 2017
Content Review *
Dec 2015
July 2016
Dec 2016
July 2017
Bias/Sensitivity Review
Sep 2016
Pilot/Field Testing
Spring 2016
Spring 2017
Spring 2018
Pilot Range Finding*
Summer 2016
Summer 2017
Spring
2018
Content Review with Data*
Fall 2016
Fall 2017
Fall 2018
Test Planning Meeting *
November 2016
Alignment Study*
Early 2018
First Operational Tests
Achievement Level Setting*
Summer 2018
WA-AIM development begins Fall 2016

31. Science Assessment Office Staff
General account
Dawn Cope — Science Assessment Lead
Amy Viveiros — Science Assessment Specialist
Jessica Cole— Science Assessment Support
Cinda Parton—Dir. of Assessment Development
November 2016

32. Thank you!