1. OSPI Science Assessment Updates WSTA CONFERENCE OCTOBER 18, 2014

2. Agenda Overview of science assessment system Resources Impact of new standards adoption Models

3. Presenters Kara Todd—Elementary Science Assessment Specialist kara.todd@k12.wa.us Dawn Cope—Secondary Science Assessment Specialist dawn.cope@k12.wa.us

4. Quick poll Who is in the room with us today? ◦Teachers ◦Administrators ◦High school ◦Middle school ◦Elementary

5. Science Assessment System

6. State Science Assessments Measurements of Student Progress (MSP) ◦5 th and 8 th grade ◦State Law ◦Federal Accountability Biology End-of-Course Exam (EOC) ◦State Law ◦Federal Accountability ◦High School graduation requirement

7. Scenarios and Items Scenarios provide context for sets of items. ◦Three categories of scenarios: ◦Systems – overview of a specific system ◦Inquiry – a controlled experiment or field study ◦Application – the technological design process ◦Life Science for Biology EOC ◦Physical, Earth & Space, and Life Science for the MSP The items associated with a scenario cover a range of standards & range of cognitive complexity. Items not connected to a scenario are called “Stand Alone” items.

8. Operational Scoring Items available for Operational Testing Data Review Committee Pilot Scoring Pilot Range Finding Committee Scenario & Item Pilot Testing Science Assessment Development Cycle Content Review & Bias/Sensitivity Committees Reviewed for content by OSPI/contractor Scenario & Item Writing Committee Develop Test and Item Specifications Operational Range Finding

9. Item TypeGrade 5Grade 8Bio EOC Multiple Choice20-2524-2930-34 Completion1-6 1-5 Short Answer455 Total Items303540 Total Points344045 Pilot Items555 Number and Types of Items

10. 5 th and 8 th grade Test Map EALR Percent of MSP 1: Systems (crossed with EALR 4 and alone) At least 20 2: Inquiry (crossed with EALR 4 and alone) 30 3: Application (crossed with EALR 4 and alone) 20 4: All domains of EALR 4 (alone) 30

11. Biology EOC Test Map EALR Percent of EOC 1: Systems (crossed with Life Science and alone) At least 15 2: Inquiry (crossed with Life Science and alone) 20 – 25 3: Application (crossed with Life Science and alone) 15 4: Life science domain of EALR 4 (alone) 45 – 50

12. Life Science Domain of EALR 4 Percent of EALR 4 Items Percent of EOC Points Processes in cells (LS1)40-4520-23 Maintenance and stability of populations (LS2) 30-3515-18 Mechanisms of Evolution (LS3)25-3014-16 Biology EOC Test Map

13. Statewide Measurements of Student Progress Results

14. Statewide Biology EOC Results* * Percentages do not include “Previously Passed”

15. 2015 testing windows Winter Biology EOC: January 5 - February 6, 2015, during the last 3 weeks of semester 5 th and 8 th grade MSP: ◦Paper/pencil—April 20 – May 19, 2015 ◦Online—April 20 – June 5, 2015 ◦Not part of the SmarterBalanced ELA or Mathematics assessments Spring Biology EOC: May 11 – June 12, 2015, during the last 3 weeks of the school year ◦Not part of the SmarterBalanced ELA or Mathematics assessments

16. Online testing Science MSP and Smarter Balanced ELA/Mathematics assessments will use the same online test engine. The science MSP is not adaptive. The science items are still based on the 2009 K-12 Science Learning Standards we’ve been assessing since 2011.

17. Biology EOC Graduation Requirement Legislative intent regarding high school science assessment (2013 Engrossed House Bill 1450, section 4): Classes of 2015 and beyond must pass the state science exam to graduate Biology EOC—Until NGSS are adopted, implemented, and assessed Comprehensive NGSS Test—When NGSS are adopted, implemented, and assessed

18. Who Takes the Biology EOC? Winter (January/February) 2015 Any student (class of 2015+) who took the EOC previously and did not meet standard Students completing a biology course mid-year Spring 2015 Any student (2015+) who took the EOC previously and did not meet standard Any student (2015+) enrolled in a high school level biology course All 10th graders who have not yet met standard Meets the high school science testing requirement of NCLB

19. Alternatives for ELL and Spec Ed Biology EOC (& Science MSP) available in 6 translation languages Spanish, Russian, Korean, Chinese, Vietnamese, and Somali IEP can designate Level 2 (Basic) to meet graduation requirement Developmentally Appropriate Proficiency Exam (DAPE) ES DAPE and MS DAPE available fall 2014, spring 2015 Locally Determined Assessment (12 th graders only) For more information: http://www.k12.wa.us/assessment/GraduationAlternatives/default.aspx John Bresko Special Education Program Supervisor john.bresko@k12.wa.us john.bresko@k12.wa.us Lesley Siegel Alternate Assessment Coordinator lesley.siegel@k12.wa.us lesley.siegel@k12.wa.us

20. Certificate of Academic Achievement (CAA) Options Begins with students in class of 2015 (current 12th graders) GPA comparison—draft Advanced Placement (AP) score of 3 or higher: Biology, Physics, Chemistry, or Environmental Sciences International Baccalaureate(IB) score of 4 or higher: Biology, Physics, Chemistry ACT Science score: Will be added when adequate data exists For more information: http://www.k12.wa.us/assessment/GraduationAlternatives/default.aspx http://www.k12.wa.us/assessment/GraduationAlternatives/default.aspx

21. CAA Options Available for Biology EOC Collection of Evidence (COE) Must take EOC once Two submission opportunities (January, June) Online Independent work of students 6-8 tasks chosen from a bank of 16 Inclusion Tasks Completed under the supervision of a teacher Two tasks “On Demand” Selected tasks must include items which assess each of the six reporting strands (System, Application, Inquiry, LS1, LS2, LS3) three-four times. Task Matrix: http://www.coe.k12.wa.us/Page/303http://www.coe.k12.wa.us/Page/303

22. Biology COE Resources ESD 113 http://www.coe.k12.wa.us/domain/49 Guidelines, policies, and sufficiency requirements The secure EDS System can be accessed by educators who have students working on a COE. Generate staff and student groups and accounts, access and complete tasks, submit collections Recorded webinars, sample COE tasks, Lessons Learned Task Matrix Brief description of the 16 tasks available

23. Biology COE Resources Biology COE Moodle http://moodle2.ospi.k12.wa.us/enrol/index.php?id=57 Forum Task development information Links to recorded webinars Salmonberry Habitat Sample COE task (adapted from Bio EOC scenario-Updates 2013) The Moths and the Trees Sample COE task (adapted from Bio EOC scenario-Updates 2014)-Posted October 2014 Scott Killough: scott.killough@k12.wa.usscott.killough@k12.wa.us

24. Resources MSP AND EOC

25. Resources coming soon… Lessons Learned 2014 October 2014 version of Test and Items Specifications Scorer Training Modules ◦COE, EOC, MSP ◦Videos with materials

26. Science Assessment Resources

32. Impact of new standards adoption WHAT ABOUT THE NGSS?

33. What happens to the MSP and EOC? NCLB and state law require a state science exam once each in elementary, middle, and high school. The state exams (based on 2009 standards) will continue until an exam based on the NGSS is developed. If you heard “2014 was the last year of the MSP,” add “except for science.”

34. Adopted: June 2009 Adopted: October 2013 Next Generation Science Standards First assessed: 2011 (MSP) 2012 (EOC) Possible first assessment: 2017 or 2018 (?) Washington State 2013 K-12 Science Standards

35. What are the steps to building a new assessment? Develop item specifications from the standards Determine the test map Develop test items (including a pilot test) Develop Achievement Level Descriptors (ALDs) from the standards (aka PLDs) Administer an operational test Set achievement level cut scores (aka Standard Setting) For the 2009 standards, this process took 25 months

36. Still to be determined: Will the science tests be developed by Washington or with a consortium of states? When and how will pilot and/or field testing occur? What year will be the first year of operational testing? If the high school test is comprehensive, will it be administered at grade 10 or grade 11? Will the elementary test cover just grade 5 standards or a broader grade band of standards?

37. Upcoming meetings for the new assessment: Exact dates and locations TBD. TimeframeEventNumber of days Winter 2015Test blue print review1 Winter 2015Item specifications review2 Spring 2015Item writing5 each Summer 2015Content review5 each

38. SALT and PEPPER Teams PEPPER PreSALTers Enthusiastically Promoting Powerful Educational Resources Science educators throughout Washington interested in receiving Science Assessment updates. Receive invitations to participate in development events. SALT Science Assessment Leadership Team Have participated in (or have applied to) development events such as Scenario Writing, Content Review, Range Finding, and Data Review. science@k12.wa.us

39. SALT Information: http://www.k12.wa.us/Science/ProfDevelopment.aspx http://www.k12.wa.us/Science/ProfDevelopment.aspx

40. Building Bridges 2009  2013

41. WA State 2013 K-12 Science Standards Science and Engineering Practices 1.Ask questions (for science) and define problems (for engineering) 2.Develop and use models 3.Plan and carry out investigations 4.Analyze and interpret data 5.Use mathematics and computational thinking 6.Construct explanations (for science) and design solutions (for engineering) 7.Engage in argument from evidence 8.Obtain, evaluate, and communicate information Crosscutting Concepts 1.Patterns 2.Cause and effect 3.Scale, proportion and quality 4.Systems and system models 5.Energy and matter 6.Structure and function 7.Stability and change Core Ideas 1.Physical Sciences 2.Life Sciences 3.Earth and Space Sciences 4.Engineering, Technology and Applications of Science

42. “Model” Items BRIDGING THE GAP

43. 4-5 INQF Models (WA 2009) Content Standard: o A scientific model is a simplified representation of an object, event, system, or process created to understand some aspect of the natural world. When learning from a model, it is important to realize that the model is not exactly the same as the thing being modeled. Item Specifications: o INQF(1) Tell what can be learned about an object, event, system, or process by using a given model. o INQF(2) Describe how a model is similar to or different from the object, event, system, or process being modeled.

44. 6-8 INQE Models (WA 2009) Content Standard: o Models are used to represent objects, events, systems, and processes. Models can be used to test hypotheses and better understand phenomena, but they have limitations. Item Specifications: o INQE(1) Describe how a model can be used to explain the relationship between two variables. o INQE(2) Describe how a model or simulation is similar to and/or different from the actual object, event, system or process being modeled.

45. 9-12 SYSC (WA 2009) Content Standard: o In complex systems, entirely new and unpredictable properties may emerge. Consequently, modeling a complex system in sufficient detail to make reliable predictions may not be possible. Item Specifications: o SYSC(1) Given a model of a complex system that is lacking sufficient detail to make reliable predictions about that system, describe inadequacies of the model. o SYSC(2) Predict the possible consequences of a change in a given complex system and/or describe why a simplified model may not be able to reliably predict those consequences.

50. Vague

51. Developing and Using Models (WA 2013) Include ◦Diagrams ◦Physical replicas ◦Mathematical representations ◦Analogies ◦Computer simulations Models in Science ◦Represent a system or parts of a system ◦Aid in the development of questions ◦Generate data used to make predictions ◦Communicate ideas to others ◦Evaluated and refined through comparing of model-based predictions with the real world ◦Based on evidence and modified as needed Models in Engineering ◦Used to analyze a system ◦Used to test solutions to a problem ◦Used to visualize and refine a design ◦Used to communicate design features ◦Uses as prototypes to test design performance NGSS—Appendix F, pages 52-53

53. MS Performance Expectation (WA 2013)

55. Things to remember: 1.We are just exploring possibilities. 2.This example could possibly serve as bridges between the 2009 and 2013 standards. 3.Item specifications have not been developed for the 2013 standards so the item presented is not an “example” based on item specifications. 4.We encourage you to play with these types of items and figure out what works best with kids.

56. MS-LS2-3 Model Pond ponderings…

59. Things to remember: 1.We are just exploring possibilities. 2.This example could possibly serve as bridges between the 2009 and 2013 standards. 3.Item specifications have not been developed for the 2013 standards so the item presented is not an “example” based on item specifications. 4.We encourage you to play with these types of items and figure out what works best with kids.

60. Think time… Are you able to think of some student responses? What could be added to the scenario to make this item more accessible? How many score points do you think this could be worth? Does the rigor fit this grade level? What would you do at your grade level?

61. Share out

62. Thank you! Kara Todd—Elementary Science Assessment Specialist kara.todd@k12.wa.us Dawn Cope—Secondary Science Assessment Specialist dawn.cope@k12.wa.us science@k12.wa.us