1. Presentation to the Nevada Council to Establish Academic Standards Proposed Math I and Math II End of Course Cut Scores December 22, 2015 Carson City, Nevada 1

2. Purpose: To provide information that will assist the Council to Establish Academic Standards in determining whether or not to approve the recommended mathematics End of Course Examination cut scores as proposed.  Background on Nevada’s Student Ready Assessment System  Standard Setting Process  Standard Setting Results  Superintendent's Recommendations Purpose and Agenda 2

3.  Nevada Academic Content Standards  Nevada Achievement Level Descriptors  New Assessments  Standard Setting Process  Reporting Results Nevada Student Ready Assessments 3

4. Background Nevada Student Ready Assessments 4

5.  The current Nevada Academic Content Standards (NVACS) based on the Common Core in English Language Arts and mathematics were adopted in June, 2010.  The State of Nevada has had academic content standards since 1998. Nevada Academic Content Standards 5

6. During the 2013 Legislative Session SB 288 requires (NRS 389.805), four End of Course (EOC) examinations starting with the class of 2017. The EOC exams will take the place of the Nevada High School Proficiency Examinations. Each EOC exam measures how well a student understands the subject areas tested. EOC exams were given for the first time in the spring of 2015 and were administered in the following subjects:NRS 389.805  Math I Emphasis on Algebra I  Math II Emphasis on Geometry  English language arts I with a focus on reading comprehension  English language arts II with a focus on writing 6 End of Course Examinations

7. During the current transition phase, the classes of 2017 and 2018 will only need to participate in the EOC Examinations and will not be required to earn a passing score in order to graduate. The Class of 2019 will be the first class of students required to earn a passing score on the EOC Examinations in order to graduate. 7 Transition

8.  Eligibility to take the End of Course Examinations occurs when students have taken a course or are enrolled in a course that includes the recommended standards for that End of Course Examination.  The spring 2015 administration included eligible 7 th, 8 th, 9 th, and 10 th graders. 2015 Administration of EOCs in Math I and Math II 8

9.  Operational field test in 2015.  Motivation effect due to no stakes for grades 9 and 10.  Difficulty of the Math I and Math II assessment items, based on p-values.  Student preparation for Math item types.  Instructional alignment to the needed complexity of the curriculum and assessment that align to the NVACS (inclusive of how students are asked questions) General Insights 9

10.  In collaboration with districts and RPDP staff, recommended standards were determined for the purpose of the End of Course Examinations in each subject area.  In conjunction with WestEd, a former NDE test vendor, blueprints, item specification, and items were developed and aligned to the recommended standards in fall of 2014.  NDE content staff reviewed the items in fall of 2014. End of Course Examination Activities 10

11.  In November 2014, Content and Bias Review of the items was conducted by representatives from across the state in both subject areas.  The operational field test was administered in May of 2015 for all students who were eligible.  Standard Setting was facilitated by Data Recognition Corporation (DRC) occurred to determine mathematic cut scores, November 2015. 11 End of Course Activities Continued

12. How Do You Set Achievement Expectations For Students On The Math End-Of-Course Assessments? 12

13.  Developed by Nevada educators  Reflect a sample of the knowledge and skills expected of students at each of the four achievement levels  Level 1 (lowest)  Level 2  Level 3  Level 4 (highest) Using Achievement Level Descriptors (ALDs) 13

14. Who Met to Recommend Achievement Expectations for Students on the Math EOCs? 14

15. Who Met to Recommend Achievement Expectations? 15 Yes/No Angoff Standard Setting Committee  November 16-17, 2015  14 Nevada Educators  4 Classroom teachers  5 Administrators (including District Office, Curriculum Staff, and RPDP staff)  4 Other Standard Setting Policy Review Committee  November 18, 2015  8 Nevada Education Stakeholders  5 Administrators (including District Office, Curriculum Staff, and RPDP staff)  1 State of Nevada Board Member  1 business/industry representative  1 higher education representative

16. Background Information The Standard Setting Process 16

17. What Is Standard Setting? 17

18.  The methodology used to define levels of performance and the range of scores on an assessment that correspond to those levels of performance  Level 1 (lowest)  Level 2  Level 3  Level 4 (highest) Standard Setting 18

19. Standard Setting as Part of a Process Content Standards DesignAdoptionImplementation Assessment Development Eligible Content Item DevelopmentForm Construction Setting Achievement Standards ALDs Cut Score Recommendations Policy Review 19

20.  Focuses on students’ expected knowledge and skills for each achievement level  Grounded in content expertise and guided by the achievement level descriptors (ALDs)  Item-centered method where participants determine expected performance of a student in each level for each item “Should a student who is just in Level 3 answer this item correctly? Yes or no?” Standard Setting Method: Yes/No Angoff 20

21. How Did Participants Make Recommendations? 21

22. How Did Participants Make Recommendations? 22 Review Standards and ALDs Review the Nevada Academic Content Standards for your grade. Then study the achievement level descriptors (ALDs). Study the Test Items On your own, take the student test. Make Yes/No Angoff Ratings Make your own judgments, then discuss with your group. Examine Cut Scores Review the recommended cut scores. If needed, recommend adjustments to promote articulation.  ❸❹

23. Standard Setting as Part of a Process Content Standards DesignAdoptionImplementation Assessment Development Eligible Content Item DevelopmentForm Construction Setting Achievement Standards ALDs Cut Score Recommendations Policy Review 23

24. How Did The Policy Review Committee Make Recommendations? 24

25. How Did The Policy Review Committee Make Recommendations? 25  The committee considered:  the ALDs,  the cut scores from standard setting, and  contextual information on how Nevada students performed on statewide End of Course Examinations in math.  The committee’s goal was:  to recommend a single set of cut scores to NDE for Math I and Math II, and  to make sure the system of achievement standards in math is well articulated and sends consistent signals.

26. Standard Setting + Policy Review = ? 26

27. Collecting Evidence of Validity 27  Standard Setting: Content View  Grounded in ALDs and content-based expectations for of students in each level  Committee of Nevada educators from across the state, using their content expertise  Confidence in the process  Policy Review: Systemwide View  Considered the intended effect on instruction, accountability, educator effectiveness, and policy  Recommend adjustments to promote cohesion and consistency of the system  Endorsement of the process

28. How Do Recommendations Translate Into Student Performance? 28

29. Results and Impact Data Math I and Math II 29

30. 30

32. Final Recommendations: Math I and Math II CourseLevel 1Level 2Level 3Level 4 Math I 200-488 46.5% 489-523 26.5% 524-600 21.4% 601-990 5.6% Math II 200-493 49.7% 494-538 28.9% 539-601 17.2% 602-900 4.2% 32

33. Superintendent’s Recommendations 33

34. Final Recommendations: Math I and Math II CourseLevel 1Level 2Level 3Level 4 Math I200-488489-523524-600601-990 Math II200-493494-538539-601602-900 34

35. Next Steps:  If the Council to Establish Academic Content Standards approves the cut scores as proposed, DRC will apply the cuts to the student performance. This will allow for reports to be generated to determine student achievement levels. 35 Questions and Discussion