Raising the Bar for Oregon

Adopt New Math Cut Scores and Final Math Achievement Level Descriptors and Policy Definitions Adopt High School Math Achievement Level Descriptors & Review 3-3 Math Achievement Level Descriptors Adopt Grades 3-8 Math Achievement Level Descriptors

 New Mathematics Content Standards were adopted for grades K-8 in 2007 and high school in  Federal law requires that Achievement Standards align with Content Standards.

 Alignment Across Grades – Earlier grades should better predict performance in later grades  Students and parents need better information about the level of skill needed to succeed in High School  Decision today is essential

 District mathematics experts created the achievement level descriptors based on their in-depth understanding of the mathematics content standards  District and School mathematics experts as well as representatives from the community recommended a cut score based on their understanding of the content standards, achievement level descriptors, ODE’s projected cut-scores and a review of the Impact data

GradeNearly Meets MeetsExceeds HS *Panel Recommended 237 for HS

Recommend Cut Scores Impact Data GradeNearly Meets MeetsExceedsProjected % Meets & Exceeds % Meets & Exceeds (Actual) % 79% % 79% % 79% % 74% % 80% % 72% HS232236*25156% *Panel Recommended 237 for HS

 Districts and Schools have received repeated financial hits, and do not have additional funds for professional development  It will be easy to misinterpret the data and believe that student performance declined  AYP targets increase to 70% this year  This will create a discrepancy with Reading that will be hard for Districts to explain

 There are built-in transitions for status and “safe harbor”  ODE plans to work with stakeholders to identify strategies to address these new complexities. We will examine state and federal accountability in light of these changes

AgreeDisagreeNo Opinion Should Oregon have a goal of establishing achievement standards that are similar to other higher performing states such as Washington and Minnesota? 69.4%25.6%5.1% Are the recommended mathematics achievement standards reasonable and appropriate? For Grades 3 and %46.5%16.2% For Grades 5 and %48.6%15.4% For Grades 7 and %41.3%25.8% For High School 37.2%34.9%27.8%

 We are 3 years into the implementation of Oregon’s Math Content Standards. The Common Core standards were finalized in  Raising our Achievement Standards and expecting more of our students now will help us transition to the higher standards of the Common Core.  These new Achievement Standards bring Oregon’s standards closer to those of other states including Washington.

Recommend Cut Scores Impact Data GradeNearly Meets MeetsExceedsProjected % Meets & Exceeds % Meets & Exceeds (Actual) % 62% % 57% % 57% % 56% % 46% % 47% HS % 50%

Mathematics, grades 3-8 and High School August Presented to the Oregon State Board of Education David T. Conley, PhD Mary Seburn, PhD Liz Gilkey, JD

Bookmarking Method Modified to include comparative data Allowed panelists to compare Oregon’s current and proposed cut scores to other states and countries Included external data from NAEP, PISA, other states Included projected (derived) scores based on higher standards and increased predictive power 42 Oregon stakeholders recommended cut scores over 3 days Panelists recommended raising achievement standards for all grade levels External evaluators monitored process and documented observed evidence of validity 21

External Evaluation conducted by The Educational Policy Improvement Center (EPIC) Observed process Conducted formal and informal Interviews with participants Conducted training and workshop evaluations Documented implementation of best practices and technical adequacy Noted evidence of procedural validity as observed Noted when standards for Educational and Psychological Testing were met Documented technical evidence required by NCLB Peer Review Documented adherence to best and emerging practice 22

 Overall, feedback on the training was positive, for example: 88.1% of participants agreed or strongly agreed with the statement, “The training materials were helpful.” 92.9% of participants agreed or strongly agreed with the statement, “I am confident I understand my role in the standards verification process.” 71.4% of participants agreed or strongly agreed with the statement, “Overall, I feel well trained and prepared to complete the standards verification task.” 23

 Overall, feedback on the process was positive, for example: 81.4% of participants agreed or strongly agreed with the statement, “The Bookmark Procedure was well described.” 97.6% of participants agreed or strongly agreed with the statement, “Overall, I am satisfied with my group's final bookmarks.” 83.7% of participants agreed or strongly agreed with the statement, “I am confident that the Bookmark Procedure used produced valid cut scores.” 100% of participants agreed or strongly agreed with the statement, “I feel this procedure was fair.” 24

“It is satisfying to know the work done better reflects the rigor of “new” standards.” “It’s scary raising the scores, but it must be done to align nationally and internationally. I work in a small rural school with lots of special needs students—raising scores really hurts—but hopefully education policymakers will help us fund more help for smaller class sizes and special education case loads.” 25

26 Met? Standard ✔ 1.Panels should be large enough and representative of the appropriate constituencies. ✔ 2.Selection and qualification of participants should be documented. ✔ 3.Two panels or subpanels should be used to check the generalizability of the standards. ✔ 4.Background and demographic information about participants should be collected and documented. ✔ 5.To ensure internal validity, the methods must be consistent so that ratings indicate increased internal consistency across rounds and panelists. ✔ 6.To ensure procedural validity, the procedures must be reasonable, carried out as intended and understood by panelists. ✔ 7.The methodology should be appropriate for the assessment, described in detail and field tested when appropriate. ✔ 8.Any non-standard methodology must be clearly documented. ✔ 9.The precise nature of participants’ judgments should be documented. ✔ 10.The rationale and procedures for establishing cut scores must be documented. ✔ 11.The methods should be designed so that participants can reasonably contribute their knowledge and experience to produce reasonable, defensible standards. ✔ 12.Participants should be suitably trained on the methodology; training should include a thorough description of the method and practice exercises, practice administration of the assessment, and practice judging task difficulty with feedback on accuracy. ✔ 13.Descriptions of performance categories must be clear to the extent that participants are able to use them effectively. ✔ 14.The process should be conducted efficiently. ✔ 15.Item booklets, rating forms and other provided documents should be easy to use. ✔ 16.Facilitators should be qualified and capable of leading appropriate discussion among the participants without biasing the process. ✔ 17.Feedback to participants must be clear, understandable, and useful. ✔ 18.Participants should be instructed on the appropriate use of provided data (including performance data, impact data, criterion reference data, etc). ✔ 19.When possible, performance levels should be established using empirical criterion reference data. ✔ 20.Process evaluations should be conducted and documented. ✔ 21.The entire process must be documented, including participant selection and qualifications, training, feedback to panelists regarding their recommendations, replicability, validity and variability over participant recommendations.

 The recommended cut scores raise Oregon’s standards to 4 th (grade 4) and 5 th (grade 8) highest in the country on the NAEP scale Raised expectations for teachers and students to some of the highest in the country On par with highly competitive international standards Improved prediction of success in college math courses Improved prediction of the probability of meeting high school standards, allowing more time for intervention 27

Innovative in the provision of external reference data for consideration when setting cut scores Feedback from participating stakeholders was positive: “It was a real pleasure working with you and the entire assessment team this week in Salem. I know that the principal who initially nominated me will be thrilled to learn that I participated. Thank you for that opportunity. ” “Thanks for including me in this process. It was enjoyable, thought provoking, and extremely helpful when viewed within the context of the work being done around math and CTE programs at the community college level. ” 28