Preparing for New Test Scores  Smarter Balanced assessments measure the full range of the Common Core State Standards. They are designed to let teachers and parents know whether students are on track to be college- and career-ready by the time they graduate.  Because the new standards set higher expectations for students—and the new tests are designed to assess student performance against these higher expectations—our definition of grade-level performance is higher than it used to be.  As a result, it means that fewer students will meet grade-level standards, especially for the first few years. Results should improve as students have additional years of instruction aligned to the new standards and become better equipped to meet the challenges they present.  This does not mean that our students are “doing worse” than they did last year. Rather, the scores represent a “new baseline” that provides a more accurate indicator for educators, students, and parents as they work to meet the rigorous demands of college and career readiness. 1

Achievement Level Timeline (2014) 2

3

Online Panel  Crowd sourcing—thousands of participants  Recommendations broken down by subgroup (teachers, higher education, etc.)  The process involved: Online orientation Review of test questions Recommended a cut score for the threshold of Performance Level 3  142 Delawareans participated—exceeded SBAC’s target 4

Three Cut Scores Defines Four Performance Levels 5

Reviewing Items in the Ordered Item Booklet (OIB) for Bookmarking 6 1. What do you know about a student who responds successfully to this item; that is, what skills must a student have in order to know the correct answer? 2. What makes this item more difficult than preceding items? This activity helps you acquire the knowledge to make content-based cut score recommendations

Selection of Placement of Bookmark  A threshold Level 3 student would have about a 50% chance of success on the item just before the bookmark  A threshold Level 3 student would have less than a 50% likelihood of success on the items from the bookmark on 7

Achievement Level Timeline (2014) 8

In-Person Panel  K-12 educators, higher education, parents, and community advocates brought together  Recommended cut scores for all Performance Levels  500 participants in grade/subject panels  Nominated by states  25 Delawareans participated  Used Bookmark procedure  Three Rounds in the process 9

Reviewing Items in the Ordered Item Booklet (OIB) for Bookmarking What do you know about a student who responds successfully to this item; that is, what skills must a student have in order to know the correct answer? 2. What makes this item more difficult than preceding items? This activity helps you acquire the knowledge to make content-based cut score recommendations

Three Rounds to Determine Three Cut Scores 11

In-Person Panel Agenda  Day 1 Morning: Study Common Core State Standards and Achievement Level Descriptors Afternoon: Study the Ordered Item Booklet  Day 2 Morning: Bookmark training and Round 1  Make bookmark recommendations individually Afternoon: Round 2  Discuss differences in Round 1 bookmark placements at each table  Make Round 2 bookmarks individually 12

In-Person Panel Agenda  Day 3 Morning: Round 3  Discuss Round 2 bookmark placements for the entire room (all panelists at all tables)  View supporting data based on Round 2 bookmarks (e.g., impact data, online panel results)  Place Round 3 bookmarks individually  Review final recommendations  Evaluate the process 13

Achievement Level Timeline (2014) 14

Cross-Grade Review Committee  Also referred to as Vertical Articulation  60 panelists selected from In-Person Panel  One Delawarean participated  Reviewed Online Panel and In-Person Panel results  Recommend changes to “smooth” the data: Motion Second Discussion Vote: 2/3 majority required Aligned cut scores across grades Eliminated scaled score reversals One Delawarean participated 15

Vertical Articulation 16 This is an example to illustrate Vertical Articulation. It is NOT actual data from Achievement Level Setting.

Achievement Level Timeline (2014) 17

Setting Achievement Levels  States Chiefs review Reviewed final recommendations from cross-grade Review Committee Approval by State Chiefs  State Board review to seek approval 18

19 Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

English Language Arts/Literacy: Threshold Scale Scores 20 Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

ELA/Literacy: Estimated Percentage of Students at Each Achievement Level 21 Estimates based on Spring 2014 Smarter Balanced field test conducted in 21 states and US Virgin Islands. Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

Proposed Scale Score Cut Points ELA/Literacy Scale Score Thresholds (Scale 2000–3000) GradeLevel 1-to-2Level 2-to-3Level 3-to For Board Action in January Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

Mathematics: Threshold Scores 23 Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

Mathematics: Estimated Percentage of Students at Each Achievement Level 24 Estimates based on Spring 2014 Smarter Balanced field test conducted in 21 states and US Virgin Islands. Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

Proposed Scale Score Cut Points 25 Math Scale Score Thresholds (Scale 2000 – 3000) GradeLevel 1-to-2Level 2-to-3Level 3-to For Board Action in January 2015 Note that Achievement Level Setting Recommendations are contingent upon approval of the State Board of Education on December 18, Final vote to adopt will take place on January 15, 2015

Preparing for New Test Scores  Smarter Balanced assessments measure the full range of the Common Core State Standards. They are designed to let teachers and parents know whether students are on track to be college- and career-ready by the time they graduate.  Because the new standards set higher expectations for students—and the new tests are designed to assess student performance against these higher expectations—our definition of grade-level performance is higher than it used to be.  As a result, it means that fewer students will meet grade-level standards, especially for the first few years. Results should improve as students have additional years of instruction aligned to the new standards and become better equipped to meet the challenges they present.  This does not mean that our students are “doing worse” than they did last year. Rather, the scores represent a “new baseline” that provides a more accurate indicator for educators, students, and parents as they work to meet the rigorous demands of college and career readiness. 26

Learn More About Achievement Level Setting  Video explanation of process  Interpretation and use of scores uploads/2014/11/Interpretation-and-Use-of-Scores.pdf uploads/2014/11/Interpretation-and-Use-of-Scores.pdf  Easy to follow flow chart content/uploads/2014/09/Smarter-Balanced-Achievement-Level- Setting-Overview-.pdf content/uploads/2014/09/Smarter-Balanced-Achievement-Level- Setting-Overview-.pdf  Q&A document content/uploads/2014/11/Smarter-Balanced-Achievement-Levels- QA.pdf content/uploads/2014/11/Smarter-Balanced-Achievement-Levels- QA.pdf 