Understanding the Smarter Balanced Assessment Results SDUSD

Understanding Smarter Balanced Assessment Results Principles of Scoring Scores Students Will Receive Key Reminders While Reviewing Results Resources for More Information Today we will discuss how the Smarter Balanced Assessments are scored, the scores students receive, and some reminders while reviewing the results. I also want to share some resources with you so you know where to find more information. Assessment Services SDUSD

Principles of Scoring Computer Adaptive Testing Pattern Scoring In order to understand the results, it is important to learn how the tests are scored. Therefore, we will review computer adaptive testing and a concept called pattern scoring. SDUSD

A Computer Adaptive Test (CAT) is Based on: Large item bank - covering all areas assessed and varying levels of difficulty Recommended blueprint - focuses the selection of questions from the test bank to appropriate content so the structure of the test is similar for every student As mentioned before, in order to understand the Smarter Balanced scores, it’s important to know how a computer adaptive test works. We refer to this as the CAT portion of the assessment. The majority of the Smarter Balanced assessment is computer adaptive. A CAT is based on a very large item bank covering all areas assessed with varying levels of difficulty. There is a recommended blueprint for the CAT that ensures that the structure of the test is similar for every student. The program selects precise question based on the individual students’ responses to previous questions. The blueprint ensures that test questions are administered in a logical way and keeps the students’ experience controlled and appropriate for him or her. * Example: blueprint for a house with number and location of windows, doors, walls compared to the test blueprint with number of items per content and item type (e.g. DOK level). Assessment Services SDUSD

A Computer Adaptive Test (CAT) is Based on (Continued): Programming language (or algorithm) - a step-by-step approach that tells the CAT what to do next based on students’ answers Algorithm rules – set of rules to ensure each student’s test contains the proper types of questions and content covered Balance – enough of the concept Question type – selected response, constructed response Reading length – short, medium, long Difficulty – appropriate for grade level There is an algorithm or programming language that tells the CAT what to do next based on a student’s answers. Therefore the test adapts as the student takes the test. There are rules that ensure each student’s test contains grade appropriate questions and content providing a balance of question types, reading passage length, and difficulty. Assessment Services SDUSD

How Does a CAT Work? Example: A Student of Average Ability Expanded Very High Ability High Med-High Medium PATTERN This picture is a very simplified example of how a CAT works. The vertical axis represents the level of difficulty of the item as well as the student’s estimated ability, ranging from very high to very low. Most students will be answering questions within the high to low range. The horizontal axis represents the test question items, number 1, number 2, number 3, and so on. Under the item number there is an R or a W. R indicates a right response while the W represents a wrong response. The first test question usually will be in the medium difficulty range. This example shows that the student got number 1 correct, so the 2nd question was more challenging. The student answered the 2nd more difficult question correctly and was given another question in the difficult range. Then the student answered the question incorrectly, so for the 4th question the student was given a question within the medium-low range. As the student moves through the test, the level of difficulty for future questions are based on previous responses. You’ll notice that the two headed arrow starts getting shorter as the program becomes more familiar with the student’s pattern of responses. The algorithm allows the CAT to focus in and customize the test, but at the same time ensuring the items are aligned to the blueprint. The CAT can customize each test so it is an accurate measure of a student’s skills. Med-Low Low Expanded Very Low 1 2 3 4 5 6 7 8 9 10 Test Questions Assessment Services Answers (R/W) R R W R W W W W R R SDUSD

Scoring the CAT Final scale scores are based on item PATTERN SCORING: As a student progresses through the test, his or her pattern of responses is tracked and revised estimates of the student’s performance are calculated Scores from the CAT portion of the test are based on the difficulty of the items that were right or wrong, NOT on the total number of correct answers The test question bank for a particular grade level are designed to include an enhanced pool of test questions that are more or less difficult, but still match the test blueprint for that grade The final score is based on pattern scoring. The pattern of responses is tracked, and revised estimates of the student’s performance are calculated. Scores from the CAT portion are based on the difficulty of the items that were right or wrong, not on the total number correct. Additionally, there is an enhanced pool of test questions that are more or less difficult, but still match the test blueprint for that grade. Assessment Services SDUSD

Key Principle of Scoring Example: Johnny and Sally answered the same number of questions correctly, but Sally received a higher score because the level of difficulty of the items was higher that those answered by Johnny. Assessment Services SDUSD

SCORES STUDENTS WILL RECIEVE Reporting Scale Overall Achievement Levels Claim Level Performance Now that we understand how the majority of the test is scored, let’s explore the actual scores that students receive. We’ll review the reporting scale, the overall achievement levels, and claim level performance. SDUSD

Reporting Scale Computer-scored items are merged with hand-scored items After estimating the student’s overall performance, it is mapped onto the reporting scale Scores are on a vertical scale Expressed on a single continuum for a content area Measures student growth over time across grade levels For each grade level and content area, there is a separate scale score range After estimating the student’s overall performance, it is mapped onto the reporting scale. Scores are on a vertical scale. This will enable us to measure student growth over time across grade levels. For each grade level and content area, there is a separate scale score range. Assessment Services SDUSD

Smarter Balanced Scale Score Ranges by Grade Level Subject Min Max 3 ELA 2114 2623 Mathematics 2189 2621 4 2131 2663 2204 2659 5 2201 2701 2219 2700 6 2210 2724 2235 2748 7 2258 2745 2250 2778 8 2288 2769 2265 2802 11 2299 2795 2280 2862 The scale score is a four digit number. There is a minimum and maximum for each grade level and for each subject. Assessment Services SDUSD

Overall Achievement Levels Achievement level classifications are based on overall scores for English language arts and mathematics Exceeded the Standard Met the Standard In addition to the scale score, students will receive one of four score levels: Exceeded the Standard, Met the Standard, Nearly Met the Standard, and Not Met the Standard. Students scoring in the two top levels are on a path to be college and career ready. Nearly Met the Standard Has Not Met the Standard SDUSD

Smarter Balanced Scale Score Ranges for ELA/Literacy Grade Level 1 (Not Met) Level 2 (Nearly Met) Level 3 (Met Standard) Level 4 (Exceeded) 3 2114–2366 2367–2431 2432–2489 2490–2623 4 2131–2415 2416–2472 2473–2532 2533–2663 5 2201–2441 2442–2501 2502–2581 2582–2701 6 2210–2456 2457–2530 2531–2617 2618–2724 7 2258–2478 2479–2551 2552–2648 2649–2745 8 2288–2486 2487–2566 2567–2667 2668–2769 11 2299–2492 2493–2582 2583–2681 2682–2795 This table shows the scale score ranges for each achievement level by grade for English language arts. Assessment Services SDUSD

Smarter Balanced Scale Score Ranges for Mathematics Grade Level 1 (Not Met) Level 2 (Nearly Met) Level 3 (Met Standard) Level 4 (Exceeded) 3 2189–2380 2381–2435 2436–2500 2501–2621 4 2204–2410 2411–2484 2485–2548 2549–2659 5 2219–2454 2455–2527 2528–2578 2579–2700 6 2235–2472 2473–2551 2552–2609 2610–2748 7 2250–2483 2484–2566 2567–2634 2635–2778 8 2265–2503 2504–2585 2586–2652 2653–2802 11 2280–2542 2543–2627 2628–2717 2718–2862 This table shows the scale score ranges for each achievement level by grade for Mathematics. Assessment Services SDUSD

Claim Results: A Deeper Look Achievement levels for claims are very similar to subscores; they provide supplemental information regarding a student’s strengths or weaknesses Only three achievement levels for claims were developed since there are fewer items within each claim A student must complete all items within a claim to receive an estimate of his or performance on a claim Below Near Above Achievement levels for claims are very similar to subscores. They provide supplemental information regarding a student’s strengths and weaknesses. There are only 3 achievement levels for claims since there are few items within each claim. A student must complete most items within a claim to receive an estimate of his or her performance on a claim. The three levels for the claims are below, near, and above standard. Assessment Services SDUSD

Achievement Levels for Claims Below Standard Above Standard Near Standard English Language Arts Reading Writing Speaking & Listening Research/Inquiry For English language arts, the claims are reading, writing, speaking & listening, and research/inquiry. The claims for math are concepts & procedures, communicating reasoning, problem solving and modeling & data analysis. For each of these areas, a student will receive an achievement level: above standard, near standard, or below standard. As mentioned before, a student must answer most items for the claim in order to receive an achievement level for that claim. Mathematics Concepts & Procedures Problem Solving & Data Analysis Communicating Reasoning SDUSD

Understanding Your Child’s Score Report The following 5 minute video produced by the CDE explains the score reports mailed home to parents. https://www.youtube.com/watch?v=pTEHryn4mPI&feature=youtu.be Assessment Services SDUSD

Test Results from Multiple Years As you saw in the video, the reports include the scores from the previous years. When comparing your child’s score to the previous year, it is important that you look at the scale score in addition to the overall performance level. Assessment Services SDUSD

Low, Medium and High Bands (cont.) 2015 2432 2016 Depending on where the scale score is within the range for each performance band, you can determine your child’s growth. This example is of a student who had “Standard Met” (Level 3) in 3rd grade and “Standard Nearly Met” (Level 2) in 4th grade. At first glance, you might think this student’s performance went down over the year. However, when examining the scale score, you’ll notice that in 3rd grade the score was right on the cusp of level 2 and 3. In 4th grade the scale score is right on the cusp of level 2 and 3 again. Remember that score ranges for each level are different for each grade, and the standards for the next grade are higher than for the previous grade. As a result, students may need a higher overall score to remain in the same achievement level as the previous year. 2470 Assessment Services SDUSD

Reminders While Reviewing Results Now that we understand the scores, let’s go over some reminders as you review your child’s results. SDUSD

Rightful Place/Purpose: Assessment Frequency and Impact on Instruction The Smarter Balanced assessments are considered summative assessments given at the end of the school year. This diagram shows all of the types of assessments teachers use in the classroom including the formative processes used daily. The other ovals in the middle of the diagram represent quizzes, unit tests, and interim assessments. As represented by the size of the ovals, the formative assessments used by teachers daily have the greatest impact on teaching and learning. The formative assessments provide frequent feedback allowing the teacher to adjust his/her teaching to improve student learning. The results from Smarter Balanced are a snapshot of student performance that must be used in conjunction with multiple pieces of evidence (other measures) to arrive at a more complete understanding of your child’s learning and progress. There’s not one test that provides everything we need to know about a child’s learning. Multiple measures are imperative! Statewide Summative Classroom Formative Assessment Services SDUSD

….but they don’t tell the entire story Statewide summative assessments are like the tip of an iceberg—it pays to pay attention Smarter Balanced assessment results don’t tell the entire story. ….but they don’t tell the entire story Assessment Services SDUSD

Rightful Place, Rightful Purpose for Statewide Summative Assessment Provide a general direction—we must dig deeper to determine cause Focus on groups, programs, and disaggregation Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice Provide an entry point into a collaborative, honest conversation The results provide a general direction and help us focus on groups of students and programs. They help us compare schools within the state and determine whether we are on track, need improvement, etc. But often, the results raise more questions than providing answers. They should be an entry point to start the conversation. Assessment Services SDUSD

Resources & tools There are a lot of resources available for parents that we’d like to share. SDUSD

Resources & Tools CAASPP Results by County, District, and School- http://caaspp.cde.ca.gov/ The California Department of Education (CDE) has a website where you can review results for all schools and districts in California. You can pull two schools at once and compare results, you can look at different subgroups, and also compare results from 2016 to 2017. Assessment Services SDUSD

Resources & Tools CDE CAASPP Information http://www.cde.ca.gov/ta/tg/ca/sbacparent guides.asp The CDE has some wonderful resources like the Parent Guides that include sample questions from the test. Assessment Services SDUSD

Resources & Tools CDE CAASPP Parent Interactive Website http://www.testscoreguide.org/ca/ There is an interactive website for parents that provides even more information about the test scores. Assessment Services SDUSD

Resources & Tools Continued Online Practice Tests http://www.caaspp.org/practice-and- training/index.html We highly recommend that you review the practice and training tests if you haven’t done so already. You’ll be able to experience the different item types used for Smarter Balanced. Assessment Services SDUSD

Resources & Tools Continued CDE Common Core State Standards http://www.cde.ca.gov/re/cc/ccssresource sparents.asp Smarter Balanced Assessment Consortium http://www.smarterbalanced.org/parents/ Finally, here are some additional links for more information. Assessment Services SDUSD