1. Understanding the Smarter Balanced Assessment Results
SDUSD

2. Understanding Smarter Balanced Assessment Results
Principles of Scoring
Scores Students Will Receive
Key Reminders While Reviewing Results
Resources for More Information
Today we will discuss how the Smarter Balanced Assessments are scored, the scores students receive, and some reminders while reviewing the results. I also want to share some resources with you so you know where to find more information.
Assessment Services
SDUSD

3. Principles of Scoring Computer Adaptive Testing Pattern Scoring
In order to understand the results, it is important to learn how the tests are scored. Therefore, we will review computer adaptive testing and a concept called pattern scoring.
SDUSD

4. A Computer Adaptive Test (CAT) is Based on:
Large item bank - covering all areas assessed and varying levels of difficulty
Recommended blueprint - focuses the selection of questions from the test bank to appropriate content so the structure of the test is similar for every student
As mentioned before, in order to understand the Smarter Balanced scores, it’s important to know how a computer adaptive test works. We refer to this as the CAT portion of the assessment. The majority of the Smarter Balanced assessment is computer adaptive.
A CAT is based on a very large item bank covering all areas assessed with varying levels of difficulty.
There is a recommended blueprint for the CAT that ensures that the structure of the test is similar for every student. The program selects precise question based on the individual students’ responses to previous questions. The blueprint ensures that test questions are administered in a logical way and keeps the students’ experience controlled and appropriate for him or her.
* Example: blueprint for a house with number and location of windows, doors, walls compared to the test blueprint with number of items per content and item type (e.g. DOK level).
Assessment Services
SDUSD

5. A Computer Adaptive Test (CAT) is Based on (Continued):
Programming language (or algorithm) - a step-by-step approach that tells the CAT what to do next based on students’ answers
Algorithm rules – set of rules to ensure each student’s test contains the proper types of questions and content covered
Balance – enough of the concept
Question type – selected response, constructed response
Reading length – short, medium, long
Difficulty – appropriate for grade level
There is an algorithm or programming language that tells the CAT what to do next based on a student’s answers. Therefore the test adapts as the student takes the test.
There are rules that ensure each student’s test contains grade appropriate questions and content providing a balance of question types, reading passage length, and difficulty.
Assessment Services
SDUSD

6. How Does a CAT Work? Example: A Student of Average Ability
Expanded
Very High
Ability
High
Med-High
Medium
PATTERN
This picture is a very simplified example of how a CAT works. The vertical axis represents the level of difficulty of the item as well as the student’s estimated ability, ranging from very high to very low. Most students will be answering questions within the high to low range.
The horizontal axis represents the test question items, number 1, number 2, number 3, and so on. Under the item number there is an R or a W. R indicates a right response while the W represents a wrong response.
The first test question usually will be in the medium difficulty range. This example shows that the student got number 1 correct, so the 2nd question was more challenging. The student answered the 2nd more difficult question correctly and was given another question in the difficult range. Then the student answered the question incorrectly, so for the 4th question the student was given a question within the medium-low range. As the student moves through the test, the level of difficulty for future questions are based on previous responses.
You’ll notice that the two headed arrow starts getting shorter as the program becomes more familiar with the student’s pattern of responses. The algorithm allows the CAT to focus in and customize the test, but at the same time ensuring the items are aligned to the blueprint. The CAT can customize each test so it is an accurate measure of a student’s skills.
Med-Low
Low
Expanded
Very Low 1 2 3 4 5 6 7 8 9 10
Test Questions
Assessment Services
Answers (R/W)
R R W R W W W W R R
SDUSD

7. Scoring the CAT Final scale scores are based on item PATTERN SCORING:
As a student progresses through the test, his or her pattern of responses is tracked and revised estimates of the student’s performance are calculated
Scores from the CAT portion of the test are based on the difficulty of the items that were right or wrong, NOT on the total number of correct answers
The test question bank for a particular grade level are designed to include an enhanced pool of test questions that are more or less difficult, but still match the test blueprint for that grade
The final score is based on pattern scoring. The pattern of responses is tracked, and revised estimates of the student’s performance are calculated.
Scores from the CAT portion are based on the difficulty of the items that were right or wrong, not on the total number correct.
Additionally, there is an enhanced pool of test questions that are more or less difficult, but still match the test blueprint for that grade.
Assessment Services
SDUSD

8. Key Principle of Scoring
Example: Johnny and Sally answered the same number of questions correctly, but Sally received a higher score because the level of difficulty of the items was higher that those answered by Johnny.
Assessment Services
SDUSD

9. SCORES STUDENTS WILL RECIEVE
Reporting Scale
Overall Achievement Levels
Claim Level Performance
Now that we understand how the majority of the test is scored, let’s explore the actual scores that students receive. We’ll review the reporting scale, the overall achievement levels, and claim level performance.
SDUSD

10. Reporting Scale
Computer-scored items are merged with hand-scored items
After estimating the student’s overall performance, it is mapped onto the reporting scale
Scores are on a vertical scale
Expressed on a single continuum for a content area
Measures student growth over time across grade levels
For each grade level and content area, there is a separate scale score range
After estimating the student’s overall performance, it is mapped onto the reporting scale.
Scores are on a vertical scale. This will enable us to measure student growth over time across grade levels.
For each grade level and content area, there is a separate scale score range.
Assessment Services
SDUSD

11. Smarter Balanced Scale Score Ranges by Grade Level
Subject
Min
Max 3
ELA
2114
2623
Mathematics
2189
2621 4
2131
2663
2204
2659 5
2201
2701
2219
2700 6
2210
2724
2235
2748 7
2258
2745
2250
2778 8
2288
2769
2265
2802 11
2299
2795
2280
2862
The scale score is a four digit number. There is a minimum and maximum for each grade level and for each subject.
Assessment Services
SDUSD

12. Overall Achievement Levels
Achievement level classifications are based on overall scores for English language arts and mathematics
Exceeded the Standard
Met the Standard
In addition to the scale score, students will receive one of four score levels: Exceeded the Standard, Met the Standard, Nearly Met the Standard, and Not Met the Standard. Students scoring in the two top levels are on a path to be college and career ready.
Nearly Met the Standard
Has Not Met the Standard
SDUSD

13. Smarter Balanced Scale Score Ranges for ELA/Literacy
Grade
Level 1
(Not Met)
Level 2
(Nearly Met)
Level 3
(Met Standard)
Level 4
(Exceeded) 3
2114–2366
2367–2431
2432–2489
2490–2623 4
2131–2415
2416–2472
2473–2532
2533–2663 5
2201–2441
2442–2501
2502–2581
2582–2701 6
2210–2456
2457–2530
2531–2617
2618–2724 7
2258–2478
2479–2551
2552–2648
2649–2745 8
2288–2486
2487–2566
2567–2667
2668–2769 11
2299–2492
2493–2582
2583–2681
2682–2795
This table shows the scale score ranges for each achievement level by grade for English language arts.
Assessment Services
SDUSD

14. Smarter Balanced Scale Score Ranges for Mathematics
Grade
Level 1
(Not Met)
Level 2
(Nearly Met)
Level 3
(Met Standard)
Level 4
(Exceeded) 3
2189–2380
2381–2435
2436–2500
2501–2621 4
2204–2410
2411–2484
2485–2548
2549–2659 5
2219–2454
2455–2527
2528–2578
2579–2700 6
2235–2472
2473–2551
2552–2609
2610–2748 7
2250–2483
2484–2566
2567–2634
2635–2778 8
2265–2503
2504–2585
2586–2652
2653–2802 11
2280–2542
2543–2627
2628–2717
2718–2862
This table shows the scale score ranges for each achievement level by grade for Mathematics.
Assessment Services
SDUSD

15. Claim Results: A Deeper Look
Achievement levels for claims are very similar to subscores; they provide supplemental information regarding a student’s strengths or weaknesses
Only three achievement levels for claims were developed since there are fewer items within each claim
A student must complete all items within a claim to receive an estimate of his or performance on a claim
Below
Near
Above
Achievement levels for claims are very similar to subscores. They provide supplemental information regarding a student’s strengths and weaknesses.
There are only 3 achievement levels for claims since there are few items within each claim.
A student must complete most items within a claim to receive an estimate of his or her performance on a claim.
The three levels for the claims are below, near, and above standard.
Assessment Services
SDUSD

16. Achievement Levels for Claims
Below Standard
Above Standard
Near Standard
English Language Arts
Reading
Writing
Speaking & Listening
Research/Inquiry
For English language arts, the claims are reading, writing, speaking & listening, and research/inquiry. The claims for math are concepts & procedures, communicating reasoning, problem solving and modeling & data analysis. For each of these areas, a student will receive an achievement level: above standard, near standard, or below standard. As mentioned before, a student must answer most items for the claim in order to receive an achievement level for that claim.
Mathematics
Concepts & Procedures
Problem Solving & Data Analysis
Communicating Reasoning
SDUSD

17. Understanding Your Child’s Score Report
The following 5 minute video produced by the CDE explains the score reports mailed home to parents.
Assessment Services
SDUSD

18. Test Results from Multiple Years
As you saw in the video, the reports include the scores from the previous years. When comparing your child’s score to the previous year, it is important that you look at the scale score in addition to the overall performance level.
Assessment Services
SDUSD

19. Low, Medium and High Bands (cont.)
2015
2432
2016
Depending on where the scale score is within the range for each performance band, you can determine your child’s growth. This example is of a student who had “Standard Met” (Level 3) in 3rd grade and “Standard Nearly Met” (Level 2) in 4th grade. At first glance, you might think this student’s performance went down over the year. However, when examining the scale score, you’ll notice that in 3rd grade the score was right on the cusp of level 2 and 3. In 4th grade the scale score is right on the cusp of level 2 and 3 again. Remember that score ranges for each level are different for each grade, and the standards for the next grade are higher than for the previous grade. As a result, students may need a higher overall score to remain in the same achievement level as the previous year.
2470
Assessment Services
SDUSD

20. Reminders While Reviewing Results
Now that we understand the scores, let’s go over some reminders as you review your child’s results.
SDUSD

21. Rightful Place/Purpose: Assessment Frequency and Impact on Instruction
The Smarter Balanced assessments are considered summative assessments given at the end of the school year. This diagram shows all of the types of assessments teachers use in the classroom including the formative processes used daily. The other ovals in the middle of the diagram represent quizzes, unit tests, and interim assessments. As represented by the size of the ovals, the formative assessments used by teachers daily have the greatest impact on teaching and learning. The formative assessments provide frequent feedback allowing the teacher to adjust his/her teaching to improve student learning.
The results from Smarter Balanced are a snapshot of student performance that must be used in conjunction with multiple pieces of evidence (other measures) to arrive at a more complete understanding of your child’s learning and progress.
There’s not one test that provides everything we need to know about a child’s learning. Multiple measures are imperative!
Statewide Summative
Classroom
Formative
Assessment Services
SDUSD

22. ….but they don’t tell the entire story
Statewide summative assessments are like the tip of an iceberg—it pays to pay attention
Smarter Balanced assessment results don’t tell the entire story.
….but they don’t tell the entire story
Assessment Services
SDUSD

23. Rightful Place, Rightful Purpose for Statewide Summative Assessment
Provide a general direction—we must dig deeper to determine cause
Focus on groups, programs, and disaggregation
Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice
Provide an entry point into a collaborative, honest conversation
The results provide a general direction and help us focus on groups of students and programs. They help us compare schools within the state and determine whether we are on track, need improvement, etc. But often, the results raise more questions than providing answers. They should be an entry point to start the conversation.
Assessment Services
SDUSD

24. Resources & tools
There are a lot of resources available for parents that we’d like to share.
SDUSD

25. Resources & Tools
CAASPP Results by County, District, and School-
The California Department of Education (CDE) has a website where you can review results for all schools and districts in California. You can pull two schools at once and compare results, you can look at different subgroups, and also compare results from 2016 to 2017.
Assessment Services
SDUSD

26. Resources & Tools
CDE CAASPP Information guides.asp
The CDE has some wonderful resources like the Parent Guides that include sample questions from the test.
Assessment Services
SDUSD

27. Resources & Tools
CDE CAASPP Parent Interactive Website
There is an interactive website for parents that provides even more information about the test scores.
Assessment Services
SDUSD

28. Resources & Tools Continued
Online Practice Tests training/index.html
We highly recommend that you review the practice and training tests if you haven’t done so already. You’ll be able to experience the different item types used for Smarter Balanced.
Assessment Services
SDUSD

29. Resources & Tools Continued
CDE Common Core State Standards sparents.asp Smarter Balanced Assessment Consortium
Finally, here are some additional links for more information.
Assessment Services
SDUSD