INTERPRETING SMARTER BALANCED SUMMATIVE ASSESSMENT RESULTS

Participant Outcomes Deepen our understanding of the Smarter Balanced system components Develop a common understanding on how to read and interpret student level reports Identify next steps and resources available to aid communication of student results to stakeholders

Two Buck Summary Looking at the learning goals, please write a two-buck summary on a Post-It note of what you feel is most important for you to walk away with at the end of our time together. Each word is worth 10 cents. Try to be right on budget! Next, give each person a chance to share his/her summary. $2 buck summary icebreaker to summarize what is most important to walk away from today ($.10 per word so stay on budget). You have 3 minutes to write your 20 word statement on a post-it You have 2 minutes to share your 20 word statement with your elbow partner Call on folks to share their 20 word statement: As folks share theirs aloud other will to listen to see if keywords or phrases are similar or different to what you wrote down. Facilitator identified a words such as “planning” or “reflect” that came up from those sharing. Then called for someone that had a different idea (i.e., subgroup)

Gimme 5!: Rate your knowledge of the SBAC system. I know that SBAC summative assessments in ELA and Math were given last spring in grades 3-8 and 11 only. 1 I know that the SBAC system contains more than just summative assessments. 2 I can explain the differences between the SBAC and CST to others and why we should not compare the two. 3 I am very familiar with the terminology and scoring format on the student level SBAC reports. 4 I know all the components of the student level SBAC report; I am prepared to share this with others. 5 I would like to quickly survey the room to quickly check our prior knowledge and/or your rating of yourself on the SBAC system and student level reports If you feel that your knowledge of understanding is at a 1 please put 1 finger up If you feel that your knowledge of understanding is at a 2 please put 2 fingers up If you feel that your knowledge of understanding is at a 3 please put 3 fingers up If you feel that your knowledge of understanding is at a 4 please put 4 fingers up If you feel that your knowledge of understanding is at a 5 please put 5 fingers up This is an awesome protocol for using with your staff and your staff using with students to quickly find out the comfort level or understanding of a topic

“Balanced” in Smarter Balanced Refers both to the model of an interconnected system of summative, interim, and formative components A balance between technologically advanced computer adaptive testing (CAT) and extended, thematically related performance tasks As you are fully aware of the California State Standards and the Smarter Balanced Assessments are a complete 360 from what we were doing before with the previous standards and CST. This paradigm shift has caused us to rethink and begin to change the ways in which we are instructing, assessing and monitoring students As the things that we want students to know and be able to do change (standards) so must the way that we assess what students understand and can do

Summative assessments benchmarked to college and career readiness (Grades 3-8 and 11) Standards set expectations on path to college and career readiness All students graduate college and career ready Teachers and schools have information and tools to improve teaching and learning Digital Library: Formative assessment tools and practices for teachers to improve instruction Interim assessments Flexible, open, used for actionable feedback This graphic summarizes the previous slide to help us to understand this paradigm shift. Remember the CCSS now called the California State Standards were developed with the end in mind…students graduating being CCR. Our previous standards had similar goals but the ways in which they were assessed and the accountability that came with that did not provide us with information to improve teaching and learning, rather it led to an atmosphere of teaching to the test, test prep, and narrowing the focus of instruction As you can see from the graphic, the focus is not just on the summative anymore, but there is a more balanced system that includes summative, interim and the digital library with the intent of providing the tools and assessment to gauge and improve teaching and learning The continuum on the bottom of the slide shows where we are as far as implementation of the smarter balanced system. It will probably take 3-5 years before we can get to full implementation of the SBAC system (Summative, Interims, and Digital Library) before it is embedded in our culture, so we want decrease fears that it all has to happen now Today, we are not going to talk about the entire system, we are just going to focus on the summative piece because our student results will be coming soon and we want to be able to understand them and explain them to others

In what ways are the SBAC summative assessments fundamentally different from the CST? Work with a partner to list a few fundamental differences, each on its own Post-It note. I would like us to quickly reflect on this paradigm shift that was mentioned. With a partner, use a post-it and respond to the question on the slide? Please use a post-it to jot down some differences Identify a recorder, and the recorder should jot down ideas from the entire table Now we will do a whip around and recorder will share some of the differences. If you hear an idea that you have mentioned scratch it off your list. When all the ideas have been scratched off, please take a seat, those with ideas remaining, continuing standing

Summative Assessment Grades 3-8 and Grade 11 ELA and Mathematics Computer Adaptive Test Performance Task Here are some obvious differences that you mentioned With the following exceptions, all students take the Smarter Balanced assessments: Students with disabilities who take the California Alternate Assessments (CAA) in ELA and mathematics English learners who have been enrolled in a school in the United States less than 12 months (ELA only

Interpreting Summative Assessment Results

Summative Assessment Scores Overall Scale Score Achievement Level Level 1. Standard not met Level 2. Standard nearly met Level 3. Standard met Level 4. Standard exceeded Claims Below Standard At or Near Standard Above Standard Please pull out handout #2, which are the sample student reports. A number of you rated yourself a 5, which pertains to this section and beyond, so whose ready to present? After being under the umbrella of STAR and CST for so many years we are hardwired with the CST terminology, but as we are well aware of, when systems, people, assessments change so does the lingo. On the SBA summative we have the overall scale score and 4 performance bands or achievement levels instead of 5 like the CST… The overall scale score is comprised of Claims, which is somewhat comparable to the content strands on the CST, in that they provide more detailed information about sets of skills within each content areas. The Claims have 3 achievement levels…

Individual Student Report (ISR) Handout 2 is the Student Score Report that parents will receive in the mail and schools will eventually receive a copy of There are 2 pages available The first page contains student information and a brief overview of the CAASPP , what it assesses, not to compare to the CST, and how students performed on the ELA and Math portion The second page contains information on some of the subtle differences between CAASPP and STAR, and reinforces the idea that this is a new test and that it is one measure of the student’s academic performance, the students performance on the claims, results from the science CST if applicable, and 11th grade EAP results if applicable. Let’s take a closer look at page 1 and 2

Scale Score and Achievement Levels Scaled Score Error Band (Margin of Error) Achievement Levels On the bottom of page 1 you can see Juan’s overall score scale score in ELA and Math Below the overall scale score is a performance scale (number line) with Juan’s scaled score. Below are the corresponding achievement levels, so we can see that Juan’s achievement was “Standard Met” because his scaled score is slightly higher than the cut off threshold to be at “Standards Nearly Met” We can also an Error Band below Juan’s scale score. There is an explanation of the Error Band on the bottom of the handout. The Error band is a reminder that no test can be 100% accurate, so it indicates the extent to which the score might have been different had the test been taken multiple times. On this occasion the student fell right where the dot is. This Error Band is analogous to the Margin of Error that newspapers report on public opinion surveys To the right of the Juan’s ELA results we see a summary of the graphic. Take 30 seconds to read the summary. So the summary mentions his achievement level and what it demonstrates regarding future success, it also mentions how he scored on the claims which comprises the overall score Take a look at Juan’s results on the Mathematics portion. Share with your neighbor the information that the graphic displays, read the summary to the right and share how it differs from the first summary

Breakdown of Overall Scores (Claims) The middle of page 2 details how students performed on the Claims that make up the overall score As was mentioned earlier, the overall score for ELA is comprised of 4 Claims and the overall score for Math is comprised 3 Claims There are only 3 achievement levels for the claims… There are enough questions for each claim to tell if a student is above standard or below standard, but not quite enough to say if the student is at or near the standard. However the overall score has 4 achievement levels, so when the number of questions for each area is totaled there are enough questions to identify where the student falls

Science CST, CMA, CAPA The bottom of page 2, depending on the grade of the student or assessment identified the IEP will have grade 5, 8, or 10 science CST performance or CMA or CAPA performance

Early Assessment Program Status If the student is in grade 11 they will also have Early Assessment Program status based on their overall scores, the overall scores help to determine student’s readiness for ELA or Math college-level coursework The Early Assessment Program (EAP) is a joint program of the California Department of Education, California State University, and California Community Colleges. The EAP provides you and your child with an early indication of your child’s readiness for college-level English and mathematics prior to starting his or her senior year. Page two, as appropriate for student This will be printed for all Grade 11 students, whether they checked the EAP Box or not. This particular students scored in “Standard Met” for ELA

Smarter Balanced ELA/Mathematics Threshold Scale Scores ELA Threshold Scale Scores Math Threshold Scale Scores And what we’ve all been waiting for…the threshold scale scores for each achievement band The far left column has the lowest obtainable score, Level 2 has the lowest score for a 2, Level 3 has the lowest score for a 3, Level 4 has the lowest for a 4, and the far right column is the highest obtainable scale score As you can see the thresholds are dynamic and change in every grade, which is much different than the CST The scale scores can be used to illustrate: students’ current level of achievement, growth over time in a relatively fine grained fashion, school level changes in performance, gaps in achievement among different groups of students Scale scores are the basic units of reporting. These scores fall along a continuous vertical scale that increases across grade levels

Communicating Student Results How many of you feel like your closer to that rating of 5?

What are some common questions teachers or parents may ask you about student level results? Have participants share some of the questions they might be asked Let them know that the DOI will be sending them a wealth of resources

What’s the Scenario? Locate the scenario on your table Have 1 person read the scenario out loud As a group, share ideas on the best way to respond to the scenario Share the scenarios out loud Provide each group with a scenario Have groups work together to determine how they would respond to the scenario Have each group share the scenario out loud Pass out the Talking Points to the scenarios

Results: Previous Year’s vs. This Year’s Scores and achievement levels from the Smarter Balanced assessments cannot and should not be compared with data from STAR or CAHSEE. Different standards are assessed. Different assessment methods and score scales are used. Different levels of cognitive rigor are assessed. It’s Like Apples and Oranges – you can’t compare The previous test measured different skills, in a different way

A Framework for Interpretation It’s a beginning– this year’s score is your baseline for making future growth. It’s a transition – results may show fewer students have the skills right now, but we are on the right path with new standards and assessments. It’s the information we need – to help prepare our students for success in college and careers.

Resources on Communication: http://www. cde. ca

Timeline for Release of Smarter Balanced Assessment Results ODA will meet with LDs to review preliminary school level scores August 21-28 Upon receipt of student score reports on paper, the ODA will be mailing the reports to parents and a copy to schools. Late August – Mid September LAUSD expects to receive a full student level file from CDE prior to public release. Once that occurs, results will be downloaded into MiSIS and MyData for school level access. September 9 – Mid September Release of board informative with results for LAUSD by subgroups. September 9

Next Steps 3-2-1 Reflection What are 3 things that you will do with this information back at your school-site? What are 2 concepts that you understand? What 1 question do you still have? Please remember to complete the 3,2,1 Goals sheet and leave it on your desk before you leave.