1. Making Connections to Summative Assessment Results for Administrators, Teachers and Parents
2 minutes
Welcome
Introduce yourself and co-presenter(s)
Points to make:
New assessments provide us with opportunities to tie assessment into the instructional cycle
The summative assessments have a particular place in the arena of assessments
The new summative assessments is different from our previous statewide test, which was primarily used for accountability.
The legislature’s intent was that this new assessment system is to improve teaching and learning
This is not to say that we don’t have an accountability piece; but for the first time the legislature’s language called out explicitly how this assessment ties into the instructional cycle.
June 22, 2016

2. Workshop Goals Participants will learn:
the rightful place and purpose of the summative assessments
to use results to inform the instructional process by connecting curriculum, instruction and assessment
to access supplementary resources and reference materials to assist local training
to use tools to reflect on current practice to improve teaching and learning
a variety of ways to display data to communicate a particular message
Take a minute and read the workshop goals. Identify the goal or goals that is most relevant and important to the work that you do. Turn to your elbow partner and share which goal(s) you identified and why.
3 min
Points to make:
What you’ve seen so far was all about what results are available to whom and how to find those reports and data sets
This afternoon, we will be talking about the rightful place and rightful purpose of the summative assessment
When you leave today, we want to make sure you take away a sense of where this assessment fits into the myriad of assessments you give in your district as well as where these assessments and their results fit into the instructional planning cycle.
Our presentation is organized into these topics:
Purpose of Summative Assessment
Evidence-Centered Design (connecting teaching and learning to the assessment design)
Review of Available Summative Scores
Tools for Analysis
Putting It All Together
Telling the Story with the Data
Applying the Learning
We will start with an overview of the intended purpose of summative assessment. Then we will talk about how the Smarter Balanced tests were constructed, and why that knowledge may be important to classroom teachers. We’ll quickly review the scores that we get back from the summative assessment, and we’ll consider tools to support the practice of data analysis and how to understand what the summative assessment results mean. We’ll practice using a data template with different data scenarios, consider communicating results to important audiences, and last, think about next steps in applying this learning from today to your LEAs instructional cycle.

3. A Balanced Assessment System
Summative assessments
benchmarked to college and career readiness
(Grades 3–8 and Grade 11)
Teachers and schools have information and tools to improve teaching and learning
Standards set
expectations on path to college- and career- readiness
All students graduate college- and career- ready
30 sec
Points to make:
The path toward College and Career Ready was initiated by rigorous Common Core State Standards and realized by Teachers and Schools collaborating to unpack the layers of these complex standards and implementing the instructional tools to improve teaching and learning.
Smarter Balanced is a balanced system that supports this path toward CCR via the different types of assessments for different purposes:
Resources to support the formative assessment process (daily and weekly) to improve instruction
Interim (throughout the year) to monitor learning
Summative (end of year) to inform the instructional program and planning
CLICK Today, we focus on the summative component
Digital Library Formative assessment tools and practices
for teachers to improve instruction
Interim assessments Flexible, open, and used for actionable feedback

4. Rightful Place/Purpose: Assessment Frequency and Impact on Instruction
3 min
(Note “CLICK” embedded in text below and in slides that follow)
Points to make:
This graphic is an adaptation of one that appears in the ELA/ELD framework and the draft Science Framework for California Public Schools
We’ve changed the size of the circles to represent the amount of information that teachers gain from each of these types of assessment and the frequency of the assessment CLICK
The classroom formative assessment process, the minute by minute and daily formative process, elicits much more detail about a student than a summative assessment and yield more information with respect to a particular learning objective. CLICK Very frequent, large impact on instruction.
Summative assessments, as indicated by the very small circle, do not give teachers the same type of immediate actionable information, particularly for individual students.
If we think about the frequency of information, and potential for impact on teaching and learning from the different types of assessments, the classroom formative assessment process provides teachers with much greater information - more often - and are more immediate actionable information.
The summative assessment have a purpose and place, but the expectation that they would provide the same level of information as classroom assessments is not reasonable nor appropriate.
Turn to an elbow partner and tell that person what I just told you. (1 min, quickly) (if you need to)
Statewide Summative
Classroom
Formative

5. Statewide summative assessments are like icebergs—it pays to pay attention
2 min
Points to make:
Summative assessments are like an iceberg (simile).
When you see ice out on the ocean, you know there’s a lot more to it.
Summative assessment is the tip of this iceberg, similar to the very small circle in the other graphic.
The other types of rich assessment – formative, diagnostic, interim – all support the tip of the iceberg.
There is a lot of information below the surface that is not available from summative assessment, and we have to dig deep to illuminate what we see on top.
Summative assessments provide information that can be very important; that we need to pay attention to, but they don’t provide the whole picture.
Summative assessments can serve as a beacon (just like the sighting of an iceberg)
Slow down
Go back
Change direction
Stay on course
If we ignore the summative assessment results, if we don’t look deeply into how different groups performed, and don’t look at additional data to confirm what we think we are learning from the summative results, we run the risk of thinking we are doing fine when we are not.

6. Rightful Place, Rightful Purpose for Statewide Summative Assessment
Provide a general direction—we must dig deeper to determine cause
Focus on groups, programs, and disaggregation
Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice
Provide an entry point into a collaborative, honest conversation
Provide a necessary story
1 min
Points to make:
What is it that summative assessments are good for?
There are limitations of summative assessment. Because of the frequency and the granularity of information, these results are not necessarily immediately actionable.
For example, a final examination is a summative assessment. Think about why you give a final. Not to diagnose, but to learn what students know and are able to do.
The Smarter Balanced summative assessment is similar but much broader in its content.
Summative assessments don’t provide ready answers; they provide a general direction about where we might go next.
They do raise a host of questions: Do our other data tell us the same thing? Is this an indicator of what we did during the year that students didn’t understand?
If appropriately used, in a collegial, comfortable environment with support, they provide an entry point into good conversations about how the students did, and what we might do differently to improve student performance.
They also can tell a necessary story as well. The summative assessment results can serve as an advocate for children who cannot speak for themselves; the undereducated, the disenfranchised. These results are important and should never be ignored, but they must be viewed in the right context. Results from the summative assessment can provide the necessary “story” to provide the impetus to make change for those students who need help.
Think about LCFF, these results play and important part in developing our LCAPs as we move forward.

7. Using Summative Results
Comprehensive, complex, and difficult process
Not a checklist
Not meant to be completed at a single staff meeting or professional development event
Focus on improving learning
Not solely about increasing scores
Collaborative process that requires:
Honesty
Willingness to commit the time
Ability to handle ambiguity
Patience
1 min
Points to make:
(After lunch or next) we are going to talk about the scores and how to use the resources provided by Smarter to help us interpret the results..
Using results appropriately and getting the most out of them is a complex process.
Let’s go back to an earlier statement about the need to change our behavior, and to use these results differently. With the CST, we often structured an hour during which grade level or content area teams would consider the current year’s results and trends, come to conclusions and move on. The process needs to be different as we look at the Smarter Balanced results. The ability to identify implications for teaching and learning will take time and will need to be supported by professional learning opportunities. It won’t be accomplished in a single professional learning event.
There are often no easy answers to the questions that these data raise. Teachers and administrators will need to feel comfortable talking about the results in an honest way and will need to be able to talk to each other about both the reasons students got the scores they got and changes to local policies, programs, and practices that might be considered.

8. . . . about the rightful place/purpose of summative assessment results
How do you currently use statewide summative assessment results in the curriculum, instruction, and assessment alignment process within your LEA?
What is the place and purpose of the summative assessment in your LEA right now?
6 minutes
(4 for them to talk; two to get quick feedback)
This section takes around minutes.

9. 3 min
Points to make:
This slide provides a graphic representation of the relationship of the claims to the content categories, assessment targets, and the related standards in the CCSS. Each of these components was important to creating the ALDs.
(walk thru the components of the graphic, and point out the labels at the bottom. Mention that these were used in the CAASPP Institutes and other workshops, so many folks have likely seen this graphic before.)

10. Math and ELA ALD’s, Claims, and Targets
TCDE Website
Math and ELA
ALD’s, Claims, and Targets

11. Review of Available Summative Scores
30 seconds
Point to make:
Let’s look quickly at what kinds of scores you get for the summative assessment.

12. Available Summative Assessment Scores
Scale scores
Achievement levels
not met, nearly met, met, exceeded
Claim levels
below, near, above
Target scores
worse, similar, better
insufficient information
1 minute
Points to make:
The scores we get from the summative assessment, for both individual students and groups of students, are:
Scale scores
Achievement levels – Levels 1 through 4, or in CA, not met, nearly met, met, and exceeded
Claim levels – below standards, near standard, and above standard
Target scores – worse, similar, better and insufficient information

13. Scale Score – Achievement Level Relationship – Grade 3 Example
Scale Score Range
Smarter Level
CA Achievement Level
≤ 2366
Level 1
Standard Not Met
2367 ―
2431 2
Standard Nearly Met
2432
2489 3
Standard Met
≥ 2490 4
Standard Exceeded
2 min
Points to make:
The achievement levels are a nice way to talk to parents about their student’s score. They have meaning from grade to grade, and allow us to show improvement over time in a way the CSTs didn’t.
They also give us a nice way to consider the performance of groups of students. For purposes of broad interpretation, it helps give us a sense of where they fit within the larger group.
They can be particularly helpful as we look at scores from year to year.

14. Smarter Balanced ELA Threshold Scale Scores
2 min
HANDOUT
Points to make:
This is a visual representation of the Threshold Scale Scores for the ELA achievement levels.
Your handout shows both the ELA and mathematics threshold scale scores.
Important to note that these scores do go up from grade level to grade level.
It’s important to use these threshold scores as you talk about performance across groups of students. Each level is comprised of a range of scores. The scores shown here are the lowest possible scale scores a student could receive in that level. This visual shows ELA, but the visual for mathematics is very similar and is shown in the handout.
Grade
Level 2
Level 3
Level 4
Grade 3
2367
2432
2490
Grade 4
2416
2473
2533
Grade 5
2442
2502
2582
Grade 6
2457
2531
2618
Grade 7
2479
2552
2649
Grade 8
2487
2567
2668
Grade 11
2493
2583
2682

15. Low, Medium and High Bands
(Divide the difference by three = 21 pts)
(Divide the difference by three = 21 pts)
3 minutes
HANDOUT
Points to make:
As you likely know, each of the achievement levels contains a range of scale scores. The range for Level 2, or Standard Not Met is 2367 to Any student with a score that falls within this range falls into achievement Level 2, or Standard Not Met. Any student with a score between 2432 and 2489 falls into Level 3, or Standard Met.
Each student, no matter the score within the range, is given the same achievement level.
As we communicate these achievement levels, particularly as we talk about improvement, or growth, from year to year, it might be helpful to think of scores within the range as high, medium, or low.
With this graphic, we offer a way to look at performance and improvement over time by dividing the ranges into bands. CLICK
Let’s use the Grade 3 scale scores divide the Level Two range into three parts. CLICK CLICK CLICK
That gives us the opportunity to look the average of a group of students, as falling into the low, medium, or high end of the range for Level 2
Similarly we have divided the range of Level 3 scores into three equal parts, again low, medium, and high. CLICK CLICK CLICK CLICK
The handout you have shows each grade level with the same information. CLICK (to show grade 4).
Please note:  This strategy of using low, medium, and high bands is an attempt to contexualize the performance in a more specific way.   The break points are placeholders to begin the conversation.
While we are focusing on the use of the scores at the group level, a teacher might be able to use this information when talking to a parent at a parent/teacher conference. It also helps to emphasize that these scores are “estimates” of a student’s performance.
2367 to
2387
2388 to
2408
2409 to
2431
2432 to
2450
2451 to
2469
2470 to
2489
(Divide the difference by three = 21 pts)
(Divide the difference by three = 21 pts)
2416 to
2433
2434 to
2452
2453 to
2472
2473 to
2491
2492 to
2511
2512 to
2532

16. Low, Medium and High Bands (cont.)
2015
5 min
Points to make:
Now let’s map some scores onto this graphic.
Let’s say a group of students, received a scale score of 2432 in grade three in That puts that group of students at the low end of Level 3, or Standard Met. CLICK
Let’s also say that the group of students received a score of 2470 in grade four in Notice that even though the scale score is higher than the score from the previous year, the score now falls into the high part of Level 2.
Using this device to consider one or two year’s worth of scale scores allows us, and our stakeholders, a more interesting way to consider whether the performance of this student or group of students shows improvement over time.
We want students to stay within the level and range (at least if met or above) or show improvement. BUT, this score all by itself isn’t enough, it’s just a heads up that needs to be corroborated with other data.
While this can be a useful tool to talk about scores, with parents and boards, a word of caution -- the intent is not to use this as a justification to focus on students “close to the edge”. Recall, in the early CST days, there was quite a bit of activity around moving students across the line – focusing on students who, if they answered one more item correctly, would move up a level.
The focus should be on improving teaching and learning – increasing achievement for all students. This isn’t about increasing scores, solely – but about using the data to inform the instructional cycle.
Turn to a team member or elbow partner and talk about the performance of this group of students.
(2 min talk, then bring back)
2432
2016
2470

17. Comparing Year to Year
2 min
Points to make:
Recall, last year, we could not compare the new CAASPP tests with the former CSTs, in fact, the legislation prohibited it. But 2016 is a new year and the second year for the Smarter Balanced assessments.
Therefore, not only will there be a natural inclination to want to compare last year to this year, but for the first time, we can actually use the scale scores to talk about improvement over time, both for individual students as well as cohorts of students.
How do we best talk about this comparison and effectively communicate that performance to parents and to the community. How do we know how much improvement is good enough? How many points represent a year’s growth? The department is currently studying models of growth to provide answers to those questions.
We have just shown you a strategy that you can apply to individual student performance or to the performance of groups of students. Essentially, we want students to stay within or move to the next level and range, but always increasing their scale scores. With a vertically scaled assessment, this would generally occur. Regardless, we want all students to improve – to move up the scale – maintain or increase their achievement level.
BUT, remember, this one data point still doesn’t give us answers to the instructional implication, but rather a “heads up.”

18. Claim Scores: Example of a Claim
Claims are broad, evidence-based statements about what students know and can do as demonstrated by their performance on the assessments. For example:
30 seconds
HANDOUT – Claim Descriptions for ELA and Math; one page
Point to make:
The next score we work with are claim scores. A claim is a statement about what students know and can do. They are the next level down from scale scores and achievement levels as we talk about scores.
The handout includes ELA which has four claims and math, which has three claims.
The English Language Arts claim #1 shown here focuses on student’s ability to read closely and analytically to comprehend literary and informational texts.
The full set of claims are found in the Content Specifications.
English Language Arts/Literacy Claim #1
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.

19. Use of Claim Scores Reported at an individual and aggregate level
Reported for all claims in ELA and mathematics
1 min
Points to make:
The claims for both individual students and groups of students, are broken down into Below Standard, Near Standard, and Above Standard.
As you look at these scores, you will want to focus on the extremes.
Both the Above Standard and Below Standard scores are more precise than the Near Standard score.
Consider review of additional local evidence of student performance related to these claims in order to learn more about the students with scores of Near Standard.

20. Assessment Targets
Assessment targets connect the content standards to evidence that will be collected from the assessment.
Targets map the Common Core State Standards (CCSS) onto assessment evidence that is required to support the content categories and claims.
Targets are used to guide the development of items and tasks that will measure the CCSS.
1 min
Points to make:
Assessment targets are new. They became available to LEAs in the early spring of in ORS.
We saw a lot about these scores this morning, and we reviewed language about the assessment targets in the content and item specifications documents.

21. Example of an Assessment Target (Grade 7 ELA Claim 1 Example)
Key Details:
Identify explicit textual evidence to support inferences made or conclusions drawn.
1 min
Points to make:
As an example, one English Language Arts assessment target focuses on the analysis of figurative and connotative meanings of words and phrases and the impact of word choice on meaning and tone.
As we learned from the logic model, Smarter Balanced developed the assessment targets by carefully examining the Common Core State Standards and identifying the specific knowledge, skills, and abilities that form the standards.

22. Use of Target Scores
Reported at an aggregate level relative to group performance
Reported for all claims in ELA, but only for Claim 1 in mathematics
Only targets with 10 or more items in the item pool for a grade are included in target reports
2 min
HANDOUT (Target FAQs)
Points to make:
The Assessment Target Reports provide information about groups of students at the more granular target level
the report is an indicator of strengths and weaknesses relative to the test performances as a whole of the group you are viewing. Unlike performance levels provided at the content area level, these strengths and weaknesses do not imply proficiency or that a particular content standard has been met. The target report shows how a group of students performed on a target compared to their overall performance on the assessment.
For example, a group of students may have performed extremely well in ELA overall, but their performance in some of the target areas may be relatively lower than the overall performance. Thus, the minus sign for a target does not imply a lack of proficiency. Instead, it simply communicates that these students’ performance on that target was statistically lower than their performance across all other targets combined. Although the students performed well on the test as a whole, an educator may want to focus instruction on these areas, particularly if other sources of evidence support that decision.
These reports may become more valuable as we have more years worth of data. In the meantime, if you find that a particular area is weak or strong across grade levels and across multiple student groups, that’s information that a PLC or district level team might want to go back and look at with additional data, in conjunction with a review of the evidence statements for that target, and a look at the guiding questions which we will get into shortly.

24. . . . about the summative assessment scores
5 min
Point to make:
Please write your answers to these questions down on the cards provided and we will collect them.
This section takes around minutes.
What’s one thing you will take away from this quick review of the scores?
What additional information do you need?

25. Tools for Analysis of the Smarter Balanced Summative Results
30 seconds
In this section we will review more resources that are important and can help us correctly interpret the summative results.

26. ELA—Example LEA
Mean Scale Scores and Corresponding Achievement Levels by Grade
ELA
Spring 2015
Grade 3
Grade 4
Grade 5
Grade 6
Grade 7
Grade 8
Grade 11
Number of Students Tested
4588
4627
4754
4720
4802
4749
4315
Percent of Students Tested 99 98
Mean Scale Score
2419.2
2457.4
2494.5
2527.4
2541.7
2555.5
2607.1
Achievement Level
Nearly Met
Met
1 min
Points to make:
This representation is a good entry point for LEA wide performance, and can be built by copying data from the public reporting site. It shows the average scale score, mean scale score and corresponding achievement level by grade for ELA in the spring of Note that the table includes both the percentage of students tested, as well as the number of students tested, in order to provide context to the audience.
We want and expect the scale score to go up as we look across the grade levels, and you can see that it does for this district. CLICK Most of the grade level scores are at Nearly Met, with the exception of the Grade 11 students who are in the Met category.
This table might be used to talk about differences across grades. You might want to add on to this the score for the Spring of 2016, which would give you both across grades and across time.

27. It’s not just about the whole: Example LEA Subgroups—ELA
ELA/Spring 2015
Levels of Performance/ Subgroup*
Black/ African American
Asian
Filipino
Hispanic/ Latino
White
Two or More Races
Students with Disabilities
SED^
English Learners
All Students
Number of Students
4,481
7,474
2,745
8,243
7,029
1,843
3,172
18,618
4,392
32,555
% Exceeded/Met
31%
58%
65%
38%
61%
50%
10%
36%
% Exceeded 8%
25%
26%
11%
18% 3% 1%
19%
% Met
23%
33%
39%
27%
32% 7% 9%
% Nearly
21%
28%
22%
% Not Met
42%
14%
34%
17%
72%
59%
% Nearly/Not Met
69%
35%
62%
90%
64%
51%
*The subgroups American Indian/Alaska Native and Pacific Islander are not released due to insufficient numbers.
^ SED: Socio-Economically Disadvantaged
2 min
Points to make:
Remember the power of looking at performance across groups of students.
Disaggregation of student performance data is important, especially if you are in a high performing district.
Here we are looking across the different ethnic groups, students in poverty, English learners, and all students. The average for All Students is 50% Exceeded/Met Standard. But, the average scale score and achievement levels for All students looks very different when we break it down into different subgroups of students.
This is about achievement and equity across groups.

28. Research, Recall, Reflect, Respond
Data review and discovery
Possible actions moving forward
3 min
HANDOUT: Data Analysis Template (with guiding questions)
Points to make:
In your packet, there are three handouts with these headers, and they are all white. Please pull out the version that has guiding questions in each of the columns.
The purpose of the template is to help us use the summative assessment results appropriately at the local level.
The next few slides describe each of these columns as we process the template and link it back to using the data in the instructional cycle.
First, we’ll do the research CLICK. We’ll write descriptive data statements and report the facts.
Next, we’ll recall the context. CLICK What happened during the year the assessment was given?
Next, CLICK we’ll use the resources that were covered in the first part of this session to bridge the conversations that consider both recalling and reflecting.
Next, CLICK we’ll reflect on how the student’s performance is connected to our practice and identify possible reasons why we got the scores we got.
Last, CLICK we’ll think about possible actions we might take as a result of our conversations and collaboration.
The questions that appear in the template you have are sample questions. They are not intended to be definitive. Depending on what your data say, and what tools you may be using, the questions you wish to ask may be different.
Possible connections
Program, practice, policies

29. Examples of Research Guiding Questions - Finding the Facts
What do you notice about the overall scores?
What are the percentages of students in each of the achievement levels?
Where do the scale scores fall, i.e., high, medium, low band?
How do the grade level scores compare with the Smarter Balanced threshold scores?
Which scores look most noticeably different from the population being studied?
2 min
HANDOUT
Points to make:
Orient participants to handout with guiding questions related to research.
Give them 2 minutes to review the questions.

30. Using the Resources to Research, Recall, Reflect, and Respond
RESEARCH for SCALE SCORES and ACHIEVEMENT LEVELS
DATA SOURCE
RESEARCH
RECALL
RESOURCES
REFLECT
RESPOND
Identify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. .
Use the information contained in the supporting documents to bridge the conversation between reflection and revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. Overall scale scores for Black or African American students in Grade 8 and Grade 11 fell into Level 2.
District wide, 5% of the Black or African American students’ scale scores fell into the Standard Exceeded level, with 15% falling into the Standard Met level. In comparison, 15% of the districts’ All Students’ scores fall into the Standard Exceeded level and 27% fell into Standard Met.
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …

31. Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE
RESEARCH
RECALL
RESOURCES
REFLECT
RESPOND
Identify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. .
Use the information contained in the supporting documents to bridge the conversation between reflection and revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
RESEARCH for CLAIM SCORES
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60% of the Grade 3 through Grade 6 students’ claim scores were at the Below Standard level.
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...

32. Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE
RESEARCH
RECALL
RESOURCES
REFLECT
RESPOND
Identify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. .
Use the information contained in the supporting documents to bridge the conversation between reflection and revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
RESEARCH for TARGET LEVELS
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in Key Details, Central Ideas, and Word Meaning.
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …

33. Recall Consider classroom program, practice and policies.
Think about last year:
What happened?
What curriculum was in place?
What professional learning occurred?
RECALL
Focusing on the prior school year(s), discuss the realities of classroom, school, district program, practice, and policies. What happened? Facts only – no conjecture.
2 min
Points to make:
After we’ve completed initial data analysis using tools such as the last two templates, we turn to the context.
We think about what was in place last year to support teaching and learning.
How did the year go?
What happened?
What curriculum were we using? Was it new? Is it complete?
What professional development did we engage in?
What other work were we engaged in, outside of the classroom?
Again, we are writing descriptive statements; no conjecture, no opinion. Just the fact’s ma’am.

34. Examples of Recall Guiding Questions – What Happened?
What curriculum did we initiate/continue last year?
What professional development opportunities were available to teachers, administrators, and support staff?
What environmental factors were at play?
What changes occurred in our population?
What particular needs did our students exhibit?
2 min
HANDOUT
Points to make:
Orient participants to handout with guiding questions related to reflection.
Give them 2 minutes to review the questions.

35. Recalling Example Recall Template—Observations Only
SCORE: Scale Score in ELA
GROUP: Grade 7 Subgroups
DATA OBSERVATIONS
CONDITIONS FROM LAST YEAR
What do you notice about the overall scale score?
The overall scale score for Grade 7 ELA is 2525.
Environment?
Curriculum?
Context?
Professional Development?
 How do these scale scores compare with the threshold scores?
The overall scale score falls into the medium band of Level 2, Standard Nearly Met.
What subgroups are struggling the most?
African American, English learners,
SED, SWD
2 min
Point to make:
This template can be used both to capture data observations (what did the data tell us) and to capture observations about our classroom programs, practices and policies.
What materials did the teachers and students have?
What PD did teachers engage in?
How was the curriculum sequenced?

36. Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE
RESEARCH
RECALL
RESOURCES
REFLECT
RESPOND
Identify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. .
Use the information contained in the supporting documents to bridge the conversation between reflection and revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials.
We had new principals at all but two of our schools.
We implemented a new student information system, district wide.
We implemented a new mathematics curriculum in Grades K through 6.
RECALL
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials.
All of our professional learning opportunities were focused on ELA.
We implemented a new student information system, district wide.
We implemented a new mathematics curriculum in Grades K through 6.
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …

37. Evidence Required for Target 1
Resources
Evidence Required for Target 1
The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text.
What should our students know or be able to do?
What are the expectations of our students, based on the range and threshold descriptors?
2 min
Point to make:
All of the resources we’ve covered, the frameworks, the Smarter documents, the example tools are all here to help you take collaborative discussion from the safe banks of Recall across the bridge to the potentially scarier banks of Reflection. CLICK
The resources column is here to help you remember the value of knowing how the test was built and the connections to the standards: the frameworks, the content specifications, the item specifications and evidence statement, and the language from the Range and Threshold ALDs.
What do the related threshold ALDs tell us about what students should know and be able to do at the Standard Met level?
Think about the expectations for student performance, as described in those documents.
Consider what types of evidence are available from local student performance measures.
Recall:
program, practice, policies
Reflect: possible connections

38. Using the Tools to Research, Recall, Reflect, and Respond
DATA SOURCE
RESEARCH
RECALL
RESOURCES
REFLECT
RESPOND
Identify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. .
Use the information contained in the supporting documents to bridge the conversation between reflection and revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
At Grade 3:
Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity.
Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to-high complexity.
Students performing at Level 3 should be able to determine intended meanings of words, including words with multiple meanings, based on context, word relationships, word structure, or use of resources in texts of moderate-to-high complexity.
RESOURCES
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials.
We had new principals at all but two of our schools.
We implemented a new student information system, district wide.
We implemented a new mathematics curriculum in Grades K through 6.
At Grade 3:
Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity.
Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to- high complexity.
Students performing at Level 3 should be able to determine…
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …

39. Reflect Use the guiding questions.
Connect performance with prior year practices, programs, and policies. Consider possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.
Use the guiding questions.
Determine possible reasons for the performance.
Keep the conversation honest.
Explore many possibilities.
2 min
Points to make:
When the conversations turn to past practice, the conversations can become uncomfortable.
It’s important to understand that these can be hard conversations.
Use the guiding questions.
This is where using good facilitation strategies and skills can really pay off. Look for those people within your organization who are good at structuring these conversations and leading them.
You might want to take advantage of opportunities that you have to use an outside facilitator.
Remember to keep the focus on the students, and on the data statements.
This is a good time to bring in our PLC norms and use them.

40. Examples of Reflecting Guiding Questions – Why and how may it have happened?
How do these results affirm areas where instruction was provided?
What would you consider is the single-most important factor contributing to the apparent successes/needs as indicated by the scores?
Consider what might explain these gaps in achievement. Think about current:
Policies
Programs
Practices
2 min
HANDOUT
Points to make:
Orient participants to handout with guiding questions related to Reveal.
Give them 2 minutes to review the questions.

41. Reflecting—Beyond the Numbers
Example Reflection Template
Teaching and Learning Inventory: Content Area: ____________________
Component
In Place
In Progress
Not Yet
How might these conditions have played a role in the observed performance?
Teachers use a curriculum aligned to the CA state standards and frameworks
Teachers are provided regular professional learning opportunities around the standards.
Teachers use ongoing formative assessment practices and tools
Teacher make use of Interim assessment(s), including SBAC IABs to measure student progress
Teachers have collaborative planning time to discuss effective and responsive teaching practices/strategies
Teachers have sufficient collaborative planning time to examine student work
2 min
Points to make:
This template provides an example of how you might want to work with a district/school group to reflect on the observed performance. Crafting statements describing possible conditions of the teaching and learning continuum can be used to promote honest reflective conversations amongst colleagues. The components listed in this example are not an exhaustive list. In addition to those statements you know would be applicable to your district or school, you may refer to the guiding questions handout to consider additional components that may be relevant to your school context.
These questions are provided to help you dig a little deeper into the context.
Give them 2 minutes to review the questions.

43. Respond Use guiding questions. Think about what you can control.
Ask yourself:
What other data are available to you?
How will you know progress is being made?
RESPOND
What might be some possible ways to move forward? These responses should tie directly to what was discovered in the reflection process.
2 min
Point to make:
Consider other evidence that surfaces facts about these students and their performance:
Teacher observation
Student work
Unit assessment results
Local benchmark results
Interim assessment results
Consider what you can control as you move forward:
Practice
Policies and procedures
Programs
Always consider the culture, climate, and context of your school/LEA
What training will be needed?
What stakeholders will need to be involved?
What communication will need to be done?
What resources will you need?
Think about how progress can be monitored and measured.

44. Examples of Responding Guiding Questions – What’s our path forward?
How might we need to alter our curriculum and/or materials to address the areas of need for the coming year?
What evidence do we need during classroom instruction to know that our students are making progress toward meeting the targets for each claim?
How might we use the performance tasks to illustrate student performance and to guide the possible intervention?
3 min
HANDOUT
Points to make:
Orient participants to handout with guiding questions related to Reveal.
Give them 2 minutes to review the questions.

45. . . . about the documents we’ve just reviewed.
5 min
Point to make:
Thinking about all of the documents we’ve covered so far (content specifications, item specifications, blueprints, achievement level descriptors) and thinking about the professional learning that happens at your LEA, when might this learning for teachers occur at your LEA?
Estimate 35 min for this section
Start thinking about how you might weave these documents into planned professional learning, instructional planning, and local assessment development in the coming school year…

46. CAASPP Resources
1 min at very end, thank them and tell them all the links are on these slides which they can download and use.