Measuring the Power of Learning.™ Connecting Summative Assessment to Improving Teaching and Learning May and June 2016 California

Measuring the Power of Learning.™ 2 Workshop Goals Participants will learn: the rightful place and purpose of the summative assessments to use results to inform the instructional process by connecting curriculum, instruction and assessment to access supplementary resources and reference materials to assist local training to use tools to reflect on current practice to improve teaching and learning a variety of ways to display data to communicate a particular message

Measuring the Power of Learning.™ 3 The Many Faces of Testing

Measuring the Power of Learning.™ 4 Overview of Major Assessment Types

Measuring the Power of Learning.™ 5 Classroom Formative Statewide Summative Rightful Place/Purpose: Assessment Frequency and Impact on Instruction

Measuring the Power of Learning.™ 6 Standards set expectations on path to college- and career- readiness Standards set expectations on path to college- and career- readiness All students graduate college- and career- ready Teachers and schools have information and tools to improve teaching and learning Interim assessments Flexible, open, and used for actionable feedback Summative assessments benchmarked to college and career readiness (Grades 3–8 and Grade 11) Summative assessments benchmarked to college and career readiness (Grades 3–8 and Grade 11) Digital Library Formative assessment tools and practices for teachers to improve instruction Digital Library Formative assessment tools and practices for teachers to improve instruction A Balanced Assessment System

Measuring the Power of Learning.™ 7 Statewide summative assessments are like icebergs — it pays to pay attention

Measuring the Power of Learning.™ 8 Rightful Place, Rightful Purpose for Statewide Summative Assessment Provide a general direction—we must dig deeper to determine cause –Focus on groups, programs, and disaggregation Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice Provide an entry point into a collaborative, honest conversation Provide a necessary story

Measuring the Power of Learning.™ 9 Using Summative Results Comprehensive, complex, and difficult process –Not a checklist –Not meant to be completed at a single staff meeting or professional development event Focus on improving learning –Not solely about increasing scores Collaborative process that requires: –Honesty –Willingness to commit the time –Ability to handle ambiguity –Patience

Measuring the Power of Learning.™ 10 Data Examination Is Complicated

Measuring the Power of Learning.™ about the rightful place/purpose of summative assessment results How do you currently use statewide summative assessment results in the curriculum, instruction, and assessment planning process within your LEA?  What is the place and purpose of the summative assessment in your LEA right now?

Measuring the Power of Learning.™ ITEMS Evidence Statements Content Claims Targets The Smarter Balanced Hierarchy of Item Development and Reporting of Scores Overall Claims Evidence-Centered Design

Measuring the Power of Learning.™ Define the domain 2. Define claims to be made 3. Define assessment targets 4. Define evidence required 5. Develop items or performance tasks Concepts of Evidence-Centered Design

Measuring the Power of Learning.™ 14 English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework English Language Arts/Literacy Content Specifications (Claims) Relationships Between Common Core Standards, California Frameworks, and Assessments

Measuring the Power of Learning.™ 15 ELA/Literacy Content Specifications (Grade 7 ELA Example)

Measuring the Power of Learning.™ 16 English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework English Language Arts/Literacy Content Specifications (Claims) English Language Arts/Literacy Item Specifications (Claims and targets per claim) Relationships Between Common Core Standards, California Frameworks, and Assessments

Measuring the Power of Learning.™ 17 Example of Item Specifications (Abbreviated) Evidence Required for Target 1 The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text.

Measuring the Power of Learning.™ 18 English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework English Language Arts/Literacy Content Specifications (Claims) English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts/Literacy Assessment Blueprint Relationships Between Common Core Standards, California Frameworks, and Assessments

Measuring the Power of Learning.™ 19 Sample ELA/Literacy Summative Assessment Blueprint

Measuring the Power of Learning.™ 20 English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects English Language Arts/English Language Development Framework English Language Arts/Literacy Content Specifications (Claims) English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts/Literacy Assessment Blueprint English Language Arts/Literacy Summative Assessment Mathematics Mathematics Framework for California Public Schools Mathematics Content Specifications (Claims) Mathematics Item Specifications (Claims and targets per claim) Mathematics Assessment Blueprint Mathematics Summative Assessment Relationships Between Common Core Standards, California Frameworks, and Assessments

Measuring the Power of Learning.™ 21 Student Response Evidence Assessment Target Claim Common Core State Standards Assessmen t Claim Assessmen t Target Evidence Student Response Building a Logical Argument

Measuring the Power of Learning.™ about the test design and the connections to the instructional cycle To what extent do the teachers at your school or LEA understand how the test is constructed and how it is connected to the standards and the frameworks? How does this connection have implications for instructional planning?

Measuring the Power of Learning.™ Review of Available Summative Scores

Measuring the Power of Learning.™ 24 Scale scores Achievement levels –not met, nearly met, met, exceeded Claim levels –below, near, above Target scores –worse, similar, better –insufficient information Available Summative Assessment Scores

Measuring the Power of Learning.™ 25 Scale Score – Achievement Level Relationship – Grade 3 Example Scale Score RangeSmarter Level CA Achievement Level ≤ 2366 Level1 Standard Not Met 2367 ― 2431 Level2 Standard Nearly Met 2432 ― 2489 Level3 Standard Met ≥ 2490 Level4 Standard Exceeded

Measuring the Power of Learning.™ 26 Smarter Balanced ELA Threshold Scale Scores GradeLevel 2Level 3Level 4 Grade Grade Grade Grade Grade Grade Grade

Measuring the Power of Learning.™ 27 Low, Medium and High Bands 2367 to to to to to to to to to to to to 2532 (Divide the difference by three = 21 pts)

Measuring the Power of Learning.™ 28 Low, Medium and High Bands (cont.)

Measuring the Power of Learning.™ 29 Comparing Year to Year

Measuring the Power of Learning.™ 30 English Language Arts/Literacy Claim #1 Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Claims are broad, evidence-based statements about what students know and can do as demonstrated by their performance on the assessments. For example: Claim Scores: Example of a Claim

Measuring the Power of Learning.™ 31 Reported at an individual and aggregate level Reported for all claims in ELA and mathematics Use of Claim Scores

Measuring the Power of Learning.™ 32 Assessment targets connect the content standards to evidence that will be collected from the assessment. Targets map the Common Core State Standards (CCSS) onto assessment evidence that is required to support the content categories and claims. Targets are used to guide the development of items and tasks that will measure the CCSS. Assessment Targets

Measuring the Power of Learning.™ 33 Target 1 Key Details: Identify explicit textual evidence to support inferences made or conclusions drawn. Example of an Assessment Target (Grade 7 ELA Claim 1 Example)

Measuring the Power of Learning.™ 34 Reported at an aggregate level relative to group performance Reported for all claims in ELA, but only for Claim 1 in mathematics Only targets with 10 or more items in the item pool for a grade are included in target reports Use of Target Scores

Measuring the Power of Learning.™ 35 What’s one thing you will take away from this quick review of the scores? What additional information do you need?... about the summative assessment scores

Measuring the Power of Learning.™ Tools for Analysis of the Smarter Balanced Summative Results

Measuring the Power of Learning.™ 37 Content Specifications Item Specifications Blueprints Achievement Level Descriptors (ALDs) −Policy, range, threshold, and reporting Claim Descriptors Score Reports Additional Data Supporting Documentation

Measuring the Power of Learning.™ 38 Text descriptions of the knowledge, skills, and processes demonstrated by students at each level. Four types of levels or categories of performance –Policy and Content –Range –Threshold –Reporting Achievement Level Descriptors

Measuring the Power of Learning.™ 39 Base for the ALDs (page xvi)

Measuring the Power of Learning.™ 40 Range ALDs Composed of grade- and content-specific descriptors Describe the cognitive and content rigor that is encompassed within particular achievement levels Describe the knowledge, skills, and processes expected of students Used by test developers to guide item writing

Measuring the Power of Learning.™ 41 Range ALDs (Grade 7 ELA Claim 1 Example) Level 1 Level 2 Level 3 Level 4

Measuring the Power of Learning.™ 42 Range ALDs (cont.) (Grades 6-8 Math Claim 2, 4 Example) Level 1 Level 2 Level 3 Level 4

Measuring the Power of Learning.™ 43 Relationship Between Range and Threshold Descriptors Adapted from Figure 1 in the Initial Achievement Levels Descriptors document. Students performing at Level 1 should be able to identify textual evidence that minimally supports a basic idea drawn about texts of low complexity. The student who just enters Level 2 should be able to… The student who just enters Level 3 should be able to… The student who just enters Level 4 should be able to… Students performing at Level 2 should be able to cite relevant textual evidence to support a simple inference, analysis, interpretation, or conclusion … Students performing at Level 3 should be able to cite sufficient and relevant textual evidence that adequately supports a complex inference… Students performing at Level 4 should be able to cite strong and thorough textual evidence to support a complex inference, analysis… Level 1 Level 2Level 3Level 4

Measuring the Power of Learning.™ 44 Are used to guide standard setting Define the minimum performance required for meeting a particular achievement-level expectation Reflect the knowledge, skills, and processes that are expected of students Note: The knowledge, skills, and processes in ALDs are cumulative. The student who has achieved the threshold Level 3 is assumed to have the knowledge, skills, and processes of the range Levels 1 and Level 2 ALDs. Threshold ALDs

Measuring the Power of Learning.™ 45 Threshold ALDs (Grade 7 ELA Claim 1 Example) Level 1 Level 2 Level 3 Level 4

Measuring the Power of Learning.™ 46 Threshold ALDs (Grade 6-8 Math Claim 2, 4 Example) Level 1 Level 2 Level 3 Level 4

Measuring the Power of Learning.™ 47 Start thinking about how you might weave these documents into planned professional learning, instructional planning, and local assessment development in the coming school year…... about the documents we’ve just reviewed.

Measuring the Power of Learning.™ Putting It All Together

Measuring the Power of Learning.™ 49 Using Summative Assessment Results in the Instructional Cycle Improving Teaching and Learning Data review and discovery What is the current state of performance? Recalling program, practice, policies, etc. What happened? Possible connections between performance and program, practice, policies, etc. How or why did the performance occur? Possible actions moving forward What should/will change?

Measuring the Power of Learning.™ 50 Research, Recall, Reflect, Respond Data review and discovery Program, practice, policies Possible connections Possible actions moving forward

Measuring the Power of Learning.™ 51 Sources of Data: Identifying the Appropriate Scores and Reports Consider the reports covered in the a.m. session. Consider the “unit of analysis” for your data research. DATA SOURCE Identify and use appropriate data sources.

Measuring the Power of Learning.™ 52 Research Engage with the data to find facts. Describe what you observe. Look for data trends. Focus on facts, not conjecture. RESEARCH Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.

Measuring the Power of Learning.™ 53 Examples of Research Guiding Questions - Finding the Facts 1.What do you notice about the overall scores? 2.What are the percentages of students in each of the achievement levels? 3.Where do the scale scores fall, i.e., high, medium, low band? 4.How do the grade level scores compare with the Smarter Balanced threshold scores? 5.Which scores look most noticeably different from the population being studied?

Measuring the Power of Learning.™ 54 Research (cont.) Example: Comparative Template Using Threshold Scores and Claim Scores Spring 2015 Smarter Balanced Scale Score Achievement Level Threshold Score Or Range Below–Above Expected Strongest Claim Area Weakest Claim Area Grade or GroupContent Area 5 Mathematics (Medium) 2455 (2455–2581) 8 Mathematics (Medium) 2504 (2504–2585) 11 Mathematics (Medium) 2543 (2543–2627)

Measuring the Power of Learning.™ 55 Average Scale Score and Performance Level of Overall Score Percentage of Students in Each Claim Performance Category % Below Standard % At/Near Standard % Above Standard GradeGroup ELA/Literacy Average Scale Score Performance Level of Overall ScoreClaims Percentage in Each Claim Performance Category 7All Reading Writing Listening Research/Inquiry 7EL Reading Writing Listening Research/Inquiry 7Hispanic Reading Writing Listening Research/Inquiry Research (cont.) Example: Comparative Template Using Claim Scores

Measuring the Power of Learning.™ 56 Recall Consider classroom program, practice and policies. Think about last year: –What happened? –What curriculum was in place? –What professional learning occurred? RECALL Focusing on the prior school year(s), discuss the realities of classroom, school, district program, practice, and policies. What happened? Facts only – no conjecture.

Measuring the Power of Learning.™ 57 Examples of Recall Guiding Questions – What Happened? 1.What curriculum did we initiate/continue last year? 2.What professional development opportunities were available to teachers, administrators, and support staff? 3.What environmental factors were at play? 4.What changes occurred in our population? 5.What particular needs did our students exhibit?

Measuring the Power of Learning.™ 58 Recalling Example Recall Template—Observations Only SCORE: Scale Score in ELA GROUP: Grade 7 Subgroup s DATA OBSERVATIONS CONDITIONS FROM LAST YEAR What do you notice about the overall scale score? The overall scale score for Grade 7 ELA is Environment? Curriculum? Context? Professional Development? How do these scale scores compare with the threshold scores? The overall scale score falls into the medium band of Level 2, Standard Nearly Met. What subgroups are struggling the most? African American, English learners, SED, SWD

Measuring the Power of Learning.™ 59 Resources What should our students know or be able to do? What are the expectations of our students, based on the range and threshold descriptors? Evidence Required for Target 1 The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text. Recall: program, practice, policies Reflect: possible connections

Measuring the Power of Learning.™ 60 Reflect Use the guiding questions. Determine possible reasons for the performance. Keep the conversation honest. Explore many possibilities. REFLECT Connect performance with prior year practices, programs, and policies. Consider possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.

Measuring the Power of Learning.™ 61 Examples of Reflecting Guiding Questions – Why and how may it have happened? 1.How do these results affirm areas where instruction was provided? 2.What would you consider is the single-most important factor contributing to the apparent successes/needs as indicated by the scores? 3.Consider what might explain these gaps in achievement. Think about current:  Policies  Programs  Practices

Measuring the Power of Learning.™ 62 Reflecting—Beyond the Numbers Example Reflection Template Teaching and Learning Inventory: Content Area: ____________________ Component In Place In Progress Not Yet How might these conditions have played a role in the observed performance? Teachers use a curriculum aligned to the CA state standards and frameworks Teachers are provided regular professional learning opportunities around the standards. Teachers use ongoing formative assessment practices and tools Teacher make use of Interim assessment(s), including SBAC IABs to measure student progress Teachers have collaborative planning time to discuss effective and responsive teaching practices/strategies Teachers have sufficient collaborative planning time to examine student work

Measuring the Power of Learning.™ 63 Respond Use guiding questions. Think about what you can control. Ask yourself: –What other data are available to you? –How will you know progress is being made? RESPOND What might be some possible ways to move forward? These responses should tie directly to what was discovered in the reflection process.

Measuring the Power of Learning.™ 64 Examples of Responding Guiding Questions – What’s our path forward? 1.How might we need to alter our curriculum and/or materials to address the areas of need for the coming year? 2.What evidence do we need during classroom instruction to know that our students are making progress toward meeting the targets for each claim? 3.How might we use the performance tasks to illustrate student performance and to guide the possible intervention?

Measuring the Power of Learning.™ 65 Using the Resources to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels

Measuring the Power of Learning.™ 66 Using the Resources to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. … The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. Overall scale scores for Black or African American students in Grade 8 and Grade 11 fell into Level 2. District wide, 5% of the Black or African American students’ scale scores fell into the Standard Exceeded level, with 15% falling into the Standard Met level. In comparison, 15% of the districts’ All Students’ scores fall into the Standard Exceeded level and 27% fell into Standard Met.

Measuring the Power of Learning.™ 67 Using the Resources to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60% of the Grade 3 through Grade 6 students’ claim scores were at the Below Standard level. Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%... The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …

Measuring the Power of Learning.™ 68 Using the Resources to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in Key Details, Central Ideas, and Word Meaning. Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in … Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%... The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …

Measuring the Power of Learning.™ 69 Using the Resources to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in … Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%... The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. … Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials. We had new principals at all but two of our schools. We implemented a new student information system, district wide. We implemented a new mathematics curriculum in Grades K through 6. Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials. All of our professional learning opportunities were focused on ELA. We implemented a new student information system, district wide. We implemented a new mathematics curriculum in Grades K through 6.

Measuring the Power of Learning.™ 70 Using the Tools to Research, Recall, Reflect, and Respond DATA SOURCE RESEARCHRECALLRESOURCESREFLECTRESPOND Identify and use appropriate data sources. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.. Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery. Use the information contained in the supporting documents to bridge the conversation between reflection and revealing. Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe.. What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause. Scale Scores Achievement Levels Claim Scores Target Levels Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in … Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%... The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. … Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials. We had new principals at all but two of our schools. We implemented a new student information system, district wide. We implemented a new mathematics curriculum in Grades K through 6. At Grade 3: Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity. Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to- high complexity. Students performing at Level 3 should be able to determine… At Grade 3: Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity. Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to-high complexity. Students performing at Level 3 should be able to determine intended meanings of words, including words with multiple meanings, based on context, word relationships, word structure, or use of resources in texts of moderate-to-high complexity.

Measuring the Power of Learning.™ using summative assessment results Choose either Scenario #1 ELA, or Scenario #2 mathematics. Treat the sample data as if it were your LEA’s data. Use the Guiding Questions and work your way through the template, answering the questions as best you can. PracticePractice

Measuring the Power of Learning.™ Telling the Story with the Data

Measuring the Power of Learning.™ 73 Different Scores Tell Different Stories What is the message you want to send? Who is your audience? What do you want your audience to remember? Manage your message through –Emphasizing –Spotlighting –Focusing  Context  Display  Level of detail

Measuring the Power of Learning.™ 74 ELA Spring 2015 Grade 3Grade 4Grade 5Grade 6Grade 7Grade 8Grade 11 Number of Students Tested Percent of Students Tested Mean Scale Score Achievement Level Nearly Met Met ELA—Example LEA Mean Scale Scores and Corresponding Achievement Levels by Grade

Measuring the Power of Learning.™ 75 Comparing ELA Scale Scores Year to Year: Grade ● 2419 Standard Exceeded Standard Met Standard Nearly Met Standard Not Met ● 2457 ● 2494 ● 2527 ● 2541 ● 2607 Spring 2015 ● ● 2433 ● 2475 ● 2505 ● 2535 ● 2558 ● 2620 Spring 2016 ● ● 2555 ● Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade Threshold Scale Score Range

Measuring the Power of Learning.™ 76 Comparing ELA Scale Scores Year to Year: Cohort X 2394 Standard Exceeded Standard Met Standard Nearly Met Standard Not Met X 2437 X 2480 X 2481 X 2526 O 2550 Spring 2015 X O 2465 O 2490 O 2525 O 2501 Spring 2016 O 2800 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade Threshold Scale Score Range

Measuring the Power of Learning.™ 77 Take Advantage of the Local Control and Accountability Plan (LCAP) Process Examine the scores in the context of your LEA’s strategic plan Examine the scores in the context of the purpose of the Local Control Funding Formula and the LCAP –Eight state priorities –Focus on underperforming groups of students –Consider opportunity gap as well as achievement gap

Measuring the Power of Learning.™ 78 Eight Areas of State Priority Must Be Addressed in LEA LCAP State Priorities Other Pupil Outcomes (8) Course Access (7) School Climate (6) Basic Services (1) Pupil Engagement (5) Implementation of CA Common Core State Standards (2) Parental Involvement (3) Pupil Achievement (4)

Measuring the Power of Learning.™ 79 ELA/Spring 2015 Levels of Performance/ Subgroup* Black/ African AmericanAsianFilipino Hispanic/ LatinoWhite Two or More Races Students with DisabilitiesSED^ English Learners All Students Number of Students4,4817,4742,7458,2437,0291,8433,17218,6184,39232,555 % Exceeded/Met31%58%65%38%61%50%10%36%10%50% % Exceeded8%25%26%11%25%18%3%10%1%19% % Met23%33%39%27%36%32%7%26%9%31% % Nearly27%23%21%28%22%25%18%28%31%25% % Not Met42%19%14%34%17%25%72%36%59%26% % Nearly/Not Met69%42%35%62%39%50%90%64%90%51% *The subgroups American Indian/Alaska Native and Pacific Islander are not released due to insufficient numbers. ^ SED: Socio-Economically Disadvantaged It’s not just about the whole: Example LEA Subgroups — ELA

Measuring the Power of Learning.™ 80 California Achievement Disparities: ELA Subgroups

Measuring the Power of Learning.™ 81 Comparable LEAs Comparable LEAs/schools based on demographic similarities ─ Enrollment ─ Poverty ─ Ethnicity ─ Programs (e.g., students with disabilities) ─ Language fluency Geographic proximity ─ Within LEA ─ Surrounding LEAs ─ Both charters and non-charters

Measuring the Power of Learning.™ 82 Example LEA Compared to Similar LEAs and State — ELA ELA Spring 2015 Levels of Performance/LEA Example LEALEA 2LEA 3LEA 4State Number of Students Tested 32,555 25,32722,60928,6343,154,463 % Exceeded/Met 50% 52%42%53%44% % Exceeded 19% 23%15%19%16% % Met 31% 29%27%34%28% % Nearly Met 25% 21%25%26%25% % Not Met 26% 33%21%31% % Nearly/Not Met 51% 47%58%47%56%

Measuring the Power of Learning.™ 83 Similar LEA Subgroup Comparisons: Students Exceeding Standard

Measuring the Power of Learning.™ 84 Similar LEA Subgroup Comparisons: Students Not Meeting Standard

Measuring the Power of Learning.™ 85 Which data displays would be useful for your: Board Community Parents Administrators Teachers... about the data displays we’ve just reviewed.

Measuring the Power of Learning.™ Applying the Learning

Measuring the Power of Learning.™ 87 Call to Action – Pay Attention but Move Beyond the Scores Comprehensive, complex, and difficult process –Not a checklist –Not meant to be completed at a single staff meeting or a single professional development event Focus on improving learning –Not solely about increasing scores –Honest conversations about what might need to change

Measuring the Power of Learning.™ 88 Call to Action – Pay Attention but Move Beyond the Scores (cont.) Use the data to communicate—as local policy and practice levers Reflect on what you can control to move beyond the scores: ─ Practices ─ Policies and Procedures ─ Programs

Measuring the Power of Learning.™ 89 Summative Assessments Have a Part to Play How do you promote the effective use of summative assessment data in your LEA? Understand the distinction between –having educators make meaning of the scores to inform instruction and practice, –and communicating a message to a particular audience.

Measuring the Power of Learning.™ 90 Process Document

Measuring the Power of Learning.™ 91 Resources Handout: Overview of Major Assessment Types English Language Arts/English Language Development Frameworks Content Specifications, Item Specifications and Blueprints Smarter Balanced Scale Score Ranges Sample Student Score Report Claim Descriptions for ELA and Mathematics Target Score Report FAQs

Measuring the Power of Learning.™ 92 Resources (cont.) Achievement Level Descriptors California Department of Education California Assessment of Student Performance and Progress Office ETS CAASPP Portal Help for LEA CAASPP Coordinators: –