State Accountability System CDE Dashboard Overview Good morning everyone! Today you will be provided with a broad overview of the state's new accountability system. This is only the first in a series of future presentations that will take us deeper into the specifics of the new accountability systemSo lets begin by reviewing our outcomes. Local District Northeast Title 1 Coordinators Meeting May 25, 2017

A Decade of Accountability Annual Yearly Progress (AYP) Federal: 18 to 22 criteria including proficiency rates, participation rates, graduation rate, API …. Program Improvement (PI) Academic Performance Index (API) State- discontinued March 2014 Alternative Schools Accountability Model (ASAM) Academic Growth Over Time (AGT) – Deasy School Accountability Report Card (SARC) School Quality Improvement Index (SQII) Data and Accountability Presenter’s Notes: Here are our objectives for today’s session (review slide contents) Link to Strategic Plan As we present, think about how this new system is similar to, or different from AYP and API- our former system.

LCFF LCAP Local Control and Accountability Plan 100% Graduation (grad. rate, A-G Rate, AP exam pass with a ‘3’ or better rate, A-G awareness) 100% Proficiency for All (Subgroup Rates) (grad. rate, A-G courses pass rate, EAP rate of exceeding college readiness standard, CST science rate of proficient or advanced, Progress on CELDT, Reclassification rate, LTEL rate, SRI basic or higher rate) 100% Attendance Staff & student 96% + proficiency rate (<8 days) , Chronic absence rate (>15 Days) Parent, Student, Community Engagement School Experience Survey participation and responses) School Safety Suspension rate (out-of-school), Expulsion rate, SES campus safety survey perception data Data and Accountability Presenter’s Notes: Here are our objectives for today’s session (review slide contents) Link to Strategic Plan As we present, think about how this new system is similar to, or different from AYP and API- our former system.

Adequate Yearly Progress (Federal) Measured Rates of Proficiency based on Annual Results Adequate Yearly Progress was the measure for federal accountability and this had consequences for being identified as Program Improvement. When these reports were released, you would look to see if you made AYP. Then you would scan for subgroups to see which subgroups didn’t meet targets.

The New State Accountability System States establish accountability indicators. States produce California Dashboard to share results for LEAs and Schools (no stakes). States establish criteria to identify LEAs and schools for intervention and assistance. State identifies LEAs and (eventually) schools for support and intervention. State provides technical assistance to identified LEAs. LEAs support and ensure that identified schools implement required processes and interventions based on identification. METRICS LEA/SCHOOL DESIGNATIONS INTERVENTION AND SUPPORT Presenter’s Notes: Federal and State accountability under one system. No Child Left Behind replaced by ESSA. No more Program Improvement. In 2013, state legislation resulted in the new state accountability. In 2015, Every Student Succeeds Act replaced No Child Left Behind, and required states to develop a new multiple-measure system of accountability. California created one coherent system that meets federal requirements. Our new system is likely to be the direction for state and federal accountability for the foreseeable future. In the past, federal accountability (AYP/Program Improvement) and state accountability (API) were in many ways separate accountability systems This slide shows us the key processes embedded in the new accountability system to show both an overview and a timeline– it is important to note that this system will be rolled out over time. There will be no stakes this school year as the state rolls out a new data dashboard The new accountability system uses multiple metrics to measure school performance. Based on the results, LEAs and Schools will be identified as needing additional support, and those LEAs and schools that are identified will be responsible for carrying out certain requirements with additional support. This system is UNDER DEVELOPMENT, and the State has only recently adopted the first step of building the accountability system: the indicators. California is releasing school and district level reports in the coming weeks through the California School Dashboard The next phase of the state’s accountability system design is to determine how the indicators will be used and what criteria will determine which districts and schools are identified for intervention. the new federal law will target fewer schools for intervention than we had under Program Improvement. The State is scheduled to identify criteria in March 201. This coming fall, LEAs (including potentially affiliated charters, which are considered LEAs under LCFF) will be identified for intervention, but schools will not be identified until the beginning of the 2018-19 school year. The final aspect of the accountability system is that the identified schools and LEAs would implement required interventions with technical assistance. We also expect to learn more about what that will look like when the state finalizes the plan this coming September. 2016-17 Criteria: by 9/2017 LEAs Identified: 2017-18 Schools Identified: 2018-19 2018-19

Moving Forward Under the New System Things to consider – New metrics and benchmarks Different growth model Comprehensive Subgroup representation Transparent & public Sourced from CALPADs Presenter’s Notes: Moving forward under the new accountability system, we need to consider the following: 1- Systems we will put in place to communicate 2- A different growth model than with our previous accountability system 3- Comprehensive report through multiple indicators 4- Subgroup representation is different under the new accountability system

Where Does the Data Come From? Majority is from CALPADS – sourced from MiSiS Part of data clean-up efforts in Certify Data and Accountability Presenter’s Notes: All state indicators are based on CALPADs, so the accuracy depends MiSIS’s accuracy; school staff play a role in ensuring accuracy. Our Certify system allows principals and data designees to get alerts so they can correct data errors. Data errors that affect CALPADS reporting are coded as critical, in red.

English Learner Progress Graduation Rate College & Career ELA (SBAC) State Indicators: Chronic Absenteeism English Learner Progress Graduation Rate College & Career ELA (SBAC) Mathematics (SBAC) Data and Accountability

State Indicators (School and LEA) Chronic absenteeism – starting 2018-19, data being collected in 2016-17 Suspension rate – in-school and out-of-school suspensions English Learner progress– Annual progress on CELDT + reclassified EL’s in prior year Graduation rate – Four-year cohort graduation rate, excludes options schools, five-year rate considered for 18-19 College & career indicator –for graduates in 4-yr. cohort, Gr. 11 SBA scores, AP & IB tests, A-G completion with “C” or above, dual enrollment, career pathway completion with “C” or above Academic indicator – Grade 3-8 scores on standardized tests, ELA and Math Data and Accountability Rearranged the sequence to match the next slide Reversed order of slides 7 and 8 Presenter’s Notes: These are all the state indicators, which apply at the school and LEA levels. These are calculated by the state. 1) Chronic Absenteeism: 2) Suspension Rate: this will include both in-school and out-of-school suspensions. As a district, we currently track out-of-school suspensions. 3) English Learner progress will be based on the number of English learners that make annual progress on CELDT and the number of EL’s that reclassified in the previous year. This indicator will be adjusted as we roll out ELPAC. 4) Graduation Rate is a 4 yr. cohort graduation rate. The state is considering a 5 yr. grad rate for future years. It’s also worth noting that options schools will not have graduation rates, and an alternative accountability system will be developed for those schools. 5.) College and Career Indicator: This is for high schools, and it is looking at the graduates in the four-year cohort. Of those graduates, the following indicators are included: gr. 11 Smarter Balanced results, AP/IB test results, A-G completion with C or above, dual enrollment, and career pathway completion (which is the completion of a capstone course). If you look at the back of your packet, you’ll see a handout that describes how they determine if a student is “prepared” or “well-prepared”, based on a combination of the various indicators. 6) The Academic Indicator includes grades 3-8 Smarter Balanced results in ELA and Math. We will spend more time on this later in the presentation, but it is no longer only based on % met or exceeded. The academic indicator will eventually include science test results when we are in full implementation. Notice that Grade 11 is not included because it is embedded into the College and Career Indicator. 7) OVERALL: Most of these indicators are familiar and we have been tracking them over the last number of years.

California Dashboard, Spring 2017 Data Data and Accountability Presenter’s Notes: This slide is really for your reference so you know which years of data are in the reports The Fall 2017 release will be updated with 16-17 data.

English Learner Progress Graduation Rate College & Career ELA (SBAC) State Indicators: Chronic Absenteeism English Learner Progress Graduation Rate College & Career ELA (SBAC) Mathematics (SBAC) Data and Accountability

Academic ~ ELA and Math State Indicator: MAIN IDEA: How far above or below the minimum scale score for standards met are your students on average? INTENT: Shows needed improvement to bring the average student to meeting standard as measured by Smarter Balance Assessments KEY SHIFTS: Based on scale scores for all students and subgroups Establishes “level 3” score for each grade level as a comparison point (lowest score needed to meet standard) Averages the “distance from Level 3” to determine performance Every student will have a positive, negative, or neutral contribution to the school-wide, LEA-wide, or student group average. Data and Accountability Presenter’s Notes: A big change in the indicators is how the academic indicator is reported. We are no longer simply considering the % of students who met or exceeded standards, and instead we are looking at each student’s scale score and how far they are from the minimum scale score for “met standards.” So the main idea is “how far above or below the minimum scale score for standards met are your students on average”? This is determined based on scale scores for all students and subgroups. It establishes “level 3” based on the minimum score at each grade level that is needed to meet standard– student scores are compared to level 3 5) Then, it averages the distance from level 3 of all of the students (or students within a subgroup).

Grade Minimum Scale Score Maximum Scale Score Achievement Level 1 Scale Score Range for Standard Not Met Achievement Level 2 Scale Score Range for Standard Nearly Met Achievement Level 3 Scale Score Range for Standard Met Achievement Level 4 Scale Score Range for Standard Exceeded 3 2114 2623 2114–2366 2367–2431 2432–2489 2490–2623 4 2131 2663 2131–2415 2416–2472 2473–2532 2533–2663 5 2201 2701 2201–2441 2442–2501 2502–2581 2582–2701 LEVEL 3: The minimum score at each grade level that is needed to meet standard– on the SBAC in math and ELA

Sample: ELA Grade 3 ‘Level 3’ Subgroup A Subgroup B +193 +148 +118 +68 +58 Level 3 (2432) +18 -28 Data and Accountability This visual example shows the calculation The green line shows the level 3 minimum scale score i.e., the minimum scale score for “met standard” at grade 3 in ELA (2432). Each dot on the graph is a student and the number next to the dot is the “distance from level 3” so you’ll notice some are above the line and some are below. Scores above the line are positive, scores below the line are negative. Then, you average those numbers to produce an overall score, or an average distance from level 3. This can be calculated for subgroups as well. [click for animated] a subgroup’s score. For ELA 3rd Grade, Level 3 equals a scale score of 2432 Each student’s distance from Level 3 is averaged Overall (+27.75) Subgroup A (+54.75)

Distance From Level 3 All scale score ranges are set by the Smarter Balanced Assessment Consortium and won’t change All students in the same grade across California are measured against the same scale score All grade 6 students across California are measured against the 2531 scale score for ELA. All grade 8 students across California are measured against the 2586 scale score for Math. Data and Accountability

English Learner Progress Graduation Rate College & Career ELA (SBAC) State Indicators: Chronic Absenteeism English Learner Progress Graduation Rate College & Career ELA (SBAC) Mathematics (SBAC) Data and Accountability

Metrics for College and Career Indicator Data and Accountability This handout shows the data that comprises the College and Career Readiness Indicator It is looking at the graduates in 4-yr. cohort and bases college/career preparedness using: Gr. 11 SBA scores, AP & IB tests, A-G completion with “C” or above, dual enrollment, career pathway completion with “C” or above

– Beginning of status and change discussion – what do performance categories mean?

Performance Levels for State Indicators Performance Levels are calculated using percentiles that combine Status and Change using a five- by-five colored table that produced 25 results represented by five colors. Blue Highest Green Yellow Orange Red Lowest Presenter’s Notes: Up to this point we have been talking about all the indicators or metrics that are included in the system. Now we will talk about how the indicators will be displayed on the new dashboard. When it comes to reports and results, all indicators will be regarded in terms of both STATUS (the result for that year) and CHANGE (the result in comparison to previous year(s)) Based on the status and change, schools will be given a performance level that ranges from Blue (high) to Red (low) Each performance level was derived based on a combination of status and change. You’ll notice that each level has wedges and might be wondering why it’s presented that way. This is so that a color blind person could read it, or a black and white copy could still communicate the results. Let’s look at an example.

Five Status and Five Change Levels Five Status Levels Five Change Levels Very High Increased Significantly High Increased Medium Maintained Low Declined Very Low Declined Significantly Blue Green Yellow Orange Red

Status and Change Sample (3 yr. average) 2015 Grad Rate: 89% 3 yr. average rate: 91% Difference between 2015 and 3 yr. average: -2 N/A Presenter’s Notes: 1) This 5 by 5 matrix is how the performance level is determined. 2) For this example, the graduation rate is 89%. If you look at the status column, you will see that the school is considered “Median” given that 89% falls between 85%-90%. (click for animation) 3) The 3 yr. average graduation rate was 91%, which is 2 percentage points higher than the 2015 result. That means that the school is considered “Declined” (click for animation) 4) What is this school’s graduation performance level or color? Orange: median and declined. (click for animation) 5) This is a big difference from the past accountability systems because it takes more than one year into account. 6) The technical manual and forthcoming tables on the state website will provide cut off points for each of the performance levels. Cut off points vary for each indicator. Gray = NA

Reported Subgroups Each state indicator will be reported for all schools and for the following subgroups (n≥30): Race/ethnicity African-American Asian Filipino Hispanic/Latino Native American Pacific Islander Two or More Races White Socioeconomically disadvantaged Students with disabilities English learners – definition varies by indicator Homeless and Foster will be added Fall 2017 Data and Accountability Presenter’s Notes: Each indicator will be reported on the state dashboard overall (all students) and by subgroup. This slide shows the reported subgroups, all of which were reported under No Child Left Behind. The dashboard will report any subgroup with more than 11 students, but there will be accountability for a subgroup for 30 or more students. This is different from API and AYP, when n-size was 50 students. A potential area of confusion is the English Learners subgroup. The students that are included in this subgroup will vary depending on the indicator.

Suspension Report by Subgroup Data and Accountability Presenter’s Notes: lastly, if you click on one of the indicators, you will get further details on the performance levels by subgroup. So in this example, while our all student group is blue (very low and declined) our students with disabilities are considered green (low and declined). You can see variation by subgroup to highlight disparities. You will also be able to click on a subgroup to see the performance levels across indicators

English Learner Progress Graduation Rate College & Career ELA (SBAC) State Indicators: Chronic Absenteeism English Learner Progress Graduation Rate College & Career ELA (SBAC) Mathematics (SBAC) Data and Accountability

E L P I The English Learner Progress Indicator measures the percent of EL students who are making progress toward language proficiency from one year to the next on the California English Language Development Test (CELDT) and the number of ELs who were reclassified from EL to fluent English proficient in the prior year. -The ELPI applies to LEAs and schools that have 30 or more annual CELDT test takers. Only student records that have both the current and prior CELDT results are included. The CELDT has five performance levels: –Beginning –Early Intermediate –Intermediate –Early Advanced –Advanced Because the CELDT Intermediate performance level has a large range of scale scores, many students stay in the intermediate level for multiple years. As a result, this level is divided for accountability purposes only, to recognize the substantial growth that can be made within this particular level on the CDE Dashboard. CDE recognizes 6 levels for accountability. Schools that did not test at least 50 percent of their EL population in the CELDT are automatically assigned an Orange performance level (pie) The EL population is determined by the number of EL students who took the Smarter Balanced mathematics assessment, as all students (even newly arrived EL students) are required to take the mathematics assessment.

How are students performing in ELA on the SBAC How are students performing in ELA on the SBAC? Are all subgroups demonstrating growth at the same rate?

Grade 3 – Grade 8 How well are students and subgroups at my school meeting ELA standards?

Are students graduating College and Career Ready? High School How well are students and subgroups at my school meeting ELA standards? Are students graduating College and Career Ready?

Resources The CDE’s Dashboard Website includes comprehensive resources, sample presentations & letters, technical guides, video overviews, and metrics used to determine the indicators: http://www.cde.ca.gov/ta/ac/cm/index.asp Calculation questions on state indicators, the college/career indicator, and grade distance from level 3: aau@cde.ca.gov tel. 916.319.0863 CALPADS: calpads@cde.ca.gov tel. 916.324.6738