1. Regional Assessment Network Meeting
Analysis, Measurement, and Accountability Reporting Division
March 2018

2. Topics Revisions to the Every School Succeeds Act (ESSA) State Plan
Additional State Board of Education (SBE) meeting to be held in April 2018
Participation Rate
Proposed Revisions to the 2018 Dashboard
We have a lot of ground to cover today. We’ll look at:
Some of the new features and reports in the Fall 2017 Dashbaord
Actions taken by the Board last week, which included revisions to the ESSA plan (in response to feedback from the US. Department of ED) and criteria used to identify the lowest performing schools
California Department of Education

3. Revisions to the ESSA State Plan
On February 22, 2018, CDE and SBE staff met with U.S. Department of Education (ED) staff in Washington, DC to discuss ED interim feedback to the ESSA Plan.
Based on ED guidance from this meeting, several changes to California’s Plan were proposed to the SBE at its March 2018 meeting.
California Department of Education

4. ED Required Revisions to State Plan
Add Grade 11 Academic Indicator.
Incorporate participation rate into Academic Indicator.
Make the College/Career Indicator (CCI) a School Quality or Student Success Indicator rather than an Academic Indicator.
ED: U.S. Department of Education
California Department of Education

5. ED Required Revisions to State Plan (Cont.)
Remove Reclassified Fluent English Proficient (RFEP) and Long-term English Learners (LTELs) from the English Learner Progress Indicator (ELPI).
Identify high schools for support based on low graduation rate (<67%): Apply three-year average rather than three consecutive years.
California Department of Education

6. Additional SBE Meeting
Because negotiations with the ED were not completed until March 13th, one day prior to the March SBE, a special SBE meeting will be scheduled in April 2018 to address Title I, Part A accountability requirements.
Allows time for impact analysis and stakeholder feedback.
Because of ongoing communication between the CDE/SBE and the ED, there was a delay in posting all components of revised ESSA State Plan in time for the March 2018 SBE meeting
California Department of Education

7. Academic Indicator for High Schools
The CDE is proposing to create an Academic Indicator for grade eleven to report Distance from Level Three (DF3) to align with ESSA requirements.
Will need to run distributions for grade eleven to establish cut scores for both Status and Change.
California Department of Education

8. Selection of Schools for Comprehensive/Targeted Support
The CDE is proposing:
A new set of color combinations to identify schools for Comprehensive Support (i.e., 5%) over three years
To identify schools for Targeted Support if they meet the same criteria for Comprehensive Support 3 out of 4 years
California Department of Education

9. California Department of Education
Changes to the ELPI
In accordance with ED requirements:
CDE must submit ESSA Plan with required changes to the ELPI (removing RFEP and LTEL students)
Once ESSA Plan is approved by the ED, CDE can submit a waiver to the ED requesting that these students be included
CDE includes in its Progress in Achieving English Language Proficiency indicator not only students who were reclassified in the current year but also students who were reclassified in the prior year, meaning that they were not English learners in the year for which accountability determinations are being made.
Section 1111(c)(4)(B)(iv) of the ESEA requires a State to include in this indicator only those students currently classified as English learners. Note that ESEA section 1111(b)(3)(B) provides a narrow exception that permits students previously identified as English learners to be included in the subgroup of English learners only for purposes of measuring the performance of English learners on the statewide reading/language and mathematics assessments for purposes of the State-determined accountability system.
California Department of Education

10. California Department of Education
Impact on Dashboard
Although the ESSA State Plan includes revisions on how the goals are established, this will not impact the Dashboard.
The Dashboard will continue to report Status; the ESSA goal is now based on Status cut scores.
The Dashboard will continue to report Change, which is now an additional Academic Indicator for ESSA.
The Dashboard will continue to report performance colors, which is part of the state accountability system.
California Department of Education

11. Reporting the Participation Rate
California Department of Education

12. Reporting Participation Rate
States must report and factor the 95 percent participation rate into their accountability systems.
Will impact the Academic Indicator, and Distance from Level 3 (DF3), beginning in
Denominator must include at least 95 percent of all students and each student group.
California Department of Education

13. Incorporation of Participation Rate into Academic Indicator
Two proposals under consideration:
Select students to bring participation rate to 95% for all students and each student group
Include 100% of all eligible students in denominator
ESEA section 1111(c)(4)(E)(ii) requires that a State calculate the Academic Achievement indicator by including in the denominator the greater of 95 percent of all students (or 95 percent of students in each subgroup, as the case may be) or the number of students participating in the assessments.
California Department of Education

14. Incorporation of Participation Rate: Proposal 1
Assign Lowest Obtainable Scale Score (LOSS) to select students needed to bring participation rate to 95%
Select students based on grade level
LOSS varies by grade level
California Department of Education

15. Incorporation of Participation Rate: Proposal 1
Example:
Olympic Elementary School has 100 students enrolled during their testing window. Of these, only 80 students took the mathematics test. The school:
Has a 80% math participation rate.
Should have tested 15 more students to reach the 95% goal.
15 of the 20 students who did not participate (but should have because they were enrolled during the testing window) will be assigned the LOSS.
Problem: Which of the 15 students should be selected?
California Department of Education

16. 80 Students Took the Math Test =
Visual
95% Participation Rate Goal
15 more students needed to reach 95% goal.
Which 15 (out of the 20 who didn’t test) should be selected to apply the LOSS to?
Reminder: LOSS varies by grade level.
Testing Window
100 Students
80 Students Took the Math Test =
80% Participation Rate
California Department of Education

17. Incorporation of Participation Rate: Proposal 1 (Cont.)
This model becomes more complicated when working at student group level
Must look at each grade level within each student group
Several students may fall into two or more groups (e.g., race/ethnicity, low socio- economic status, English learners)
Duplication will occur when rolling up to school level.
California Department of Education

18. Incorporation of Participation Rate: Proposal 2
Include 100% of students who should be tested:
For each grade level, find lowest scale score obtained among testers
Assign that scale score to each student at that grade level who did not test
Factor all scale scores for all grades into calculations for school’s DF3.
California Department of Education

19. Incorporation of Participation Rate: Proposal 2
Example: If Minnie had the lowest scale score among all 5th grade test takers, then her score will be applied to any 5th grade student who did not participate (but should have).
California Department of Education

20. Visual
100 Students at Olympic High Enrolled During Testing Window
Minnie had the lowest math scale score out of all of her 5th grade classmates.
Minnie’s score is assigned to each of the three Grade 5 students who did not test.
20 Students Didn’t
Take the Test
Repeat this process for all grade levels until all 20 students are assigned the lowest observed score.
Of the 20, three
were in Grade 5.
California Department of Education

21. California Department of Education
Feedback from RAN
Which proposal do members prefer?
Are there other methodologies that RAN members would like CDE to explore?
Should CDE continue with its plan to display icons in the Dashboard to report different levels of participation?
Should LEA, school, and student group participation rates be reported in the Detailed Reports as well?
CDE will work with Technical Design Group (TDG) over coming months to develop methodology.
California Department of Education

22. Proposed Revisions to the 2018 Dashboard
On February 22, 2018, CDE and SBE staff met with U.S. Department of Education (ED) staff in Washington, DC to discuss ED interim feedback to the ESSA Plan.
Based on ED guidance from this meeting, several changes to California’s Plan were proposed to the SBE at its March 2018 meeting.
California Department of Education

23. California Department of Education
Changes to 2018 Dashboard
Chronic Absenteeism Indicator
College/Career Indicator (CCI)
Modified methods for Dashboard Alternative School Status (DASS) schools
Graduation Rate Indicator
Application of Safety Net Methodology for Student Groups
Student-Level Growth Model
ELPI: Not reported in 2018 Dashboard
California Department of Education

24. California Department of Education
Chronic Absenteeism
Cut scores for both Status and Change will be recommended to the SBE in fall 2018
2018 Dashboard will report for the first time:
Status,
Change, and
Performance levels (colors)
California Department of Education

25. New Data Reported for CCI
2018 Dashboard will report:
Status (Class of 2018) and Change
Performance levels (colors)
New career measures for the Class of 2017
California Department of Education

26. New Career Measures Reported for Class of 2017
State Seal of Biliteracy
Articulated Career Technical Education (CTE) courses
Golden State Seal Merit Diploma
Leadership/Military Science (formerly ROTC)
The CDE will run simulations on the following career data already collected in CALPADS:
Leadership/Military Science (formerly ROTC)
Work Experience Education
Articulated CTE Pathways
Golden State Seal Merit Diploma
Based on simulation results, both groups will begin making decisions on “Prepared” and “Approaching Prepared” levels.
California Department of Education

27. Modified Methods for DASS Schools
DASS schools will receive Dashboard for the first time
Modified methods under consideration include:
Graduation Rate Indicator: One-Year Graduation Rate
CCI: Modified Career Measures
Simulations conducted on one-year graduation rate, using “90 Day Continuous Enrollment” methodology:
Based on calendar year: July 1 through June 30
All grade 12 students must be enrolled for at least 90 consecutive calendar days
One-year rates produced for Class of 2015, Class of 2016, and Class of 2017
Alternative Task Force will meet on March 23 to determine whether simulation results can be used to set Status and Change cut scores.
California Department of Education

28. Graduation Rate Indicator
Revision to calculation for Change: Use one- year rate (rather than the three-year average):
Easier to understand and communicate
Consistent with other state indicators
More accurately represents actual change
More transparent and easier to use
California Department of Education

29. Five-Year Graduation Rate
SBE agreed that CDE should pursue five- year rate and requested further analyses on five-year graduates:
Are they students with an IEP or 504 Plan?
Are they LTELs?
CDE will be pursuing options for incorporating the five-year cohort rate into the Dashboard.
California Department of Education

30. Feedback on Five-Year Graduation Rate
To meet ESSA requirements, criteria for five- year rate must be more rigorous.
Suggestions from RAN?
How should it be reported in the Dashboard?
Criteria must be more rigorous. One board member gave example of student completed their Associates Degree while also completing the 5th year of high school)
How should it be reported in the Dashboard?
SBE would like it reported separately.
One proposal is to report it as an alternative indicator.
Feedback
California Department of Education

31. Safety Net Methodology Applied at Student Group Level
SBE approved that methodology be applied to student-group level (n ≥ 30) for two state indicators:
Graduation Rate Indicator: If student group has fewer than 150 students in graduating cohort
Suspension Rate Indicator: If student group has fewer than 150 students who are cumulatively enrolled.
California Department of Education

32. Safety Net Methodology
Results in a refiguring of performance level tables (from a five-by-five colored table to a three-by-five) by removing two Change levels:
Increased Significantly
Decreased Significantly
Prevents large swings in data triggered by results of a few students.
California Department of Education

33. California Department of Education
Student Growth Model
2018 Dashboard will include a Student Growth Model.
Simulations have been conducted for three proposed growth models:
Change-in-distance-to met
Conditional percentile rank of the gain
Residual gain
California Department of Education

34. Student Growth Model (Cont.)
Three criteria were used to evaluate models:
Strength of relationship with student demographic characteristics
Sensitivity to school grade configuration and assessment content area
Statistical precision
California Department of Education

35. Student Growth Model: Recommendation
Residual Gain model has most desirable properties overall.
Conditional percentile rank model appears to be more influenced by student demographic characteristics
Change-in-distance-met model appears to be more sensitive to grade configuration and content area and has slightly lower statistical precision.
California Department of Education

36. Student Growth Model: SBE Timeline
May 2018: Select student growth model
July 2018: Decide on how to display and report data for student growth model
September 2018: Decide on how to incorporate student growth model into Dashboard
California Department of Education

37. Questions and/or Discussion
California Department of Education