1. Regional Assessment Network Meeting
Analysis, Measurement, and Accountability Reporting Division
May 2018

2. Topics
Growth Model
Dashboard Reports versus Data Reported to the U.S. Department of Education (ED)
Changes to 2018 California School Dashboard (Dashboard)
Incorporation of participation rate into Academic Indicator
We have a lot of ground to cover today. We’ll look at:
Some of the new features and reports in the Fall 2017 Dashbaord
Actions taken by the Board last week, which included revisions to the ESSA plan (in response to feedback from the US. Department of ED) and criteria used to identify the lowest performing schools

3. Student Level Growth Model
On February 22, 2018, CDE and SBE staff met with U.S. Department of Education (ED) staff in Washington, DC to discuss ED interim feedback to the ESSA Plan.
Based on ED guidance from this meeting, several changes to California’s Plan were proposed to the SBE at its March 2018 meeting.

4. SBE Action Approved Residual Gain for further exploration.
Requested further information at July SBE meeting:
What are the benefits of the residual gain model over the distance from level 3 already included in the Dashboard?
Should parent education level be included as a variable in the model?
How should the model be used (i.e., in the Dashboard for accountability, reporting only, etc.)

5. RG = Student’s current year score – predicted score
Residual Gain (RG)
Represents difference between student’s predicted and actual test score
Predicts student’s current-year score in either mathematics or ELA using the student’s prior-year mathematics and ELA scores.
RG = Student’s current year score – predicted score
Provides a relative measure of student growth on the current test scale.

6. Next Steps (May – July)
May Webinar: Obtain feedback on the growth model interest
June 2018:
Obtain feedback on growth model from LCFF stakeholder group
Review RG model for needed technical refinements at the TDG meeting
July 2018 SBE Meeting: Present growth model simulation results and any recommended refinements and discussion around expected metrics and indicator placement.

7. Next Steps (September)
September 2018 SBE Meeting: SBE approval of final model and methodology for possible inclusion in the Dashboard.
California Department of Education

8. Feedback and Questions
Is there still interest in a student growth model?
If so, how would districts and county offices use the information in growth model to improve classroom instruction?
What is the best option for reporting the growth model results?
The Change component for DF3
Separate Indicator
Detailed Reports Only
Other

9. One-Year Graduation Rate

10. SBE Action and Next Steps
At the May 2018 meeting, the SBE approved a one-year graduation rate for DASS schools for implementation beginning with the 2018 Dashboard.
Next Steps: Work with the Alternative Schools Task Force, the LCFF Stakeholder Group, and the Technical Design Group (TDG) to determine:
How to include the one-year graduation rate in the district’s Dashboard.
If other indicators need to have modified methods
If indicators for DASS schools need their own cut scores.

11. Data Reported in 2018 Dashboard Reports versus Data Reported to U. S
Data Reported in 2018 Dashboard Reports versus Data Reported to U.S. Department of Education
On February 22, 2018, CDE and SBE staff met with U.S. Department of Education (ED) staff in Washington, DC to discuss ED interim feedback to the ESSA Plan.
Based on ED guidance from this meeting, several changes to California’s Plan were proposed to the SBE at its March 2018 meeting.

12. Every Student Succeeds Act (ESSA) Requirements
Three required indicators for which states must set goals:
Academic Achievement (ELA and Math assessments)
Graduation Rate
English Language Proficiency
ED’s statutory interpretation is that these indicators must be based only on current year data (Status)

13. Revisions to ESSA Plan to Meet Federal Requirements
Treat “Status” and “Change” components of the Dashboard Indicators as separate indicators for federal purposes
Status is the required indicator
Change is an additional indicator
Set goal relative to Status

14. ESSA Long-Term Goals
For federal purposes, only Status will be used to set long-term goal:
Graduation rate = 90 percent
ELA (grades 3-8) = +10 Distance From Level 3 (DF3)
Math (grades 3-8) = zero DF3
English Learner Proficiency = 75 percent of students make progress
Technical work must be completed before determining ELA and math goals for grade 11

15. Elements of ESSA Plan that Remain Intact
Calculate Dashboard indicators using both status and change
No changes to the five-by-five color-coded grids that the SBE approved for the Dashboard Indicators
Use color-coded performance levels to identify the lowest performing five-percent of Title I schools
Based on both Status and Change
The goals for these indicators are not materially different in practice

16. Changes to the 2018 Dashboard
On February 22, 2018, CDE and SBE staff met with U.S. Department of Education (ED) staff in Washington, DC to discuss ED interim feedback to the ESSA Plan.
Based on ED guidance from this meeting, several changes to California’s Plan were proposed to the SBE at its March 2018 meeting.

17. Chronic Absenteeism
Cut scores for both Status and Change will be recommended to the SBE in fall 2018
2018 Dashboard will report for the first time:
Status,
Change, and
Performance levels (colors)
Cut scores will be determined at the November SBE meeting

18. New Data Reported for CCI
2018 Dashboard will report:
Status (Class of 2018) and Change
Performance levels (colors)
Possible new career measures for the Class of 2017
State Seal of Biliteracy
Articulated Career Technical Education (CTE) courses
Golden State Seal Merit Diploma
Leadership/Military Science (formerly ROTC)
Change cut scores will be determined at the November SBE meeting

19. Graduation Rate Indicator
Calculation for Change based on one-year rate (rather than the three-year average).
Note: New federal requirements necessitate changes to the calculation of the four-year cohort rate. See Flash 136 at
Incorporation of five-year cohort rate: To be determined
Scheduled for consideration at the September SBE meeting

20. Safety Net Methodology Applied at Student Group Level (n ≥ 150)
For two state indicators:
Graduation Rate Indicator: If student group has fewer than students in graduating cohort
Suspension Rate Indicator: If student group has fewer than students who are cumulatively enrolled.
SBE will consider at the July SBE meeting

21. Incorporation of Participation Rate into Academic Indicator

22. Incorporation of Participation Rate into Academic Indicator
In accordance with ED direction:
Use of icons not sufficient to meet ESSA requirements
Participation rate must be factored into Academic Indicator
Specifically, for schools and student groups that did not test 95 percent, the number of students needed to bring the school and/or student group up to 95 percent shall be included in the Academic Indicator.

23. Proposed Option
Calculate the percentage points the school or student group is below the 95 percent participation rate target and reduce the DF3 by a fraction of the number of points short (i.e., ¼ the number of points short of participation rate)

24. ELA Example Emerald Elementary School (K-6)
Number of students enrolled during testing window:
357
Number of students tested:
281
Calculated participation rate (rounded):
79%
Number of additional students needed for 95% participation: 55
(281+55) / 357 = 94.1%
Number of tested and continuously enrolled students (valid):
258
DF3 sum of 258 valid tests
4355

25. ELA Example Calculated
Current DF3
𝐷𝐹3 𝑠𝑢𝑚 𝑜𝑓 𝑣𝑎𝑙𝑖𝑑 𝑡𝑒𝑠𝑡𝑠 # 𝑜𝑓 𝑣𝑎𝑙𝑖𝑑 𝑡𝑒𝑠𝑡𝑠 = 4, =16.9 (𝐻𝑖𝑔ℎ)
Proposed DF3
4,355+55(𝐿𝑂𝑆𝑆) = 4,355+(−15,555∗) 313 = −12, =−39.0 (𝐿𝑜𝑤)
*-301 (LOSS Score) multiplied by 55

26. Example: Calculation of the Adjusted DF3
For this example, the DF3 is being reduced by ¼ point for each percentage point the school is below the 95 percent participation rate target.
Adjusted DF3 Calculation
16 points shortage X .25 = 4 points
16.9 Current DF3 – 4 points reduction = 12.9 (High)

27. Pros and Cons—Using Adjusted DF3
All entities receive equal penalty for missing the 95 percent participation rate.
The further away an entity is from the 95 percent participation rate, the greater the penalty.
Simple to calculate.
Minimizes the impact of the of incorporating the participation rate into the Academic Indicator.
May be more complicated to explain.

28. Discussion Questions
Do RAN members have any concerns with the proposed methodology for incorporating the participation rate?
Are there other methodologies that should be explored?

29. RAN Request School climate surveys
A slide deck and flyer that explain Residual Gains—do the slides used at the SBE meeting work?
A slide deck to use when talking to parents about the dashboard—Does the slide deck posted on the Web page meet your needs?
Communication talking points for residual gain and student growth
Federal LCAP Addendum training materials and Timeline
Crosswalk of changes to the Dashboard & accountability in general—More information needed.
More clarification for the specific indicators—More information needed.
A tool to help school sites determine their DF3(LEAs)—Excel spreadsheet?
There should be a key that lists definitions for indicators for each year
How are districts developing metrics to check state indicator progress—More information needed

30. Growth Slides Discussion Reference Only

31. How Does RG Work: Scatter Plot of Scores

32. How Does RG Work: Line of Best Fit

33. How Does RG Work: Calculate RG for Each Student

34. How Does RG Work: Results for One Student