1. Results Driven Accountability Pilot
Kyle Laughlin - TIE
Jamie Morris - South Dakota DOE

2. Data Confirmation – South Dakota

3. Data Confirmation – South Dakota (Cont.)

4. Data Confirmation – South Dakota (Cont.)

5. Stakeholders RDA Work Group – Core Work Group
Group of diverse individuals – teachers, SPED directors, school psychologists, etc.
Provided input and guidance.
Core Work Group
Results Driven Manual
Risks Analysis Rubric
On-going guidance and support

6. Direction: TIE – SIG TIE SIG
SD Systemic Collaborative Data Process Pilot and Trainings
SIG
Provided information to State on how other States were designing an RDA system
Collaborated together to develop pilot timeline and manual
Moving forward!
Provided information to State on how other States were designing an RDA system

7. RDA Pilot Project Timeline
Proposed District #s
January-June 2017
July 2017 – June 2018
July 2018 – June 2019
July 2019 – July 2020
Begin process with three Pilot Districts: Small, Medium, Large (Select from Level 1, 2, or 3)
Full support from TIE/SIG
District team begins learning system and putting systems and processes into place
DOE Support Team Members shadow TIE/SIG
District team continues implementing systems and processes
DOE Support Team Members shadow and practice some processes with TIE/SIG as co-presenters
Continued support from TIE/SIG
Partial release of responsibility to district
Continued support from TIE/SIG with partial release of responsibility to DOE Support Team Members
Full release of responsibility to district
Full release of responsibility to DOE Support Team Members
Requested support from TIE/SIG as needed
Add Two or Three districts (Level 3 only)
Options offered to all districts
Beginning January 2018, offer multiple regional trainings for all districts to attend to learn about the new RDA process, compliance, rubric, guidelines, etc. (Not a requirement but a jumpstart)
Add Five Districts (Level 2 or 3)
Full implementation of RDA process for all districts in SD
Kyle
Handout/Manual
What is this timeline missing – what other information do you need?

8. Development of Pilot Project
Selection of Pilot Schools
Size and Effect
Geographical Demographics
Risk Analysis Rubric – indicators and parameters
Capacity at state and district level
Implementation of RDA Pilot – Informative trainings

9. Pilot Forms Risk Analysis Rubric Internal Review
Weighed reading & math indicators
Internal Review
Quantified to become data set
Professional Development Form
Directed PD for districts
Child Count Assurance Form

10. Pilot Districts Overview
Student Population
1 Small (less than 300)
1 Medium (between 300-1,000)
2 Large (over 1,000)
1 Large District is considered high needs (Level 3)
Kyle
How each district was selected.

11. Low Needs – Level 1 (80% of Districts)
(Optional) -Assembles RDA Committee to review SPED data, conduct root cause analysis, and/or develop action plan.
(Optional) -Participate in Regional RDA informative training.
(Optional) -Participate in a SPED regional and/or local data retreat.
Every SPED teacher in district will complete internal review of compliance practices and the district SPED director submits a statement of assurance of completion of review.
Submits Professional Learning Plan to SDDOE. (Form will be given by the state).
RDA coaches will be available to provided support if the district requests it.
Arlene
Discuss how this information is useful, but will not impact the pilot districts.

12. Medium Needs - Level 2 (15% of Districts)
Assembles RDA Committee
Participates in Regional RDA informative training.
Participates in regional and local data retreats. Reviews local SPED data, conducts root cause analysis, and develops action plan to address local areas of concern.
Districts will receive on-going support from RDA coaches on implementing action plan.
Every SPED teacher in district will complete internal review of compliance practices and the district SPED director submits a statement of assurance of completion of review.
Submits Professional Learning Plan to SDDOE. (Form will be given by the state)
Arlene
Discuss how this information is useful, but will not impact the pilot districts.

13. High Needs – Level 3 (5% of Districts)
All Pilot districts will meet the requirements of level 3,
Assembles RDA Committee-participate in regional and local data retreats. Reviews local SPED data, conducts root cause analysis, and develops improvement plan to address local areas of concern.
RDA committee will participate in data retreat each year.
Districts will receive monthly support from RDA coaches on the development and implementation of their improvement plan.
Every SPED teacher in district will complete internal review of compliance practices and submits a statement of assurance of completion of review. On-site support from RDA coaches when conducting the yearly internal review.
On-site support for conducting root cause analysis and development of improvement plan (for both compliance and results areas). RDA coaches will guide this improvement plan.
Districts will submit improvement plan to SDDOE, with the assistance of RDA coaches
On-site TA provided monthly to support districts improvement strategies.
Submits Professional Learning Plan to SDDOE. (Form will be given by the state)
District will report to local school board on activities.
Arlene

14. Next Steps Each District has Informational Training
Two-Day Data Retreat
Data retreat will customized year long process
On-site support

15. SEE YOU IN 2019 TO HEAR OUR RESULTS!

16. How Colorado Balances Compliance and Results in its LEA Determination
Miki Imura
Supervisor of Data Accountability & Achievement
Colorado Department of Education
Jon Paul Burden Director- Exceptional Student Services
Weld Re-4 Windsor School District

17. CDE Convened a Results Matrix Work Group
Purpose:
Create an LEA evaluation system that values academic results
Use the product as part of LEAs’ Annual Determination
The work group first met in January 2015
The work completed (for now) in March 2017
2017 LEA Determination
Included the newly developed Results Matrix
Accounted for 25% of the LEA Determination AU
CDE

18. Work Group’s Wish List Do:
Create similarity between District/School Performance Framework and the Results Matrix
Emphasize student growth
Identify attainable targets
Acknowledge areas of strength
Utilize as an evaluation/improvement/communication tool
Don’t:
Create winners and losers (we are all in this together)

19. Similarity With District Performance Framework Emphasis on Growth
Similarity between SPF/DPF and the Results Matrix
State Assessments
Colorado IEP Accountability Participation in ELA and Math
OSEP Accountability Participation in ELA and Math
Regular Assessment Mean Scale Score in ELA and Math
Alternate Assessment Proficiency Rate in ELA and Math
Preschool Skill (Indicator 7)
Positive Social-Emotional Skills
Acquisition and Use of Knowledge and Skills
Use of Appropriate Behaviors to Meet their Needs
Graduation rate
Dropout rate (Indicator 2)
Post-School Outcomes (Indicator 14)
The percent of former students selected in the post- school outcome interview sample whom AU attempted to reach.
The percent of former students who participated in the post-school outcome interview.
Of the former students who participated in the post- school outcome interview, the percent who are:
Enrolled in higher education, or
In some other post-secondary education or training programs, or
Competitively employed, or
In some other employment
Academic Achievement 15%
Median Growth Percentile in ELA and Math
Rise Up in ELA and Math
Keep Up in ELA and Math
Emphasis on Growth
Postsecondary & Workforce Readiness 35%
Academic Growth 50%

20. Attainable Targets – “Norm & Freeze”

21. Attainable Targets – Example of Grad Rates

22. Attainable Targets – Example of Grad Rates

23. Slide 23
CO’s students on IEPs are not doing well across the states, so no AU would get any points in the results matrix!
That’s not true!!
The top 10% of AUs receive the full 3/3 points first year (2017). And the number of AUs that receive 3/3 points will (hopefully) increase from the 2nd year on!

24. 2017 LEA Determinations

25. Results Matrix

26. Academic Achievement 15%

27. Academic Growth 50%

28. Rise Up / Catch Up / Keep Up
Colorado Growth Model
A student who is currently proficient is on track to keep proficiency for the next 3 years or until 10th grade = Keep Up
Unsatisfactory
Partially Proficient
Proficient
Advanced
Keep Up
Worth 30 points of the Results Score

29. Rise Up / Catch Up / Keep Up
Colorado Growth Model
A student who is under proficient is on track to reach proficiency in 3 years or before 10th grade = Catch Up
Unsatisfactory
Partially Proficient
Proficient
Advanced
Catch Up
But it’s EXTREMELY hard for a student currently in unsatisfactory to be on track to reach proficient in 3 years or before 10th grade.

30. Rise Up / Catch Up / Keep Up
A student who is under proficient is on track to go up a step in 3 years or before 10th grade = Rise Up
Unsatisfactory
Partially Proficient
Proficient
Advanced
Rise Up
Rise Up
Worth 90 points of the Results Score
We put a HEAVY emphasis on the growth of students who are under proficiency (78.5% of Special Education Population in TCAP 2014, 93% in PARCC 2016).

31. Postsecondary & Workforce Readiness 35%

32. Summary Page

33. LEA LEVEL ANALYSIS
Utilization of the norm charts helps to determine the gap to additional points and higher performance.
Provides for targeted analysis of specific areas of performance rather than overall achievement.
Growth data allows for deep dive into building level data
Can drill down to the classroom or student level for targeted improvement
Identifies achievable and meaningful improvements at student and system level.

34. ELA Median Growth Percentile (15 points)

35. Slide 35

36. Dropout Rate (42 points)

37. JP’s section With small “n” sizes these measures are highly sensitive
1 less dropout puts me over the 50th percentile
1 less graduate drops me below the 50th percentile
This provides a clear urgency for staff to establish creative and innovative programming to keep students in school
Over 130 school districts of 178 are considered small or rural in Colorado…thus small “n” sizes affect 2/3’s of all districts.

39. Questions and Comments?
Colorado
Jon Paul
Miki
Toby – CO’s State Director
South Dakota
Kyle Laughlin
Jamie Morris