1. Structure and purpose of webinar Being recorded for future viewing
Introductions
Our Team
Structure and purpose of webinar
Being recorded for future viewing
Will go through slides and respond to questions at the end
Please feel free to type in questions into chat box
Or at the end, we will open up microphones
Nazie

2. Monitoring Overview
Nazie
October 15, 2018

3. Purpose of this Monitoring Webinar
Introduce monitoring framework and purpose
Orientation on the plans
Preparing for monitoring
Timeline
Preview future years’ plans
Answer any remaining questions N

4. Monitoring Update
Every Student Succeeds Act (ESSA) provided an opportunity to review, update, and revamp monitoring processes, protocols, and focus
Monitoring Stakeholder Group
Meeting and working for past two years
Updated and upgraded monitoring process and protocols
Aligned with ESSA and Colorado’s ESSA State Plan (approved by U.S. Department of Education in May, 2018) N

5. Purpose Diagram Improved Student Outcomes Improved Implementation
Compliance
Strengths
Improved Implementation
Effective Practice N:
Under the ESSA statute, each state educational agency that accepts federal funds is responsible for ensuring compliance with ESSA requirements through monitoring of local educational agencies’ use of funds and implementation of ESSA-funded programs, strategies, and practices. But monitoring also provides us a tremendous opportunity to focus on what matters and that is improved outcomes for students.
Therefore, in addition to meeting the statutory requirement to ensure compliance with ESSA, Colorado will be using the monitoring process to
Identify and highlight areas of strength and effective implementation within each local educational agency (LEA; includes districts and BOCES)
Collaborate with LEAs to implement effective programs and practices to improve outcomes for students
Identify exemplary practices within the state to learn from and disseminate

6. Plans Based on Stakeholder Input
Focus efforts on a limited number of high-impact indicators
Prioritize a select number of indicators
Phase-in indicator categories over time
Provide training, guidance, and technical support before monitoring
Provide follow-up and follow-through support based on monitoring results
Collaboratively build monitoring plans and reports with each LEA N

7. Structure: Three Levels of Review
All LEAs will be randomly selected to be reviewed over the course of the next 6 years
Prioritized indicators
Desk and phone reviews
Annual selection of several (between 10 to 15) LEAs for more focused look at implementation and support
Selected based on size of award, newness of staff, or other eligibility criteria (to be discussed in more detail later in this training)
Select prioritized indicators aligned to reasons for selection
Desk, phone, and possibly onsite reviews
Annual selection of a few (2 to 5) LEAs for a very focused review of implementation and continuous support
Selected based on school performance
Select prioritized indicators
Desk, phone, and onsite N

8. Question: What to Call the Levels?
ESSA State Plan refers to universal, targeted, and intensive
Stakeholders struggle with those terms
At the end of the webinar, we’ll collect a vote on suggested names for each level N

9. Structure: Four Categories of Indicators
Fiscal
Meaningful Stakeholder Engagement and Partnership
Identification and Delivery of Services
Evaluation of Program Impact N

10. Structure: Four Categories of Indicators ~ Phase-in Plans
and Beyond
Fiscal
Meaningful Stakeholder Engagement
and Beyond
Identification and Delivery of Services
and Beyond
Identification and Delivery of Service
Evaluation N:
To encourage deeper conversations around specific indicators, CDE will monitor 2 categories of indicators in the first year – – fiscal and meaningful stakeholder engagement.
In , indicators pertaining to identification of students for supports and services, as well as delivery of those services, will be added.
In , indicators pertaining to the evaluation of program effectiveness will be added.

11. Coordinated Monitoring
CDE Partners Collaborating and Coordinating Monitoring Efforts
Fiscal requirements and use of funds
In collaboration with ESSU and Grants Fiscal
Meaningful stakeholder engagement and partnerships, including stakeholder engagement in various processes such as CS improvement planning, equitable services, etc.
In collaboration with UIP and Parent and Family Engagement Coordinator
Identification and delivery of services (implementation) aligned back to the needs assessment, and for schools identified for support and improvement aligned to reasons for identification
In collaboration with CLDE, Homeless, Dropout Prevention, School and District Improvement, and other units that provide supports and services
Local Evaluation of program impact
Training resources are being built in collaboration with REL N:
Unit of Federal Programs Admin has been and will continue to coordinate monitoring efforts with other CDE units, to
Minimize burden on LEAs
Submitted evidence will be used to meet multiple indicators and for multiple programs/funding sources (e.g., ESSA and IDEA)
Allow districts to decide whether CDE units should visit together or separately (if visiting in the same year)
Provide coordinated follow up support for implementing any follow up or follow through actions required as a result of monitoring

12.
Joey

13. *Intensive* (Desk, Phone, and Onsite) Reviews and Support: Compliance, Implementation, and Program Effectiveness
Selection Criteria
LEAs with 3+ and more than 10 percent of their schools identified for comprehensive (CS) support and improvement this year (in the future it will be high number and percentage of CS and TS)
Purpose
Check for compliance with ESSA and implementation of effective or evidence-based interventions, strategies, programs to improve student outcomes
Support LEAs and schools with improvement efforts
Ensure reasons for ESSA identification are addressed, student performance is improving, and the school(s) can exit improvement status
Ensure there is equitable distribution of resources to minimize opportunity gaps for historically underserved students
Identify exemplary practices
Process
Multi-year relationship and ongoing support
Onsite visit (1.5-2 days) that builds on universal and targeted reviews and high leverage indicators J
Collaboration with CLDE, Drop Out Prevention, Homeless,

14. *Intensive* Reviews Select 3-5 per year Timeline
Will not be reselected for another 5 years but will receive ongoing support, universal and possibly targeted monitoring
Timeline
January – May ~ onsite visits
7+ days of work (pre-, during, and post-visit); multi-years of ongoing support, in collaboration with other CDE units
30 days after visit ~ reports
Subsequent year ~ follow up support
3 districts
and Beyond
Select new LEAs that meet the criteria, not previously selected J
45 calendar days to receive docs and 45 to publish report.
CDE is estimating the number of days to prepare and conduct the onsite visit; we will be tracking time, effort, and expenses associated with onsite visits for both the LEA and CDE to be able to provide that information with LEAs in the future.

15. *Targeted* (Desk, Phone, and Onsite) Reviews: Compliance and Implementation
Selection Criteria
Annual fiscal, programmatic, & accountability risk assessment
Purpose
Ensure compliance with ESSA and implementation of effective or evidence-based interventions, strategies, programs to improve student outcomes
Ensure that LEAs address any potential risks that might hinder their performance
Ensure that the schools identified for ESSA improvement are addressing reasons for id, improving student outcomes, and are moving towards exiting status
If have CS or ATS schools, equitable distribution of resources; minimizing opportunity gaps for historically underserved students
Identify exemplary practices
Process
Multi-year relationship and ongoing support
Mostly desk and phone reviews, with meetings that occur during other events (e.g., CFS site visit, RNM in the area, Cons App planning meeting) J

16. Selection Criteria for Targeted Review
Admin criteria
Date of last monitoring visit
Turnover in district personnel
Fiscal criteria
Maintenance of Effort
Audits & Corrective Actions
Size of the Award
Reverting Funds/Excess Carryover
Suspended-Terminated Grants
Comparability
Late Fiscal Reporting
Accountability and program criteria
# and % of the LEA’s schools that have been identified for ESSA improvement & support
Non-compliance with equitable distribution of services to non-public schools
Inequitable distribution of teachers based on CDE analyses
Non-compliance with supplement, not supplant provisions of Title I, Part A
Unauthorized delay in submittal of the LEA’s Consolidated Application (excludes any late submissions based on delays in the system and/or deadline extensions awarded by CDE)
Non-compliance with universal indicators from the previous year’s desk review (first identification to occur in based on spring 2019 desk review results) J

17. Universal (Desk and Phone) Reviews: Compliance and Program Impact
Selection Criteria
All LEAs ~ 6 year cycle
Purpose
Ensure compliance with ESSA and implementation of effective or evidence- based interventions, strategies, programs to improve student outcomes
Identify exemplary practices
Process
Predominantly desk review; some follow-up phone conversations
Review of existing CDE processes and systems (e.g., the Consolidated Application, Unified Improvement Plans, data collections)
Might require additional evidence submitted for desk review J

18. Universal Targeted 28-30 LEAs 10 LEAs
Plans
Universal
28-30 LEAs
Indicators reviewed based on data/evidence already submitted to CDE:
Meaningful stakeholder engagement and partnership
Fiscal
Timeline
Reviews conducted in January – February 2019
Feedback provided March- May 2019
Follow up provided until LEA is in compliance (could be into the next year)
Targeted
10 LEAs
Indicators – more detailed look into implementation of (could require additional evidence):
Meaningful stakeholder engagement and partnership
Fiscal
Related to selection criteria
Timeline
Reviews conducted in March – May 2019
Feedback provided June – July 2019
Follow up provided until LEA is in compliance (could be into the next year) J

19. 2018-2019 Timeline October and November
Conduct a recorded webinar explaining process
Send communication to LEAs selected for monitoring process and protocol
Provide guidance and training on fiscal and stakeholder engagement indictors; monitoring process and feedback on each type of review
December ~ One-on-one technical support to prepare for targeted and universal reviews
January – February ~ universal reviews
March – May ~ targeted and intensive reviews
June – following year ~ ongoing support until LEAs meet requirements and are implementing ESSA effectively J

20. Timeline 2019-20 and Beyond Selection Criteria
Notification to Selected LEAs
August
Training and Guidance
Technical Assistance and Planning
Sept. - October
LEAs Submit Evidence
Collaboratively build plans for spring onsite visits
Nov.
Document Review and Feedback
Dec. - January
Collaboratively determine follow up and follow through actions
Collaboratively build report
January - March
Onsite visits
March - May
Collaboratively build reports from onsite visits
May – June
Provide follow up supports (ongoing)
As Needed J:
Although some compliance indicators may require slightly different procedures, in general, the following represents the basic desk review protocol and timeline to be used:
August: CDE notifies LEAs of the specific indicators to be monitored as a desk review during the following school year, including identification of the timeline for district/LEA response, the evidence needed, and the process for submission.
September – October: CDE will provide training, technical assistance, and guidance on the year’s desk review indicators.
November: LEAs submit required documentation.
December - January: Program consultants review the submitted evidence and provide a response within 30 business days to the district/LEA regarding the evidence submitted and a determination of compliance. If appropriate, this notification will include recommended or required actions to be taken and the timeline for response by the LEA and/or a request for an onsite visit in January-March.
February - March: LEA responds to the actions to be taken with either additional or corrected evidence or a corrective action plan that has been collaboratively built with CDE. When changes are required, CDE will work in partnership with LEAs to design an action plan that is reasonable for the LEA to implement within an agreed upon timeline.
January-May: Follow-up onsite visits for LEAs identified through desk monitoring.

21. Training on Monitoring: Proposed Regional Networking Meetings Plans
Targeted workshops and interactive meetings at the end of Regional Networking Meetings (RNM)
Opportunity to connect with CDE representatives to prepare for upcoming monitoring events (universal, targeted, and intensive
Opportunity to hear from LEAs with exemplary practices
Opportunity to review, discuss, and plan follow-up actions based on monitoring results
Timeline
November-December, February, and April
Structure
Topical trainings in the morning
Afternoon workshops specific to monitoring J

22. Gathering Input Beyond CoP
Who else should have an opportunity to give feedback?
Should we do a survey? J

23. Vote on Names for Levels 2 and 3
Targeted Review
Targeted
Focused
Adaptive
Universal Plus
Intensive Review
Intensive
Focused Plus
Personalized
Tier III
Customized
Onsite N
Click here to vote.

24. We would love to hear your feedback on the plans and this training

25. We are in the process of contacting 2018-2019 LEAs
Next Steps
We are in the process of contacting LEAs
After phone call, a notification letter will be send to the LEA explaining what to expect of the type of monitoring
CDE will schedule planning calls with any selected for onsite visits
Trainings and technical assistance will be provided through the end of December
Webinars will be recorded and posted on CDE website for future viewing:
Webinar Recordings N

26. Upcoming Trainings Training Topic Date Times
Existing resources, where to find them, & how they connect to monitoring indicators
10/24/2018
1:00 - 3:30
Stakeholder engagement (implementation of Cross-Program Cons App Question)
10/29/2018
District and school public reporting requirements
11/6/2018
1:00 - 3:00
Title III parent notification requirement
11/15/2018
DMS System and submittal of evidence
11/27/2018
Parent involvement in planning and implementing programs under Titles I and III
12/4/2018
Office Hours (Q&A) on Monitoring Process (how to submit evidence, how to plan for onsite visit); will first go through F&Q and then open up lines for Qs
12/11/2018
1:00 - 2:00

27. Contacts
Joey Willett
(303)
DeLilah Collins
(303)
Nazie Mohajeri-Nelson
(303)