Results-based ON-SITE Monitoring for Selected Districts 2016-17 ESEA Directors Institute August 2016

Dr. Alyson F. Lerma Director of Monitoring Consolidated Planning & Monitoring Alyson.Lerma@tn.gov (615) 770-3871

Session Overview

Agenda CPM mission and commitment Overview of monitoring A new framework for 2016-17 On-site monitoring Selection method for identifying districts Phases of the on-site monitoring process On-site monitoring tool Resources Review and questions

Session Objectives Discuss principles and goals of monitoring Summarize monitoring process in 2015-16 Explain the new multi-tiered monitoring framework Outline the phases of the on-site monitoring Pre-visit On-site Visit Follow-up Review the updated monitoring tool Share resources Answer questions

CPM Mission & Commitment

CPM Mission To provide excellent support and service to LEAs and other stakeholders in the effective implementation of federal education programs designed to prepare students for post-secondary and career success

CPM Commitment CPM is committed to: working collaboratively with partners – both internal and external; communicating clearly with all stakeholders; providing consistent and timely feedback; demonstrating competency in effective ESEA and IDEA program implementation; ensuring compliance with federal and state regulations; supporting flexibility and innovation where opportunities exist; and focusing on what is best for the students of Tennessee.

Monitoring Overview

Monitoring Responsibility Federal requirements mandate that each state educational agency (SEA) oversee and monitor the implementation of compliant federal programs at least annually and provide performance reports. [EDGAR, 200.328] SEAs are also required to provide technical assistance (which can be informed by monitoring).

Characteristics of Successful Monitoring What Monitoring IS Focused on student outcomes Oversight to ensure compliance Opportunity to provide TA & collaborate with LEA partners Dialogue on program effectiveness based on many data sources Discussion of promising practices, needs, & unique circumstances of LEAs What Monitoring is NOT A “gotcha” A biased review A surprise Based on a single data source A static process

Guiding Principles Use of risk-based analysis Two LEAs chosen randomly Monitoring for compliance of IDEA and ESEA programs Focus on student outcomes and program effectiveness Collaboration within TDOE, with districts and stakeholders Clear communication Standardized protocols Dynamic process

Goals Provide an accurate, comprehensive review Highlight successful initiatives and exemplary practices Allow opportunities for partners to request assistance Engage in meaningful and open dialogue Provide individualized follow-up assistance Reduce the number of findings of non-compliance Empower districts and schools

Review of 2015-16 Monitoring Risk analysis to determine which LEAs would be monitored More than 60+ factors related to: prior findings, staff turnover, OCR findings, amount of federal funds received, fiscal issues, audit findings, IDEA complaint findings, and more. On-site monitoring of 20 LEAs in 2015-16: Seventeen were programmatic OR programmatic and fiscal Three were fiscal On-site monitoring visits to all large-urbans

Feedback from 2015-16 Monitoring Online survey link emailed to all participants 130 TOTAL responses 61 District-level responses 69 School-level responses Feedback from TDOE divisions, district and school partners, CPM Advisory Council, and M.A.S.S.

A New Framework for 2016-17

Three Tier Monitoring Process 10% of LEAs On-site monitoring 10% of LEAs Desktop monitoring 80% of LEAs Self-assessment

Framework of Monitoring Processes Self-assessment NEW 80% of LEAs will use a protocol to assess their ESEA & IDEA programs CPM Monitoring Coordinator will review all submissions & report on patterns, TA needs for upcoming year Will NOT result in corrective actions* Desktop NEW Two CPM consultants will review the submissions of required paperwork for monitoring an additional 10% of LEAs Regional consultants who support monitored LEAs will provide TA, but not participate in monitoring Can result in findings of non- compliance On-site Teams will visit 10% of LEAs on-site to conduct monitoring* Large urbans will rotate each year Increased standardization Regional consultants who support monitored LEAs will provide TA, but not participate in monitoring Can result in findings of non- compliance

Rationale for 2016-17 Changes Implement feedback from a variety of stakeholders Provide oversight of and tailored assistance to more LEAs Support LEAs demonstrating low to no risk in monitoring Utilize limited resources more effectively and efficiently Closer alignment to Tennessee Succeeds Further improve and streamline the process Decrease number of findings of non-compliance Assist in the transition to ESSA

On-site Monitoring in 2016-17

On-site Monitoring Use of risk analysis from 2014-15 District programs demonstrating significant risk 26 districts total in 2016-17; of those, 7 are fiscal only Random selections & focus monitoring Includes programmatic only, fiscal only, or both Monitoring cycle is Sept.-March *Large urbans: will be monitored each year but rotate between on-site & desktop

Phases of On-site Monitoring Process Pre-Visit Visit Post-visit & follow-up

Phase 1: Pre-Visit The monitoring team will: schedule and conduct a conference call with district leaders email student IDs (2 per school) to IDEA director review information available in ePlan, EIS, state report Approx. 6 weeks before the visit The CPM regional consultant will visit in person to provide assistance & support.

Phase 1: Pre-Visit The district will: review the monitoring tool (on ePlan, TDOE Resources); upload all required evidence (Appendix A) to ePlan; LEA Document Library; 2017; Results-based Monitoring; On-site Monitoring; Required Evidence collect and prepare required evidence for on-site review (Appendix A); participate in a conference call with monitoring team submit agenda draft to monitoring team and schedule meetings with district and school personnel work with schools selected for monitoring to prepare for visit

Phase 2: On-site Visit Duration of 3-5 days (depending on district size) Begins with a meeting with district leadership Interview central office staff and leadership team SCHOOL-LEVEL: Interview leadership team, staff, and parents; classroom visits; review delivery of services and implementation of IEPs Review district and school-level documentation Visit approximately 10% of Title I schools, including non-public, and charter (if applicable) Exit meeting with district leaders to review findings

Phase 3: Post-visit & Follow-up Findings are shared with district leaders at exit meeting Follow-up with CPM regional consultant and other TDOE staff for technical assistance and support Official email from TDOE with final monitoring report within 10 business days (after exit meeting) Completed report and signature pages posted to ePlan Official status letter mailed, emailed, and posted to ePlan Survey link emailed to monitoring participants

Results-based Monitoring Tool Available on ePlan TDOE Resources; Monitoring – ESEA/IDEA/Fiscal; 2016-17; On-site Focuses on outcomes Includes ESEA & IDEA compliance areas Organized around levers that impact student achievement

Sections of the Monitoring Tool Introduction and District & School Information Quality Leadership (District & school levels) Effective Educators Instructional Practices Climate & Culture Family & Community Engagement Additional Areas Appendix A: Evidence & Documentation Monitoring Signatures Practices, Recommendations, & Corrective Action Items

Refinements to Monitoring Tool Comprehensive step-by-step instructions in Introduction Formatting change Yes/No to document compliance Minimum open-ended questions Additional notes Combined some sections Removed duplication Expanded some sections

Refinements to Monitoring Tool ALL required evidence is listed in Appendix A

Resources & Support

Resources ePlan: www.eplan.tn.gov CPM staff Coordinated Spending Guide TDOE Resources CPM staff Monitoring coordinator & director of monitoring Project directors Regional consultants Office of Local Finance (for fiscal or joint fiscal monitoring) Coordinated Spending Guide

Contacts & Support Carissa Sacchetti Alyson F. Lerma ESEA & IDEA Monitoring Coordinator Carissa.Sacchetti@tn.gov 615-917-3393 Alyson F. Lerma Director of Monitoring Alyson.Lerma@tn.gov 615-770-3871

Contacts & Support Vacant Renee.Palakovic@tn.gov (615) 253-3786 CPM & Fiscal Regional Consultants District Map STEWART MONTGOMERY ROBERTSON MACON CLAY PICKETT SUMNER CLAIBORNE SCOTT HANCOCK SULLIVAN CAMPBELL HAWKINS JOHNSON LAKE OBION TROUSDALE WEAKLEY HENRY JACKSON OVERTON FENTRESS HOUSTON CHEATHAM WILSON SMITH UNION GRAINGER HAMBLEN WASHINGTON CARTER DICKSON DAVIDSON BENTON PUTNAM MORGAN GREENE DYER ANDERSON JEFFERSON UNICOI GIBSON CARROLL HUMPHREYS DEKALB CUMBERLAND KNOX WHITE COCKE WILLIAMSON CROCKETT HICKMAN RUTHERFORD CANNON ROANE LAUDERDALE SEVIER HENDERSON WARREN VAN BUREN LOUDON BLOUNT HAYWOOD MADISON DECATUR PERRY MAURY BLEDSOE RHEA TIPTON LEWIS MARSHALL BEDFORD CHESTER COFFEE GRUNDY SEQUATCHIE MEIGS McMINN MONROE Central Time Zone Eastern Time Zone MOORE SHELBY LAWRENCE FAYETTE HARDEMAN McNAIRY HARDIN WAYNE GILES LINCOLN FRANKLIN MARION HAMILTON BRADLEY POLK 1 Vacant, CPM Cindy Smith, Fiscal Michelle Mansfield, CPM Brad Davis, Fiscal Bridgett Carwile, CPM Rob Mynhier, Fiscal Shalonda Meeks, CPM Brian Runion, Fiscal Deborah Thompson, CPM Dustin Winstead, Fiscal Henry LaFollette, CPM Jackie Broyles, Fiscal 2 3 4 5 6 Vacant Renee.Palakovic@tn.gov (615) 253-3786 Janet (Michelle) Mansfield Janet.Mansfield@tn.gov (731) 225-3627 Bridgett Carwile Bridgett.Carwile@tn.gov (615) 626-3466 Shalonda Meeks Shalonda.Meeks@tn.gov (615) 864-5471 Deborah Thompson Deborah.Thompson@tn.gov (615) 864-5162 Henry LaFollette Henry.LaFollette@tn.gov (615) 580-2038

Review & Questions

Review CPM mission and commitment Overview of monitoring A new framework for 2016-17 On-site monitoring Selection method for identifying districts Phases of the on-site monitoring process On-site monitoring tool Resources Review and questions

Questions?

Notifications can also be submitted electronically at: FRAUD, WASTE, or ABUSE Citizens and agencies are encouraged to report fraud, waste, or abuse in State and Local government. NOTICE: This agency is a recipient of taxpayer funding. If you observe an agency director or employee engaging in any activity which you consider to be illegal, improper or wasteful, please call the state Comptroller’s toll-free Hotline: 1-800-232-5454 Notifications can also be submitted electronically at: http://www.comptroller.tn.gov/hotline