Welcome 2018 TA&D Program Area Meeting If your project is represented by more than one person, please distribute yourselves throughout the room. Meeting new people is a good thing! Office of Special Education Programs U.S. Department of Education July 23, 2018

OSEP TA&D Program Performance Measurement OSEP Project Directors’ Conference July 23-25, 2018

TA&D Program Performance Measures Annual Measures of New Service or New Product Quality Relevance Usefulness Efficiency – milestones completed within budget Long-tem Measures Implementation of evidence-based practices Promotion of effective models Findings are presented as a draft in September but Budget/OMB has until the following May to ask for special analyses or additional rationale statements. Findings aren’t considered final until May of the following calendar year after OSEP has provided responses to Budget/OMB’s questions. These findings became final in May 2016, although the data were collected spring/summer 2015.

Review for Annual Measures Science Expert Review Panel Panel receives the New Product and New Service Description Guides as well as electronic copies of the product and supporting materials for services Rates the quality of evidence-based products and services Stakeholder Expert Review Panel Rates the quality of policy-related products and services and the relevance and usefulness of all products and services

How is Quality Rated by the Panel? Percentage of Special Education Technical Assistance and Dissemination (TA&D) products and services deemed to be of high quality by an independent (Scientific) review panel. Substance: Does the product content or the content delivered through the service reflect evidence of conceptual soundness and quality, grounded in recent scientific evidence, legislation, policy, or accepted professional practice? Communication: Is the product content or the content delivered through the service presented in such a way so as to be clearly understood, as evidenced by being well-organized, free of editorial errors and appropriately formatted?

TA&D Program Performance Outcomes for Annual Measures

How is Relevance Rated by the Panel? Percentage of Special Education Technical Assistance and Dissemination (TA&D) Products and services deemed to be of high relevance by an independent (Stakeholder) review panel. Need: Does the product content or the content delivered through the service attempt to solve an important problem or deal with a critical issue? Pertinence: Does the product content or the content delivered through the service address a problem or issue recognized as important by the target audience(s)? Reach: To what extent is the product content or the content delivered through the service applicable to diverse segments of the target audience(s)?

TA&D Program Performance Outcomes for Annual Measures

How is Usefulness Rated by the Panel? Percentage of Special Education Technical Assistance and Dissemination (TA&D) Products and services deemed to be useful by an independent (Stakeholder) review panel. Ease: Does the product content or the content delivered through the service address a problem or issue in an easily understood way, with directions or guidance regarding how the content can be used to address the problem or issue? Suitability: Does the product or service provide the target audience(s) with information or resources that can be used again or in different ways to address the problem or issue?

TA&D Program Performance Outcomes for Annual Measures

Quality: Distribution by Score 2015 2016 2017 9.00 4 (11%) 1 (3%) 7 (25%) 8.00-8.99 13 (37%) 10 (33%) 6 (21%) 7.00-7.99 9 (26%) 7 (24%) 5 (18%) 6.00-6.99 5 (15%) 5 (17%) 3 (11%) 5.00-5.99 3 (10%) 4.00-4.99 4 (13%) 1 (4%) 3.00-3.99 2.00-2.99 1.00-1.99 0.00-0.99 TOTAL 35 30 28

Relevance: Distribution by Score 2015 2016 2017 9.00 6 (17%) 2 (7%) 8.00-8.99 17 (49%) 18 (60%) 3 (11%) 7.00-7.99 7 (20%) 5 (17%) 20 (71%) 6.00-6.99 4 (11%) 3 (10%) 5.00-5.99 1 (3%) 4.00-4.99 3.00-3.99 2.00-2.99 1.00-1.99 0.00-0.99 TOTAL 35 30 28

Usefulness: Distribution by Score 2015 2016 2017 9.00 1 (3%) 2 (7%) 5 (18%) 8.00-8.99 17 (48%) 15 (50%) 13 (47%) 7.00-7.99 12 (34%) 7 (24%) 4 (14%) 6.00-6.99 3 (9%) 4 (13%) 5.00-5.99 4.00-4.99 3.00-3.99 2.00-2.99 1.00-1.99 0.00-0.99 TOTAL 35 30 28

Efficiency Measure The cost efficiency of the Special Education Technical Assistance and Dissemination Program as measured by milestones achieved and funds drawn down in a reporting year. This measure applies to all 84.326 Centers in their second, third, fourth, and fifth years of funding, but not those in their first year or operating under a no cost extension. Year Actual 2017 93.2 (% milestones achieved) 118.6 (% funds drawn down) 2016 98.5 (% milestones achieved) 85.9 (% funds drawn down) 2015 98.1 (% milestones achieved) 79.3 (% funds drawn down)

Efficiency Measure The cost efficiency of the Special Education Technical Assistance and Dissemination Program as measured by milestones achieved and funds drawn down in a reporting year. This measure applies to all 84.326 Centers in their second, third, fourth, and fifth years of funding, but not those in their first year or operating under a no cost extension.

Long-Term Measure Percentage of states successfully promoting the implementation of EBP with local programs States and other recipients of Special Education Technical Assistance and Dissemination program services will implement scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities. (Long-term objective. Target areas: assessment; literacy; behavior; instructional strategies; early intervention; and inclusive practices)   Measure 1.1 of 1: The percentage of States receiving Special Education Technical Assistance and Dissemination services regarding scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities that successfully promote the implementation of those practices in school districts and service agencies. Year Target Actual 2017 87.0 100 2015 2013 86.0 83.3 2011 Set Baseline 85.7

Long-Term Measure Percentage of states successfully promoting the implementation of EBP with local programs States and other recipients of Special Education Technical Assistance and Dissemination program services will implement scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities. (Long-term objective. Target areas: assessment; literacy; behavior; instructional strategies; early intervention; and inclusive practices)   Measure 1.1 of 1: The percentage of States receiving Special Education Technical Assistance and Dissemination services regarding scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities that successfully promote the implementation of those practices in school districts and service agencies. *2011 Set Baseline

Our Partnership is the Key to Successful Performance

Thank You

Strengthening Capacity & Measuring Capacity Change State perspectives about systems context and how TA providers can help Rorie Fitzpatrick, WestEd/NCSI | OSEP TA&D Meeting | 07.23.18

(to improve outcomes for children with disabilities) Slide 2 Components related to successfully scaling and sustaining systems change (to improve outcomes for children with disabilities)

Managing human talent and other resources State agency capacity needed for systems change Managing human talent and other resources Navigating politics, opportunities and constraints Communicating & Collaborating with stakeholders and partners Leading with vision to inspire and sustain transformation

Leader / Staff Turnover = Big Impact on Capacity

Slide 5

NCSI is housed at WestEd and supported through the TA&D Network More information: Rorie Fitzpatrick | rfitzpa@wested.org | 415.205.3155 | ncsi.wested.org Slide 6

Strategies for Strengthening Capacity, and Sustaining and Scaling Systems Change Lynn Holdheide, American Institutes for Research 2018 OSEP Project Directors Conference

Sustainability Programs or services continue because they are valued, have buy-in and resources Sustainability Is neither accidental or last-minute Requires thoughtful, purposeful, and timely planning

Planning for Sustainability Is the substance of the work – its relevance, practicality, and value Is the vision shared among partners and the demonstrated impact collected and shared Is the way the sustainability plan is carried out, including strategies employed, alignments and collaboration strengthened, and the collection and reporting of impact Is related primarily to the leadership team and partners– mindset, influence, and relationships

GTL Center TA Approach

GTL Center TA Approach Micro-credentials offer an opportunity for professional development to be available to teachers anywhere, and at the time of their choosing. The MC is designed with resources for teacher learning. But if a group of teachers or a school decides to pursue face-to-face PD, that’s fine too. What makes micro-credentials different is that they assess whether a teacher has learned anything from professional development. That’s important. We spend upwards of $18 billion per year on PD. But there isn’t any solid evidence that something works. Critics say we are wasting money on PD that doesn’t affect teaching. MCs provide an answer to the question of whether teachers learned. MCs also allow teachers to demonstrate competencies that could lead to a new role in teaching, like teacher leadership, or getting a special certification.

CEEDAR TA Approach

Systems Change Rubric Team Membership, Leadership Support, & Partner Commitment/Engagement Leverage & Align Resources and Efforts Data-based Analysis, Prioritization, and Vision Setting Blueprint Development Blueprint Implementation & Accountability Successes and Impact Captured and Communicated

Achieving our Mission: Winning Aspirations States have systemically adopted practices and policies which propel the CEEDAR aspiration. States have strengthened plans, advanced progress, stimulated innovation, and extended their reach. States have committed to focused, actionable change, building momentum for the CEEDAR aspiration. CEEDAR has been a widespread catalyst that spurs actions to achieve the CEEDAR aspirations. Universal TA Services Targeted TA Services Intensive TA Services Innovation & Inquiry Incubator

Incubator Process is…. Designed to be as objective as possible State Blueprints, Progress Reports & System Change Rubrics CEEDAR Staff Reflections & Recommendations State-based Participant Feedback CEEDAR Center Expertise & Availability TA Service & Staffing Decision Designed to be as objective as possible Uses multiple data points (both qualitative and quantitative) to inform & validate TA service decisions Provides for consistency in TA services across states based upon state needs, level of engagement, and notable progress Provides CEEDAR staff a mechanism to reinforce state readiness criteria

Slide 10 Lynn Holdheide 615-308-2525 lholdheide@air.org 1000 Thomas Jefferson Street NW Washington, DC 20007 General Information: 202-403-5000 TTY: 887-334-3499 www.air.org

11/16/2018

A story of collaboration in the Peach State Network for Transforming Educator Preparation (NTEP Cohort 1) Collaboration for Effective Educator Development, Accountability and Reform (CEEDAR Cohort 2) Georgia is making leaps and bounds toward dismantling silos with many statewide efforts. To provide context to today’s discussion, Georgia was one of seven states invited to join the Network for Transforming Educator Preparation. That same year Georgia initiated an informal partnership with the CEEDAR Center. The following year GA was awarded a CEEDAR grant and become a member of CEEDAR’s Cohort 2. An example of silo-breaking efforts is the partnerships between the CEEDAR Center, the Georgia Department of Education, the Georgia Professional Standards Commission, University System of Georgia and six IHEs. This team collaborates to develop effective teachers and leaders who can improve Georgia’s future by graduating ALL students who are ready to learn, live, and lead.

Responding to the Need FOR: Powerful P20 partnerships that focus on continuous school improvement and student learning and growth for all Systems of support to ensure a seamless transition from pre-service to in-service and beyond On-going professional learning that supports learning for all: candidates, teachers, leaders and students Alignment and communication of numerous Georgia initiatives Meg and Ericia from the CEEDAR Center have been most helpful in listening to our needs, and sharing resources and support. In the early days, their strong facilitation and communication skills accelerated the work of the GA CEEDAR SLT. The GA team was intentional in creating a common language to develop a common understanding not just across programs but across our agencies. the GA team has been able to ensure that dots are connected to leverage—not layer—as we respond to the needs of GA P-20. 11/16/2018

Slide 4 The alignment committee is hard at work leveraging efforts such as the GA Equity and ESSA Plans, the Network for Transforming Teacher Education, and the State Strategic Improvement Plan. Meanwhile, the committee is also ensuring multilevel, consistent, and ongoing communication and collaboration within and beyond the SLT. They have already started outreach across the state to build buy-in and create structures to assist in broadening the SLT work and establishing sustainability of efforts. 11/16/2018

Sustaining and Scaling UP Examples Moving from three to six IHEs Engaging P12 in Georgia’s CEEDAR work through High Leverage Practices After many how to sustain and scale up conversations during GA SLT convenings a plan was developed to bring onboard more IHEs and to engage P-12 partners in this work. Guiding questions for these conversations included what was feasible, what needed support to be purposeful in the initial planning and how could we be strategic to make a BIG impact. Always keeping in mind the needs of Systems of support to ensure a seamless transition from pre-service to in-service and beyond and learning for all: candidates, teachers, leaders and students. We identified two structures that were already in place: Dean’s Conference (Georgia Educator Preparation Provider) and GA/s nine regional P-20 Collaboratives which both convene twice a year. GEPP – initial CEEDAR introduction through talking points and one-page quick reference flyer. Poster sessions to assess interest, initial follow-up to those who expressed interest, invitation to Chicago, CEEDAR 101 webinar in preparation for the Chicago convening, team time for new IHES was used to identify needs and a mentor for each IHE. Additionally, a just in time monthly support webinar was facilitated by Dr. Sheryl Moss who served as the mentor coordinator. Sheryl has done a phenomenal job in supporting the CEEDAR mentors from the original IHES. We believe the key to making this work scalable is to create systems that are clear and that have transparent processes. There needs to be a common language and a common understanding for the work that we are doing. This will allow us to develop a common message so when we are communicating with other universities, departments or agencies, we are all sharing the same message. That is how we can make this work scalable. GaDOE, USG, GaPSC and the CEEDAR Center have partnered to empower educators to maximize P20 student success Statewide convening to create an awareness for High-Leverage Practices in Special Education and share how HLPs align with a Multi-tiered System of Supports Kick-off for year-long P12 support 11/16/2018

We may not have it all together… But together we have it all Slide 6 We may not have it all together… But together we have it all 11/16/2018

Slide 7 State Contact Information Georgia Department of Education Dr. Karen J. Wyler Induction and IHEs Evaluation Specialist kwyler@doe.k12.ga.us Georgia Professional Standards Commission Dr. Angie Gant Director, Program Approval Angie.Gant@gapsc.com University System of Georgia Dr. Bob Michael Associate Vice Chancellor Educator Preparation and Policy Robert.Michael@usg.edu 11/16/2018