1. Welcome TA&D Program Area Meeting If your project is represented by more than one person, please distribute yourselves throughout the room. Meeting new people is a good thing!
Office of Special Education Programs
U.S. Department of Education
July 23, 2018

2. OSEP TA&D Program Performance Measurement
OSEP Project Directors’ Conference July 23-25, 2018

3. TA&D Program Performance Measures
Annual Measures of New Service or New Product
Quality
Relevance
Usefulness
Efficiency – milestones completed within budget
Long-tem Measures
Implementation of evidence-based practices
Promotion of effective models
Findings are presented as a draft in September but Budget/OMB has until the following May to ask for special analyses or additional rationale statements. Findings aren’t considered final until May of the following calendar year after OSEP has provided responses to Budget/OMB’s questions. These findings became final in May 2016, although the data were collected spring/summer 2015.

4. Review for Annual Measures
Science Expert Review Panel
Panel receives the New Product and New Service Description Guides as well as electronic copies of the product and supporting materials for services
Rates the quality of evidence-based products and services
Stakeholder Expert Review Panel
Rates the quality of policy-related products and services and the relevance and usefulness of all products and services

5. How is Quality Rated by the Panel?
Percentage of Special Education Technical Assistance and Dissemination (TA&D) products and services deemed to be of high quality by an independent (Scientific) review panel.
Substance: Does the product content or the content delivered through the service reflect evidence of conceptual soundness and quality, grounded in recent scientific evidence, legislation, policy, or accepted professional practice?
Communication: Is the product content or the content delivered through the service presented in such a way so as to be clearly understood, as evidenced by being well-organized, free of editorial errors and appropriately formatted?

6. TA&D Program Performance Outcomes for Annual Measures

7. How is Relevance Rated by the Panel?
Percentage of Special Education Technical Assistance and Dissemination (TA&D) Products and services deemed to be of high relevance by an independent (Stakeholder) review panel.
Need: Does the product content or the content delivered through the service attempt to solve an important problem or deal with a critical issue?
Pertinence: Does the product content or the content delivered through the service address a problem or issue recognized as important by the target audience(s)?
Reach: To what extent is the product content or the content delivered through the service applicable to diverse segments of the target audience(s)?

8. TA&D Program Performance Outcomes for Annual Measures

9. How is Usefulness Rated by the Panel?
Percentage of Special Education Technical Assistance and Dissemination (TA&D) Products and services deemed to be useful by an independent (Stakeholder) review panel.
Ease: Does the product content or the content delivered through the service address a problem or issue in an easily understood way, with directions or guidance regarding how the content can be used to address the problem or issue?
Suitability: Does the product or service provide the target audience(s) with information or resources that can be used again or in different ways to address the problem or issue?

10. TA&D Program Performance Outcomes for Annual Measures

11. Quality: Distribution by Score
2015
2016
2017
9.00
4 (11%)
1 (3%)
7 (25%)
13 (37%)
10 (33%)
6 (21%)
9 (26%)
7 (24%)
5 (18%)
5 (15%)
5 (17%)
3 (11%)
3 (10%)
4 (13%)
1 (4%)
TOTAL 35 30 28

12. Relevance: Distribution by Score
2015
2016
2017
9.00
6 (17%)
2 (7%)
17 (49%)
18 (60%)
3 (11%)
7 (20%)
5 (17%)
20 (71%)
4 (11%)
3 (10%)
1 (3%)
TOTAL 35 30 28

13. Usefulness: Distribution by Score
2015
2016
2017
9.00
1 (3%)
2 (7%)
5 (18%)
17 (48%)
15 (50%)
13 (47%)
12 (34%)
7 (24%)
4 (14%)
3 (9%)
4 (13%)
TOTAL 35 30 28

14. Efficiency Measure
The cost efficiency of the Special Education Technical Assistance and Dissemination Program as measured by milestones achieved and funds drawn down in a reporting year.
This measure applies to all Centers in their second, third, fourth, and fifth years of funding, but not those in their first year or operating under a no cost extension.
Year
Actual
2017
93.2 (% milestones achieved)
118.6 (% funds drawn down)
2016
98.5 (% milestones achieved)
85.9 (% funds drawn down)
2015
98.1 (% milestones achieved)
79.3 (% funds drawn down)

15. Efficiency Measure
The cost efficiency of the Special Education Technical Assistance and Dissemination Program as measured by milestones achieved and funds drawn down in a reporting year.
This measure applies to all Centers in their second, third, fourth, and fifth years of funding, but not those in their first year or operating under a no cost extension.

16. Long-Term Measure
Percentage of states successfully promoting the implementation of EBP with local programs
States and other recipients of Special Education Technical Assistance and Dissemination program services will implement scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities. (Long-term objective. Target areas: assessment; literacy; behavior; instructional strategies; early intervention; and inclusive practices)
Measure 1.1 of 1: The percentage of States receiving Special Education Technical Assistance and Dissemination services regarding scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities that successfully promote the implementation of those practices in school districts and service agencies.
Year
Target
Actual
2017
87.0
100
2015
2013
86.0
83.3
2011
Set Baseline
85.7

17. Long-Term Measure
Percentage of states successfully promoting the implementation of EBP with local programs
States and other recipients of Special Education Technical Assistance and Dissemination program services will implement scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities. (Long-term objective. Target areas: assessment; literacy; behavior; instructional strategies; early intervention; and inclusive practices)
Measure 1.1 of 1: The percentage of States receiving Special Education Technical Assistance and Dissemination services regarding scientifically- or evidence-based practices for infants, toddlers, children and youth with disabilities that successfully promote the implementation of those practices in school districts and service agencies.
*2011 Set Baseline

18. Our Partnership is the Key to Successful Performance

19. Thank You

20. Strengthening Capacity & Measuring Capacity Change
State perspectives about systems context and how TA providers can help
Rorie Fitzpatrick, WestEd/NCSI | OSEP TA&D Meeting |

21. (to improve outcomes for children with disabilities)
Slide 2
Components related to successfully scaling and sustaining systems change
(to improve outcomes for children with disabilities)

22. Managing human talent and other resources
State agency capacity needed for systems change
Managing human talent and other resources
Navigating politics, opportunities and constraints
Communicating & Collaborating with stakeholders and partners
Leading with vision to inspire and sustain transformation

23. Leader / Staff Turnover = Big Impact on Capacity

24. Slide 5

25. NCSI is housed at WestEd and supported through the TA&D Network
More information: Rorie Fitzpatrick | | | ncsi.wested.org
Slide 6

26. Strategies for Strengthening Capacity, and Sustaining and Scaling Systems Change
Lynn Holdheide, American Institutes for Research
2018 OSEP Project Directors Conference

27. Sustainability
Programs or services continue because they are valued, have buy-in and resources
Sustainability
Is neither accidental
or last-minute
Requires thoughtful,
purposeful, and timely planning

28. Planning for Sustainability
Is the substance of the work – its relevance, practicality, and value
Is the vision shared among partners and the demonstrated impact collected and shared
Is the way the sustainability plan is carried out, including strategies employed, alignments and collaboration strengthened, and the collection and reporting of impact
Is related primarily to the leadership team and partners– mindset, influence, and relationships

29. GTL Center TA Approach

30. GTL Center TA Approach
Micro-credentials offer an opportunity for professional development to be available to teachers anywhere, and at the time of their choosing.
The MC is designed with resources for teacher learning. But if a group of teachers or a school decides to pursue face-to-face PD, that’s fine too.
What makes micro-credentials different is that they assess whether a teacher has learned anything from professional development. That’s important. We spend upwards of $18 billion per year on PD. But there isn’t any solid evidence that something works. Critics say we are wasting money on PD that doesn’t affect teaching. MCs provide an answer to the question of whether teachers learned.
MCs also allow teachers to demonstrate competencies that could lead to a new role in teaching, like teacher leadership, or getting a special certification.

31. CEEDAR TA Approach

32. Systems Change Rubric
Team Membership, Leadership Support, & Partner Commitment/Engagement
Leverage & Align Resources and Efforts
Data-based Analysis, Prioritization, and Vision Setting
Blueprint Development
Blueprint Implementation & Accountability
Successes and Impact Captured and Communicated

33. Achieving our Mission: Winning Aspirations
States have systemically adopted practices and policies which propel the CEEDAR aspiration.
States have strengthened plans, advanced progress, stimulated innovation, and extended their reach.
States have committed to focused, actionable change, building momentum for the CEEDAR aspiration.
CEEDAR has been a widespread catalyst that spurs actions to achieve the CEEDAR aspirations.
Universal TA Services
Targeted TA Services
Intensive TA Services
Innovation & Inquiry
Incubator

34. Incubator Process is…. Designed to be as objective as possible
State Blueprints, Progress Reports & System Change Rubrics
CEEDAR Staff Reflections & Recommendations
State-based Participant Feedback
CEEDAR Center Expertise & Availability
TA Service & Staffing Decision
Designed to be as objective as possible
Uses multiple data points (both qualitative and quantitative) to inform & validate TA service decisions
Provides for consistency in TA services across states based upon state needs, level of engagement, and notable progress
Provides CEEDAR staff a mechanism to reinforce state readiness criteria

35. Slide 10 Lynn Holdheide 615-308-2525 lholdheide@air.org
1000 Thomas Jefferson Street NW
Washington, DC 20007
General Information:
TTY:

36. 11/16/2018

37. A story of collaboration in the Peach State
Network for Transforming Educator Preparation (NTEP Cohort 1)
Collaboration for Effective Educator Development, Accountability and Reform (CEEDAR Cohort 2)
Georgia is making leaps and bounds toward dismantling silos with many statewide efforts. To provide context to today’s discussion, Georgia was one of seven states invited to join the Network for Transforming Educator Preparation. That same year Georgia initiated an informal partnership with the CEEDAR Center. The following year GA was awarded a CEEDAR grant and become a member of CEEDAR’s Cohort 2. An example of silo-breaking efforts is the partnerships between the CEEDAR Center, the Georgia Department of Education, the Georgia Professional Standards Commission, University System of Georgia and six IHEs. This team collaborates to develop effective teachers and leaders who can improve Georgia’s future by graduating ALL students who are ready to learn, live, and lead.

38. Responding to the Need FOR:
Powerful P20 partnerships that focus on continuous school improvement and student learning and growth for all
Systems of support to ensure a seamless transition from pre-service to in-service and beyond
On-going professional learning that supports learning for all: candidates, teachers, leaders and students
Alignment and communication of numerous Georgia initiatives
Meg and Ericia from the CEEDAR Center have been most helpful in listening to our needs, and sharing resources and support. In the early days, their strong facilitation and communication skills accelerated the work of the GA CEEDAR SLT. The GA team was intentional in creating a common language to develop a common understanding not just across programs but across our agencies. the GA team has been able to ensure that dots are connected to leverage—not layer—as we respond to the needs of GA P-20.
11/16/2018

39. Slide 4
The alignment committee is hard at work leveraging efforts such as the GA Equity and ESSA Plans, the Network for Transforming Teacher Education, and the State Strategic Improvement Plan. Meanwhile, the committee is also ensuring multilevel, consistent, and ongoing communication and collaboration within and beyond the SLT. They have already started outreach across the state to build buy-in and create structures to assist in broadening the SLT work and establishing sustainability of efforts.
11/16/2018

40. Sustaining and Scaling UP Examples
Moving from three to six IHEs
Engaging P12 in Georgia’s CEEDAR work through High Leverage Practices
After many how to sustain and scale up conversations during GA SLT convenings a plan was developed to bring onboard more IHEs and to engage P-12 partners in this work.
Guiding questions for these conversations included what was feasible, what needed support to be purposeful in the initial planning and how could we be strategic to make a BIG impact. Always keeping in mind the needs of Systems of support to ensure a seamless transition from pre-service to in-service and beyond and learning for all: candidates, teachers, leaders and students.
We identified two structures that were already in place: Dean’s Conference (Georgia Educator Preparation Provider) and GA/s nine regional P-20 Collaboratives which both convene twice a year. GEPP – initial CEEDAR introduction through talking points and one-page quick reference flyer. Poster sessions to assess interest, initial follow-up to those who expressed interest, invitation to Chicago, CEEDAR 101 webinar in preparation for the Chicago convening, team time for new IHES was used to identify needs and a mentor for each IHE. Additionally, a just in time monthly support webinar was facilitated by Dr. Sheryl Moss who served as the mentor coordinator. Sheryl has done a phenomenal job in supporting the CEEDAR mentors from the original IHES.
We believe the key to making this work scalable is to create systems that are clear and that have transparent processes. There needs to be a common language and a common understanding for the work that we are doing. This will allow us to develop a common message so when we are communicating with other universities, departments or agencies, we are all sharing the same message. That is how we can make this work scalable.
GaDOE, USG, GaPSC and the CEEDAR Center have partnered to empower educators to maximize
P20 student success
Statewide convening to create an awareness for High-Leverage Practices in Special Education and share how HLPs align with a Multi-tiered System of Supports
Kick-off for year-long P12 support
11/16/2018

41. We may not have it all together… But together we have it all
Slide 6
We may not have it all together… But together we have it all
11/16/2018

42. Slide 7
State Contact Information
Georgia Department of Education
Dr. Karen J. Wyler
Induction and IHEs Evaluation Specialist
Georgia Professional Standards Commission
Dr. Angie Gant
Director, Program Approval
University System of Georgia
Dr. Bob Michael
Associate Vice Chancellor
Educator Preparation and Policy
11/16/2018