1. Technical Assistance Partnerships in Action: Lessons Learned from Evaluation
Pat Mueller, EEC
David Merves, EEC
Vitaliy Shyyan, NCEO
OSEP Project Directors’ Conference,
August 2016

2. Purpose of Presentation To SHARE…
Evaluation Design
Logic Model
Instrumentation
Case Studies
Reporting Formats
Discuss Focus Questions

3. National Center on educational Outcomes Mission Statement
To support the development of inclusive assessment in a rapidly changing state (and consortium) assessment system environment in order to promote improved educational results for students with disabilities.
Collaboration, influence, and outcome mapping
Measure numbers and strengths of partnerships (e.g., frequency of contacts)
Indicate collaborative TA activities and collaborative activities not specific to TA (e.g., collaborator participates in a jointly sponsored event vs. collaborating on a TA activity)
Track state TA and identify when/who collaborated on the TA
Collect data from clients, state/consortium team leaders, case studies
Displaying the data – Network Analysis

4. EvALUATION DESIGN Evaluation Questions
Development of Theoretical Model
Evaluation Methodology
Triangulation of Data Sets
Collaboration, influence, and outcome mapping
Measure numbers and strengths of partnerships (e.g., frequency of contacts)
Indicate collaborative TA activities and collaborative activities not specific to TA (e.g., collaborator participates in a jointly sponsored event vs. collaborating on a TA activity)
Track state TA and identify when/who collaborated on the TA
Collect data from clients, state/consortium team leaders, case studies
Displaying the data – Network Analysis

5. NCEO Partner collaborative Logic model
Employment of common strategies and language
Development of common products, tools, and services
Non-duplication of efforts
Cost efficiency
Handout the Logic Model with NCEO & EEC logos

6. Instrumentation Internal TA tracking system
TA Documentation
Where in the World is NCEO?
Website linkage analysis
Surveys with key partners and clients
Interviews with key partners and clients

7. Case studIES Consortium Topical Focus State Collaborative
Multi-Case Report

8. Case studY Methodology
Selected Success Case Method (SCM) that aligned with goals & logic model:
What is happening?
What results are being achieved?
What is the value of the results?
How can the initiative be improved?
Conducted interviews with 3 partnership types:
Consortium
Topical Focus
State Collaborative
Written reports, briefs, & comparative study describe collaborative work:
Products
Benefits of process
Results
Impact

9. Case study findings PROCESS RESULTS IMPACT
Understanding of deliverables
Expert facilitation, leadership and responsiveness
Solicitation and openness to feedback
RESULTS
Relationships
Collaborative Presence
Collaborative TA
IMPACT
Effective Work with Partners
Enhanced Assessment Policies and Practices

10. Reporting formats Standard evaluation reports Infographics
Have hand out with logic model on 1 side and infographic on the other side

11. Discussion questions
What methods do Centers employ to evaluate their collaboration efforts, outcomes, and impacts?
What types of collaboration assist Centers in developing high-quality products, tools, and services?
What types of reporting formats do Centers use to communicate the efforts and effects of their work?

12. Contact information
Pat Mueller & David Merves
Evergreen Evaluation & Consulting, Inc.
Vitaliy Shyyan
National Center on Educational Outcomes