1. Building Evidence for School Improvement: Leveraging Networked Improvement Communities
January 30, 2019
Copyright © 2018 American Institutes for Research. All rights reserved.

2. (added from Insert tab, Header & Footer icon, Fixed Date and time) 9/28/2015
Welcome
[Cortney] Welcome people to the session and direct them to session materials before beginning (What specifically will we be using for this session and where can they find it?).
Presentation Title (added from Insert tab, Header & Footer icon)

3. Presenter Introductions
Cortney Rowland, Principal Technical Assistance Consultant, American Institutes for Research (AIR)
Amy Feygin, Senior Researcher, AIR
Kelly Burgess, Assistant Superintendent, Kannapolis City Schools (North Carolina)
[Cortney] Introduce speakers. Now, let’s find out a little about who is in the room today. Stand up/raise your hand if you are representing….
A state agency? A district? An IHE? A school – teacher, coach, school leadership, etc.?
Stand up if you primarily work in or have experience in an urban setting. A rural setting?
Stand up if you consider yourself a NIC guru. Been part of a NIC? Are new to NICs.
Stand up if you have at least five years experience in education. Ok, sit down if it’s 10 years or less experience. Sit down if it’s 15 years or less experience. Sit down if it’s 20 years or less. [Keep going until just a few people who are the most experienced are standing].

4. Session Objectives
Learn about strategies for conducting continuous improvement research and building networked improvement communities (NICs) to systematically test practices and innovations to address persistent problems of practice, and build local evidence of what works in school improvement.
Through practice scenarios, engage with tools and resources available to support states and districts interested in engaging in disciplined inquiry to develop evidence about what works (and does not work) in their context.
[Cortney]
This workshop will feature district leaders who have engaged in Plan-Do-Study-Act cycles within a NIC. Participants will learn from those experiences about opportunities to build evidence of what works for school improvement in their local context and create a culture of learning at all levels. Also, through practice scenarios, participants will learn about tools and resources to help them engage in authentic continuous improvement. Finally, participants in small groups will brainstorm problems of practice that could be solved through collaborative means of continuous improvement, like NICs, and share next steps.

5. Agenda Introductions, Objectives, and Agenda – 10 minutes
The Challenge and the Opportunity – 10 minutes
Improvement Science and NICs in Action: Beyond Accountability – 10 minutes
The Work of Kannapolis City Schools – 50 minutes
Q & A – 10 minutes
Closing and Resources – 10 minutes
[Cortney]

6. The Challenge and the Opportunity
[Cortney]
Networked improvement communities (NICs) provide states, districts, and schools with a disciplined approach to working together to solve persistent problems of education practice. Although educators have long engaged in continuous improvement, NICs offer the opportunity to accelerate their learning together as they test innovations addressing a common problem of practice.

7. Challenge
Schools and districts, particularly those that are low- performing or in high-need contexts, often struggle with persistent problems of practice related to teaching and learning.
The Every Student Succeeds Act (ESSA) requires low- performing schools to implement evidence-based interventions.
However, the evidence base in education can be limited and practices or strategies that have been developed and evaluated in one context may not apply in other contexts.
[Cortney]

8. Opportunity
In some cases, schools and districts have the option to identify what works in their contexts by generating real-time data on implementation and impact.
However, most districts are too small or do not have sufficient resources, personnel, or time to test interventions using rigorous and intensive research methods, such as a randomized controlled trial (RCT).
Improvement science and NICs may allow schools and districts to build a local evidence base about what works in their context to solve persistent problems of practice.
[Cortney]
Be specific about the opportunity – how can improvement science and NICs build an evidence base in school improvement? How can it be useful to states and districts?
Why is this important? It’s inherently tied to evidence-based improvement processes and identifying, implementing, and scaling evidence-based practices.

9. The Improvement Process
[Cortney]
Adapted from Associates in Process Improvement

10. Networked Improvement Community
(added from Insert tab, Header & Footer icon, Fixed Date and time) 9/28/2015
Networked Improvement Community
A NIC is a collaborative research partnership that uses the principles of improvement science within a group of organizations to learn from promising practices developed in each context and how they may be adapted to other contexts (Bryk, Gomez, Grunow, & LeMahieu, 2015).
[Cortney]
Presentation Title (added from Insert tab, Header & Footer icon)

11. “Rather than asking whether an ‘intervention works,’ a network improvement community asks, ‘What works, when, for whom, and under what sets of circumstances?’”
—Bryk, Gomez, Grunow, & LeMahieu, 2015

12. Improvement Science in Action: Beyond Accountability
[Amy]
Mention the report available on the app/conference website.

13. Why This Project?
During the past decade, states and districts have invested in developing systems to measure teacher performance.
In the best of these systems:
Educators know the expected teaching standards and receive timely, relevant, and actionable data about their professional practices.
Districts align supports and resources for improvement.
Staff monitor data about how instructional practices influence student learning.
Unfortunately, not all systems work this well.
Implementation often has focused on compliance rather than growth in teachers’ professional practice.
[Amy]

14. Project Overview
Beyond Accountability was initiated to address challenges related to how well teacher evaluation systems support improvements in teacher practice.
Summer 2016–Spring 2018
Four districts:
Kannapolis City Schools (North Carolina)
School District of Osceola County (Florida)
Jackson-Madison County School System (Tennessee)
Boone County Schools (Kentucky)
[Amy]

15. Project Driver Diagram
Evaluation system that improves teacher practice and instructional quality
High-quality data and feedback on instruction
Teacher engagement and trust
Aligned supports and professional development in schools and the district
Evaluator training
Specificity of rubric
Frequency of observations
Communication process
Curriculum/instructional materials
Coaching or peer support
Formal professional development
Evaluation policy
Aim
Secondary Drivers
Primary Drivers
[Amy]

16. Project Approach Within-district work Cross-district work
Ongoing Plan-Do- Study-Act (PDSA) cycles
Quarterly within-district meetings to share progress
Cross-district work
Community of practice
Three meetings over the course of the project
[Amy]
Make it clear that the cross-district component was the “NIC” and how it helps accelerate learning. Later, they will have a chance to think about how they might create a NIC or networked approach to learning in their own contexts.

17. Project Goals For participating districts:
Instructional quality improves.
Teacher satisfaction with opportunities or mechanisms to improve their practice increases.
District capacity to engage in improvement efforts increases.
For other districts and the education community:
Other districts learn about successful (or unsuccessful) strategies.
Other districts learn about methods used to develop, test, and refine strategies.
[Amy]

18. Kannapolis City Schools

19. First Steps
Protect time.
Get the right people around the table.

20. Adhere to the PDSA Process
Folder with protocol for conducting PDSA in a NIC

21. Project Driver Diagram
Evaluation system that improves teacher practice and instructional quality
High-quality data and feedback on instruction
Teacher engagement and trust
Aligned supports and professional development in schools and the district
Evaluator training
Specificity of rubric
Frequency of observations
Communication process
Curriculum/instructional materials
Coaching or peer support
Formal professional development
Evaluation policy
Aim
Secondary Drivers
Primary Drivers

22. Data Analysis Conduct self-assessment (Handout 1).
What do you notice about the tool?
Research-based
Best practices
Categories
Specific
Beliefs and perceptions
Analyze and synthesize a variety of data.

23. Problem of Practice and Root Causes
Teachers do not have sufficient opportunities to learn from one another’s instruction.
Root causes:
These opportunities to learn are not given the high priority needed.
There is no supportive structure for peer observation.

24. Root Cause Analysis
Use self-assessment results and data to identify a problem of practice and its root causes (Handout 5).
Scenario #1
There aren’t enough qualified teachers.
Employers are not connected to the schools.
Students do not participate in career planning.
Scenario #2
Teachers’ lessons are not aligned with math standards.
Elementary student engagement is low.
Parents do not support math instruction at home.
Posters have possible problems of practice for each scenario. Participants use post it notes to complete process on handout 5.

25. Create a Plan Determine a Strategy & Develop Theory of Action
Create a Plan for Implementation
Consider scale & sustainability
Develop Measures & Predictions
Practical Measures

26. Change Strategy 1
Pilot a peer observation structure to support teachers in learning from one another.
Participants:
Four teachers
One facilitator
Four observations conducted

27. Practical Measures Instruments: Observer reflection
Observation facilitator reflection
Instructional logs

28. Implementation and Scale
Think of a time you implemented a new process, resource, or tool that did NOT go as well as you had hoped.
What would you do differently in regards to scale of your initial implementation? What adjustments could have been made?

29. Change Strategy 2
Refine the peer observation process and implement it with prioritized teachers throughout the district.
Participants:
86 teachers
11 facilitators
27 observations
conducted

30. Findings: Observer Reflection
The support I received while participating in the peer observation process was tailored to my professional development needs.
Participating in the peer observation process was valuable.

31. Findings: Observation Facilitator Reflection
The peer observation process is valuable for peer observers.
The peer observation process is valuable for teachers being observed.

32. Peer Consultancy Protocol
Valuable protocol for NIC.
Can be used at several different times during the inquiry cycle.

33. Reflections on Conditions for Success and Sustaining the Work
Get comfortable being uncomfortable.
The work is more effective when it is collaborative.
Consider a variety of data sources to measure improvement and ensure they are aligned with the desired outcome.
Leadership must model the use of improvement science and be transparent.
Make the use of improvement science a priority.
Embed and sustain the work organically in your organization.
Mention how the work is funded.
Maybe allow participants to think through how they might do improvement science and/or create a NIC or a networked approach to learning in their own contexts.

34. Q&A
[Amy]

35. [Amy]

36. Closing and Resources

37. Tools and Resources
Beyond Accountability report and handouts: Improving-Teaching-Learning-October-2018.pdf Measuring for Improvement one-pager: Brief-Oct-2018.pdf NIC page on AIR website: improvement-communities-improve-educational-practice
[Amy or Cortney]
Pull up the NIC page on the AIR site to showcase resources, handouts, and projects.

38. (added from Insert tab, Header & Footer icon, Fixed Date and time) 9/28/2015
References
Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Cambridge, MA: Harvard Education Press.
Presentation Title (added from Insert tab, Header & Footer icon)

39. Cortney Rowland
1000 Thomas Jefferson St. NW
Washington, DC
General Information: