A District’s Journey of Implementing Effective Data Use Practices August 2, 2017 Margie Johnson, Ed.D. Stephanie Wilkerson, Ph.D.

Today’s Outcomes Provide an overview of district’s journey toward implementing effective data use practices. Provide an overview of our approach and process to building a practitioner- researcher partnership around data use. Share lessons learned for implementing effective data use practices. Wilkerson & Johnson, 2017

Metropolitan Nashville Public Schools 42nd largest school district in the US 88,000 students; 6,000 teachers; 4,000 support staff Students speak 100 + different languages 160 buildings As with many other industries, educational organizations collect a wealth of data. Prior to age of technology, how was most data collected? Via paper and pencil…. When spreadsheet software hit the scene, many organizations begin collecting data digitally in a spreadsheet. Of course, each group collected the data they needed at the time. As a result.........

Conducting a Needs Assessment “Data” professional development documentation Data warehouse utilization report Data use research Just as effective data use practices suggest, multiple data sources were used when conducting the needs assessment. After compiling the needs assessment data, I synthesized it into a working model for driving my work of implementing effective data use practices throughout the district.

Data-Informed Decision Making Ecosystem Culture of Collaborative Inquiry Common Language Data Literacy & Analysis Data Access Johnson, 2016 After consolidating research from multiple sources and conducting a needs assessment where I used multple sources of data, including teacher interviews. Here is the ecosystem model I developed for data-informed decision making that guides my work. I have to share that I originally called it a framework, but that word never seemed to fit as a framework makes me think of hierarchy of steps that must be followed in order to ensure the goal is reached. Thinking back to my background as a secondary science teacher and after presenting at a business conference about building capacity for using data, I began thinking about ecosystems and how they have many parts that work together. There’s not one specific entry point, but the parts need to work together to balance the system. Of course, you will notice that the center part of the ecosystem is grounded in the research about data use because the heart of making sure data is understood is a culture of collaborative inquiry and the other circles work together to support that culture. Let’s take a deeper dive into the each component of the ecosystem. First, culture…. it is ever evolving and must be modeled by leadership, so one key thing I did with my work is to collaborate with others. I sought input along the way and worked to build collaborative partnerships within the district and even outside the district….

Data Access

Data Warehouse The area where the ecosystem was strong was Data Access. We had a data warehouse where users could now access the area. Where we needed additional support through a collaborative partnership was common language and data literacy and analysis…. Therefore, we became partners with a common cause with REL Appalachia…

Partners in a common cause TN partner 2014-2016 Data use workshops Teacher Data Use Survey Evaluation capacity building School data teams Principals Data coaches Teachers Central office staff Pilot schools Removing barriers to effective data use REL AP partnered with us with developing a common language and data literacy and analysis. Collaborative Inquiry

Common Language

Collaborative Inquiry Collaborative Inquiry is stakeholders working together to uncover and understand problems and to test out solutions together through rigorous use of data and reflective dialogue. Assumption: This process unleashes the resourcefulness of stakeholders to continuously improve learning. When I was conducting the needs assessment for data use, here’s the research definition we adopted. However, research many times needs to be translated into practice, so that’s where our REL partnership was a tremendous support for helping us implement effective data use practices. They provided us with a variety of technical assistance activities. First we had a workshop by Dr. Laura Lipton and Bruce Wellman around their book Got data? Now what? This book provided foundational knowledge about collaborative inquiry and how to lead it. Then, we engaged a stakeholders in helping use develop a common language for collaborative inquiry. N. Love, K.E. Stiles, S. Mundy, and K.DiRanna, 2008

Developing a Common Language Logic Models: Intended Outcomes Fishbone: Root Cause Analysis Tool: Teacher Data Use Survey Innovation Configuration Map: Practices Margie When MNPS began the partnership with REL AP in 2014, we first explored the question, “What are the barriers to using a collaborative inquiry approach for effective data use?” As described previously, we engaged 41 of our partnership members in a fishbone, or root-cause, analysis to identify the key barriers in the district. We then asked ourselves, “What outcomes would we expect to see if we implemented a collaborative inquiry approach to data use without any barriers?” To answer this question, we developed logic models that identified short, intermediate and long-term outcomes for implementing collaborative inquiry. We also identified a need to create a common language and measure of teacher data use practices in MNPS, and so Dr. Jeff Wayman led a team of researchers, including Stephanie, in developing the Teacher Data Use survey. [Margie elaborates more here] Lastly, we delved into defining the collaborative data use practices that would need to be in place to achieve intended outcomes. We developed an Innovation Configuration Map to define what collaborative inquiry looks like in action.

MNPS IC Map for Collaborative Inquiry

MNPS Collaborative Inquiry Collaborative Inquiry is a data-based team process that consciously uses the collaborative learning cycle and the qualities of effective groups (fostering a culture of trust, maintaining a clear focus, taking collective responsibility and data-informed decision-making). MNPS Collaborative Inquiry Community of Practice

Data Literacy & Analysis

Data Literacy & Analysis MNPS had some supports for data literacy and analysis including employing district data coaches and linking data guide within the data warehouse. If you recall, one barrier identified was walking the walk. While leading implementation of effective data use, it was imperative that we continued to build our data literacy and analysis capacity. That’s another place where REL AP came to the rescue by providing us with a variety of technical assistance activities around evaluation. I’ll turn it over to Stephanie Wilkerson to share how she partnered with us to build our capacity in evaluation……. Rankin, 2015

Building Capacity for Evaluation: Our approach Begin with the end in mind Engage key stakeholders and users in the evaluation process Be systematic and pragmatic Be transparent 1. Identified information needs and priorities of stakeholders and how they would use evaluation findings. This helped to identify key evaluation questions. Intended use by intended users guided all aspects of the evaluation 2. Worked collaboratively to build capacity to sustain evaluation activities – participatory approach, multiple voices represented 3. Started slow to run fast: began with root cause analysis, then logic models, then aligned evaluation questions with intended outcomes, choose feasible and realistic qualitative and quantitative methods that practitioners could implement 4. Made information about evaluation activities and findings accessible at anytime. Anyone who wanted to be involved could participate. Kept central office informed of all partnership activities.

Our Process Reported evaluation findings Developed an evaluation plan Implemented the evaluation plan Reported evaluation findings

Our Process Developed an evaluation plan Practitioners created purpose statement Assigned tasks and responsibilities

Our Process Implemented the evaluation plan Developed an evaluation plan Practitioners created purpose statement Assigned tasks and responsibilities Implemented the evaluation plan Jointly developed instruments PD on instrument administration (training, modeling)

Our Process Reported evaluation findings Developed an evaluation plan Practitioners created purpose statement Assigned tasks and responsibilities Implemented the evaluation plan Jointly developed instruments PD on instrument administration (training, modeling) Reported evaluation findings Full report 3-page executive summary Infographic

Lessons Learned Establish shared agreements for the partnership Honor commitments Work small, share big Scaffold support for data collection Design evaluation tools for sustainability Use evaluation data to inform implementation

Establish shared agreements Practitioners’ Role Context knowledge Data access Relevance and utility of findings Researchers’ Role Technical skills and expertise Training Time to lead activities Improving data use & student learning Establishing shared agreements involves determining how researchers and practitioners will join in the work together. This includes discussing roles and responsibilities around a shared goal and purpose.

Honor Commitments Tenable timelines for evaluation activities Respect competing district and school priorities Central office support Honoring commitments is about researchers and practitioners honoring the partnership, the activities designed to accomplish partnership goals and the timelines to do it in. This means setting tenable timelines for evaluation activities that respect the commitments practitioners have to other competing district and school priorities. To sustain the partnership it was essential to have the commitment of time and resources from the district’s central office. Without MNPS honoring its commitment to the partnership we would not have been able to make the progress we did over the three years.

Work Small Share Big Use various communication methods Deeper, hands-on professional learning, instrument development and data collection Use varied communication channels to reach everyone from teachers to district leadership Feedback, piloting, buy-in

MNPS Collaborative Inquiry Toolkit www.mnpscollaboration.org

Scaffold Support for Data Collection Sharing research-based best practices for data collection Modeling data collection techniques Allow time to debrief, ask questions, and build inter-rater agreement

Design Evaluation Tools for Sustainability Data collection and reporting templates Summary statistics Graphic displays Report organization and writing Infographics

Data displays and dashboards

Use Data to Inform Implementation Monitor implementation progress Provided targeted differentiated support Promote continuous learning and reflection on practice Engage all stakeholders in defining and refining what implementation looks like

Wrap-Up

Reflection What might be some ideas you take from this session to implement in your organization and/or share with others?

Stephanie Wilkerson, Ph.D. Contact Information Margie L. Johnson, Ed.D. margie.johnson@mnps.org Twitter: @MargieLJohnson3 www.mnpscollaboration.org Stephanie Wilkerson, Ph.D. stephanie@magnoliaconsulting.org magnoliaconsulting.org Margie & Jenny Q&A Any questions?

References Johnson, M. (2016). Experience from the field. In J. Rankin, How to make data work: A guide for educational leaders (pp. 171). New York City, NY: Routledge Love, N., Stiles, K.E., Mundy, S., & DiRanna, K. (2009). The data coach’s guide to improving learning for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin. Rankin, J. (2016). Over the counter data website. Retrieved from https://overthecounterdata.com/ Wilkerson, S. & Johnson, M. (2017 April). Partners in a common cause. The Learning Professional, 38(2). https://learningforward.org/publications/jsd/jsd-blog/jsd/2017/04/10/the- learning-professional-april-2017