Establishing or Enhancing a Process and Approach for Data-Driven Inquiry Abby introduce Robb and Anne Robb Geier, Abby Winer & Anne Lucas Supporting Local Data Use for Program Improvement Meeting Day 1, November 4, 2015

Aims of this Session Provide an overview of the components of a high-quality process and approach for data-driven inquiry at the state and local level Provide examples of different approaches and models for data-driven inquiry Provide an opportunity to reflect on the strengths and areas for improvement in your state, your vision for data-driven inquiry and ideas for how to get there Abby During this session we aim to…

Data Use Theory of Action Sustained | Systematic | Child Centered Increased Child Outcomes Focused Results Policy Approach/ Model Data-Driven Actions Practice Settings Robb This is a ToA we use to describe the necessary conditions for data use (on the bottom in blue) to make data driven actions (in purple) that lead to outcomes (in green). Quality Capacity Culture Conditions for Data Use Accurate Timely Relevant Complete Data Integration Analysis Tools Skills Structures Commitment Beliefs Collaboration Leadership © 2015 Public Consulting Group.

DaSy Framework for Data Use Planning for data use DU1. Plan for data analysis, product development, and dissemination to address the needs of the users Analyzing and disseminating for data use DU2. Conduct data analysis activities and implement procedures to ensure the integrity of the data DU3. Prepare data products to promote understanding of the data and inform decision-making DU4. Disseminate data products to users to meet their needs Using and promoting capacity for data use DU5. State and local staff use data to inform decisions DU6. Support the use of data at state and local levels Abby Let’s revisit the DaSy framework for data use 3 Main sections/areas and 6 indicators http://dasycenter.org/resources/dasy-framework/data-use

A Data Leadership Model Create and articulate the vision for data use Set and model expectations Implement and uphold policies for data use in the district/region/program Identify data to be collected Manage data infrastructure and access Design meaningful data displays Select or develop models for inquiry and data use that will be used state-wide Model the inquiry process publicly Provide training and professional development to support district/regions, agencies, local administrators, local data teams, and providers/teachers in their use of data Use data to identify professional development needs Monitor the progress of the district/region toward achieving its vision for data use Establish the lines of communication necessary to share results and best practices Vision and Policy Management (DU1) Data Management (DU2, 3) Inquiry, Analysis, and Action (DU1, 4, 5 ) Professional Development (DU6) Monitoring and Communication (DU1-6) Functions of a District Data Team © Public Consulting Group The components of this model (and the slides that follow) also come from our work with school districts and state DOEs. We may want to swap this out for the DaSY Framework. Many of the components are similar, and we may want to pare down to just focus on a few to shape the afternoon. In general, the slides that follow include a pair for each component that elaborate on the piece with questions and examples. What matters is that people come into agreement in what the steps and the process is and what do we call those things. For monitoring and communication, review and evaluation is the final element of each quality indicator– reviewing the effectiveness of the data displays, data use for accountability, and professional development

Vision and Policy Management Setting the vision for how data are to be used. Setting and modeling expectations. Implementing and upholding the policies for data use across the organization. Robb

Vision and Policy Management Questions to Consider Actions to Take What is the current vision for data use in your organization? How does the district/region/program specifically support, through structures and policies, the implementation of that vision? Who needs to be part of the conversation? Meet with local administrators, coaches, and other local leaders to communicate the vision for how data are to be used. Review scheduling and common planning time to ensure that all local staff have the time to use data collaboratively. Robb Additional Questions: Is it a powerful vision that will improve children’s progress or outcomes? Where is improvement needed? Additional Actions Develop and communicate a organizational vision for data use

Data Management Identifying data to be collected. Managing data infrastructure and access. Designing meaningful data displays. Robb

Data Management Questions to Consider Actions to Take Do users have the data they need when they need it? How do we know? What feedback channels exist to help improve our data quality and reports? What specific steps can you take to improve how data is collected, managed, and reported? Ensure teacher/provider access to existing data and data analysis tools. Monitor data quality and provide documentation and ongoing training to data collection staff. Identify reports needed for particular data use needs. Robb Some types of reports might include student child roster reports (by caseload, program/agency, district/region) with historical data; reports to monitor key performance indicators for principals.

Inquiry, Analysis, & Action Selecting and implementing a model for data use. Developing focusing questions. Analyzing data to make decisions about practices/curriculum, staffing, resources, and professional development. Robb

Inquiry, Analysis, & Action Questions to Consider Actions to Take Does your state have a common approach to data-driven inquiry? What is it called? Is use of the model widespread? Why or why not? Who needs to be involved in a conversation about how your locals use inquiry? Analyze data at the state or local level to identify essential questions across schools/programs; e.g., “What factors are associated with better child outcomes” Coach local improvement teams to use data to set goals and develop action plans. Robb Other specific actions that can be taken: Use data to assess the efficacy of the region’s coaching; e.g., are our coaches working with the right teachers/providers on the right strategies based on children’s needs?

Approaches to Data-Driven Inquiry to Consider Add picture

Preparation – Plan to Succeed Define purpose and the issue Identify who needs to be involved Timelines Identify relevant data Identify relevant questions Generate hypotheses Evidence – Dig Into Data Analyze the data Develop methods and materials for displaying the data Inference – Interpret & Share With Others Share data materials Check support for hypotheses Connect inferences with root causes Action – Contribute to Success Celebrate success Develop & implement improvement plans Evaluate progress Abby Key Components in a Process for Supporting Local Data Use – Using the Evidence Inference Action Framework (which we’ve used in ECO data workshops and trainings) Preparation – Plan to succeed Identify who needs to be involved = identify team members Need someone who… Evidence- Digging into the data Inference – Interpreting and Sharing Data with Others Action – Next Steps to Contribute to Success Use group to make inference and determine the real actions – using stakeholder to finish that inference and decide what appropriate action should be taken EIA gets repeated in cycle, but preparation occurs at beginning and gets revised occasionally (as needed, not part of regular cycle) Depending on state infrastructure and system, locals may be ones completing process and development materials or state may be the one developing materials – completing preparation and evidence phases and then involving/sharing with locals for inference and action

Abby http://marylandlearninglinks.org/277829

Cycle of Inquiry and Action Identify an Issue And formulate questions to help define it 2. Understand the Issue By analyzing data and refining questions 3. Diagnose the Cause Through deeper analysis, observation, and best practice research 4. Plan Action By setting measurable goals and articulating strategies 5. Take Action While monitoring data to make adjustments 6. Evaluate And reassess our situation 1 6 2 The Cycle of Inquiry and Action 5 3 Robb This example comes from NJ 4 www.pcgeducation.com | NJ SMART http://www.nj.gov/education/njsmart/

Cycle of Inquiry Cycle of Inquiry Focusing Questions Results © 2008 Public Consulting Group. Used with permission. If we ask the right questions Results Information …Then we will see increased student learning and improved student achievement …And collect and analyze data to help us answer those questions Cycle of Inquiry Robb This example comes from MA Instructions: Talk through the slide and the key messages. Stress the “If … then” logic. Give examples where appropriate. Especially needed in the “Action” phase. Key Messages: A disciplined application of this kind of data-driven approach builds a district and school environment that is focused on continuous improvement grounded in evidence. Data collection and analysis is useless if appropriate action based on what the data say isn’t taken. Action Knowledge …And apply that meaning to improve systems, programs, and classroom instruction …And use current research and shared experience to make meaning of those data

Professional Development Providing training and professional development to support data use by: Districts/regions/programs Local Administrators Local Leadership Teams Teachers/providers Using data to differentiate professional development according to need. Robb

Professional Development Questions to Consider Actions to Take What kinds of professional development about data use have been provided? How has this professional development been differentiated? How is assistance deployed to support issues identified through using data? Identify and train staff to serve as facilitators of professional learning communities in high need districts/programs. Use child and family outcome data and information collected from coaching/supervision to target embedded professional development where it is most needed. Robb Other Questions to Consider What structures and teams exist in your district/program/agency? How is professional development delivered through each of these? Who needs to be involved in conversations about professional development in your district/program? Other Actions to Take Conduct targeted professional development for new providers and local administrators in local data use protocols to ensure effective use.

Communication & Monitoring Communicating local-level focusing questions and findings. Monitoring local-level use of data. Collecting and sharing results. Robb

Communication & Monitoring Questions to Consider Actions to Take Establish quarterly roundtable meetings with leaders to review key performance indicators for program improvement and child and family outcomes. Collect child and family challenges being addressed by teams to identify patterns across programs and ways to provide assistance. How do your locals currently foster communication across districts/regions/programs about efforts to improve child and family outcomes? How do local administrators currently monitor the effectiveness of the supports they provide? Robb What steps can you take to improve both of the first two questions? How can you make use of available technology to help with these activities? Who would need to be involved in conversations about policies, technology, time, and other resources to support data use?

Culture: the set of shared attitudes, values, goals, and practices that characterizes an institution or organization Robb Let’s look at our definition again. What does it take to align attitudes, values, and goals in an organization? How do we get people to faithfully implement practices, every time, every day? http://www.merriam-webster.com/dictionary/culture

State Sharing In relation to the components discussed, what is a strength in your state? In relation to the components discussed, what is an area you are focusing on for improvement in your state? Abby Kyla from VA and Emily from PA will share a little about their states – an area that they believe is a strength and an area they are focusing on for improvement with regards to data leadership and data-driven inquiry

Tools and Resources PCG White Paper –Theory of Action for data use culture http://www.publicconsultinggroup.com/education2/library/white_papers/EDU_Data%20Driven%20District_Practical%20Ideas_White_Paper_2013.pdf Critical questions for EI/ECSE http://dasycenter.org/critical-questions-about-early-intervention-and-early-childhood-special-education/ Guidance Table for Analyzing Child Outcomes for Program Improvement http://ectacenter.org/eco/assets/pdfs/AnalyzingChildOutcomesData-GuidanceTable.pdf Planning, Conducting & Documenting Data Analysis for Program Improvement http://dasycenter.org/planning-conducting-and-documenting-data-analysis-for-program-improvement/ Abby Just a few resources

State Examples Maryland TAP-IT model: http://marylandlearninglinks.org/277829 NJ SMART Ed Data System website: http://www.nj.gov/education/njsmart/ MA Dept. of Elem. & Secondary Ed. District Data Team Toolkit:http://www.mass.gov/edu/docs/ese/accountability/dart/district-data-inquiry.pdf Abby Links to state examples for those that are web-accessible

State Team Time 5-10 min. for personal reflection Think about where your state and/or program is now in relation to having a process and approach for data-driven inquiry 10-15 min. for state team sharing and reflection Then spend the next 25-30 min. Brainstorming new ideas and ways to improve process/exploration of alternative system processes Identifying your vision and plan for where to go/what to do next Anne Personal and state-team reflection on where we are in relation to having a consistent process and vision for data use Structure: Personal reflection State team reflection Brainstorming vision and plan for what’s next Share reflections with partner state Transitioning from an effort to collect and report data to an inquiry and action effort Came into meeting with high-level priority areas, has that shifted at all from what you’ve heard today? As wrapping up, identify someone to share reflections and plans for where they want to go next with partner state

Paired State Reflection & Sharing State Pairs: PA & NE, TX & NJ, KS & MA, VA & WI For the next 30 min. - Each state take turns sharing reflections and plans for where you want to go next What did you identify as strengths? Weaknesses? What did you identify as your vision and approach for data-driven inquiry? What do you need to do to get there? States will then report out on each other What did you hear was a strength in the other state? What did you hear were key next steps? Abby Spend the next 30 min. taking turns sharing reflections and key next steps with another state Each state should spend about 10 min. sharing, allowing time for the other state to respond and provide feedback

The contents of this presentation were developed under a grant from the U.S. Department of Education, # H373Z120002. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. Project Officers, Meredith Miceli and Richelle Davis. Instructions to presenters: This slide is to be included as the last slide in your deck but you are not expected to show it to the audience. Please be sure to delete these instructions from this slide’s notes page in your presentation.