Data Wise Improvement Process Module 5 Examining Instruction Created by Prince George’s County Public Schools: Ebony Cross Shields, Felice N. DeSouza, Donna Drakeford, Rotunda Floyd-Cooper, Tasheka Green, Paula Harris, Rhonda Hawkins, Trina Hayes, David Rease, Jr., Anthony Sims, Jennifer Williams

Objectives for Module 5 ACE Habits of Mind Norms Protocols Participants will: Briefly revisit the Data Wise Improvement Process and the ACE Habits of Mind Learn the importance of examining instruction within the Data Wise Improvement Process Practice maintaining a relentless focus on evidence when discussing classroom data In the introduction module, teams/schools were given the opportunity to learn about Data Wise as an improvement process and the new way of thinking that is used when using data to drive inquiry and what that means for building a “data culture”. You engaged in tasks that covered an introduction and overview of the Data Wise Improvement Process, and an overview of the foundational thinking that drives the Data Wise process: ACE Habits of Mind Norms Protocols Today we will... (Read the objectives)

Objectives for Module 5 Participants will: Learn what a problem of practice is and practice identifying one Integrate the ACE Habits of Mind Continue to read the objectives to the participants.

Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice We will take a moment to revisit the Data Wise Improvement Process, ACE Habits of Mind and the Ladder of Inference. Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

Coherence Narrative

Revisiting DataWise Data Wise is… A school improvement PROCESS that organizes and brings coherence to the work of improvement. A specific PROCESS that facilitates intentional thinking and utilizing a more disciplined way of looking at data as a collaborative group. A PROCESS that helps all educators in all positions to learn how to analyze data in a manner that contributes to improved instruction and increased student learning. Facilitator Notes: Allow the participants to read the slide. Ask what they notice about the slide. If not mentioned, highlight that Data Wise is a process that is integrated into schools/systems to provide coherence to processes that influence practice and organizational learning. It is about bringing together a skilled team of educators that work together to implement a coherent instructional plan to identify the learning needs every student, and to meet those needs. There are parts of the Data Wise process that informs the all of our traditional meetings that we are accustomed to such as collaborative planning, data utilization, analyzing student work, etc.

What Data Wise IS What Data Wise IS NOT Inquiry - questions drive learning and next steps Declarative - stating what has to be done Collaborative - all staff have a voice in determining what actions to take and what questions to explore Directive - one person decides what happens in isolation Designed to create spaces for learning about instruction, assessment, and practice for everyone A process that assumes skill and knowledge exist without having evidence of intentional knowledge and skill development. Data Wise is a broader framework Before showing the slide, review the slide and allow the participants to share: What Data Wise is and is not? (3 minutes)

What Data Wise IS What Data Wise IS NOT Means of organizing and bringing coherence to staff members’ collective efforts at improvement A program to implement A means to help school leaders organize the work of school improvement around a process that has specific, manageable steps A compliance measure or a checklist to be completed Cyclical; recursive A linear process that doesn’t allow for assessment and adjustment A series of yearlong coherent inquiry meetings A singular meeting (collaborative planning, data utilization, analyzing student work, etc.) Data Wise is a broader framework Before showing the slide, review the slide and allow the participants to share: What Data Wise is and is not? (3 minutes)

The ACE Habits of Mind The ACE Habits of Mind are the foundation of Data Wise. At least as important as what you and your colleagues do is how you approach your shared work. Successful teams cultivate a disciplined way of working and thinking that we at the Data Wise Project call the ACE Habits of Mind. It’s about naming the intangibles.

The Ladder of Inference I take: actions   I draw: conclusions   I add: interpretation   I select: some data   The facilitator will describe the purpose of the ladder of inference. Describing the Ladder of Inference One of Data Wise’s favorite tools to help educators stay focused on evidence during data conversations is the “ladder of inference, developed by Chris Argyris. The ladder of Inference is a way of organizing the progression of thought from simply observing something out in the world to doing something about it. At the lowest rung of the ladder, you start by selecting some data to observe and describe. As you climb the ladder, you add interpretations, draw conclusions, and take action based on the data. We take action at a higher rung on the ladder, which are informed by your conclusions and interpretations. Staying lower on the ladder is when you learn the skill of analysis - here, people cite what they see and dissect the data (graphs, student work, classroom observations). As we cite what we see, we allow opportunities for groups to clarify misconceptions and see the data more objectively. (3 minutes) All Data

Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice We will take a moment to look at Step 5: Examine Instruction Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

Learner-Centered Problem Key Tasks within Step 5 Problem Of Practice Examine a wide range of instructional data Get clear about the purpose of observation Use protocols to stick to evidence Identify a problem of practice Learner-Centered Problem For each step, there are key tasks which provides evidence of the work being done in each step. The key tasks for step 5 examining instruction are: 5.1 Examine a wide range of instruction data 5.2 Get a clear about the purpose of observation 5.3 Use protocols to stick to evidence 5.4 Identify a problem of practice Step 5 doesn’t disappear just because you’ve done it once - we’re continuing our work of inquiry, to determine solutions to the problem of practice.

Step 5: Examining instruction to collect data and determine a Problem of Practice FOCUS Review the LCP and brainstorm hypotheses about the POP OBSERVE Collect evidence about the POP DEBRIEF share evidence and come to consensus on the POP ADJUST Collect ideas & strategies about how to address the POP FOLLOW UP Select an option to address the POP Linking Teaching & Learning Developing Skill in Observation p. 116 If the observation is going to lead to real lasting improvements in instruction, it is critical that observation be embedded in a formal protocol. Effective protocols for observing practice includes the following key elements: Focus, observe, debrief, adjust, and follow up. Develop a Shared Understanding of Effective Practice Analyze Current Practice Learner-Centered Problem (LCP) Problem of Practice (POP) p. 116

Agenda Stoplight Protocol Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice We will take a moment to look at what we will do within preparing to observe. Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

Revisiting the Learner-Centered Problem (LCP) A learner-centered problem is : Directly linked to the priority question Based on multiple forms of evidence found when digging into data Within the school’s control A statement about student learning, not a question Specific and Small The learner center problem is (read the bulleted list on the slide) Priority question: -generated collaboratively Focused on educational matters (not student related issues within our spheres of influence) Narrows the focus for inquiry Takes into account accessible resources Question formulation technique protocol (The Right Question Institute) may be used to help develop a priority

Focus Collaborate within your teams to brainstorm hunches about the learner-centered problem (LCP) exists. When we focus, the team reviews problems of learning and teaching, provides context for lesson(s) to be observed, and discusses how observers will focus their attention during the observation. Our task when we focus is to review the learner-centered problem revealed by the data and brainstorm hypotheses for why the learner-center problem exists. This will eventually lead us to defining the problem of practice. Linking Teaching and Learning pp. 111-114

LCP: Students have trouble using details in narrative writing Affinity Protocol LCP: Students have trouble using details in narrative writing (2 min) Work independently to brainstorm hunches about the learner-centered problem (1 min) Place all notes on chart paper (5 min) Sort and categorize sticky notes (7 min) Share categories The Learner Center Problem that has been provided for you for the purposes of the protocol we will engage in is: Students have trouble using details in narrative writing The Affinity Protocol allows teachers to brainstorm hypotheses about the learner-centered problem. If the learner-centered problem is “students have trouble using details in narrative writing,” consider reasons why students might be having trouble using details in narrative writing. Focus on what teacher do rather than the students. Example: I do not teach strategies about using details in narrative writing. (2 min) Work independently to brainstorm hunches about the learner-centered problem Place participants in small groups and give the following instructions. (1 min) Place all notes on chart paper (5 min ) Sort and categorize sticky notes (7 min) Share categories Have teams share their categories. Ask for comments and whether there were any patterns. Have a discussion about how the results can inform the team's ongoing work. Linking Teaching and Learning pp. 111-114

Preparing to Observe In order to effectively define the problem of practice (POP), we must first develop our skill in collecting evidence. When we observe the team views one or more classrooms, taking notes that capture details about what the teachers and students are saying and doing and what tasks students are asked to complete. If recommended during the focus meeting, observers may interact with students. Facilitator should reference any specific tools that will be utilized across the school for collecting observation evidence Our purpose for observing practice is to examine instruction so we can collect data about what the problem of practice might be. Our task when we observe is to collect evidence ideally from many classrooms in our school, about the problem of practice that we may need to address. During Step 5, teachers are working together to examine their instruction and determine a teacher-created problem of practice based on their learner-centered problem. This is not supervisory or evaluative. Linking Teaching and Learning pp. 111-114

How do we help each other climb up the ladder? Step 5: Examining instruction to collecting data to determine a Problem of Practice Judgmental Objectivity Descriptive Collecting Evidence How do we help each other climb up the ladder? Ask- “What’s the evidence that makes you say that?” or “What do you see?” Use the language of the ladder “That sounds up the ladder to me. What’s your evidence?” Ask the participants the following question What is the most important ACE habit of mind when we observe instruction? ANSWER: “Relentless focus on evidence” Review the ladder of inference – invite audience participation: “What is the Ladder of Inference?” We help each other to climb up the ladder by asking for the evidence and the observable data that we see. Develop the Skill of Observing Practice p. 116

Collecting Evidence p. 130 Click 1- Axes appear Judgmental Objectivity Descriptive Specific and Judgmental General and Judgmental Specific and Descriptive General and Descriptive Specific and Judgmental Students have trouble using details in narrative writing. Specific and Descriptive General and Descriptive p. 130 Click 1- Axes appear Click 2 – Objectivity/ Specificity Matrix appears. Facilitators should say: The objectivity/specificity matrix provides guidance about what type of evidence to collect during the observation, evidence should be both descriptive and specific, instead of general and judgmental. Click 3- Matrix Disappears Click 4- New Matrix with Specific and Descriptive Highlighted appears. Facilitators should say: When examining instruction we want to write down only the specific and descriptive evidence observed. We want to focus on the yellow highlighted quadrant of the matrix. Specific Specificity General Adapted from Learning Walkthrough Guide, MA Dept. of Elementary & Secondary Education. See Data Wise p. 130 for more details. p. 1

Practicing Evidence Collection Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice We will take a moment to practice collecting evidence. Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

A Descriptive Writing Experience Practicing Evidence Collection Please write down specific and descriptive notes based on what you observe in the following video Collaborate with your table group to identify the Learner-centered problem: A Descriptive Writing Experience Participants will review a video from the Teaching Channel entitled : A Descriptive Writing Experience (hyperlinked in this side) https://www.teachingchannel.org/videos/descriptive-writing-experience We will watch a video entitled A Descriptive Writing Experience. During the video, capture specific and descriptive evidence observed during instruction. After viewing the video and sharing the evidence, we will use this information to collaborate within teams to identify a leaner-centered problem. Linking Teaching and Learning pp. 111-114

Students have trouble using details in narrative writing Debrief What did you see? What patterns did you observe? How did what you saw relate to the learner-centered problem? Students have trouble using details in narrative writing Facilitator should read the guiding questions on the slides verbatim. Facilitator should allow for a few moments (5-7 min) to gain feedback from the audience.

Defining a Problem of Practice Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice We will take a moment to define the problem of practice. Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

Definition: Problem of Practice A problem of practice is: directly related to the learner-centered problem based on evidence found when examining instruction within our control a statement about practice (not a question) specific and small. Data Wise, p. 110 p. 110 As educators we are constantly solving problems which range from simple to complex. Many of the problems are too complicated to solve quickly. In order to solve the learner-centered problem identified in Step 4, we will need to understand the teaching practices within our school. As educators we can control the the internal factors which impact student’s learning, rather than the external factors. Therefore, the focus of the action plan is teaching. In order to allow the teachers to have a process and take responsibility for solving the problem, we must reframe the the learner-centered problem as a “problem of practice”. Solving the problem of practice means progressing toward the larger goals.

Developing a problem of practice Frame for all Problems of Practice: If the learner-centered problem is “Students have trouble using details in narrative writing then the problem of practice is as teachers we do not…” Facilitators should share that there is a frame for the development of a practice of practice. Problems of Practice are if, then statements Linking Teaching and Learning pp. 111-114

Adjust and Follow Up As part of the protocol for examining instruction we engage in the following final steps: Adjust and Follow up In the Adjust Phase: Teams carry out the next steps agreed upon during debriefing During the Follow Up- the team discusses what they learned during adjustments and plan for future work

Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice Stoplight Protocol At this time we will look at the Stoplight Protocol ACE within Step 5 Plus/Delta Protocol

5. Examining Instruction Examine a wide range of instructional data Get clear about the purpose of observation Use protocols to stick to evidence Identify a problem of practice Now that we have gained insight into Step 5 and the key tasks of each one, lets reflect upon your school teams current level of functioning with each of the four key tasks. We will do this through repeating the Coherence Protocol. Reflect on your coherence protocol thinking and where you placed the green, yellow, or red dot. (Participants should see their dots from the first time they completed the Coherence protocol at the Step level) Facilitator Note: Review the Coherence Protocol purpose and directions. Click to bring in each of the key tasks as the participants repeat the Coherence Protocol

Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice Stoplight Protocol At this time we will discuss how the ACE Habits of Mind ACE within Step 5 Plus/Delta Protocol

Action, Assessment, & Adjustment Phrase the problem of practice practice to focus on teachers and that teachers take ownership “As teachers, we do not…” p. 128 Cultivating the habit of shared commitment to action, assessment, and adjustment allows the problem of practice to be phrased in a way to ensure that the problem of practice is truly focused on teachers and that teachers are taking ownership.

Intentional Collaboration Look at videos of instruction and visit classrooms so teachers can practice taking observation notes to specifically describe what they see. Discussions after the observation focus on what the students were doing and saying p. 128 Cultivating the habit of intentional collaboration involves having colleagues giving one another rich descriptive information on their practice. Colleagues observe and analyze practice collaboratively challenges individual assumptions, allows everyone to notice different things, see the same things in a new way, and leads to better understanding of practice observed. Adhere to norms that were established when organizing for collaborative work to ensure productive and safe collaborative conversations.

Relentless Focus on Evidence To ensure effectiveness of conversations that occur post-observation, requires planning before entering a classroom It is important to collaborate to name the evidence to be collected during the observation that will help shed light on certain instructional practices This keeps the focus during debrief on the evidence that participants in the observation were asked to collect Facilitators may read this slide verbatim. Transitions/Animations are included in these slides

5. Examining Instruction Key Task Protocol/Learning Task Artifact/Evidence Examine a wide range of instructional data Staff members observed examining instruction, either on video, in person, or through other means Staff members observed holding one another accountable for maintaining a relentless focus on evidence Get clear about the purpose of observation Process for observing practice, (perhaps drawing on the process described in Key Elements of Observing Practice: A Data Wise DVD and Facilitator’s Guide Use protocols to stick to evidence Protocol for discussing data collected during classroom observations Identify a problem of practice Statement of problem of practice resulting from examining instruction As schools begin engaging in Data Wise, recording school improvement will be captured through the artifacts/evidence of actually engaging in doing the work within each step. The artifacts/evidence allow teams to maintain a relentless focus on evidence. Additionally, the artifacts/evidence provide a list of possible sources to support your claim about what stage of development your school is in with respect to a particular task.

Reflect on Step 5 Objectives Participants will: Briefly revisit the Data Wise Improvement Process and the ACE Habits of Mind Learn the importance of examining instruction within the Data Wise Improvement Process Practice maintaining a relentless focus on evidence when discussing classroom data Reflect on the objectives (on two slides) and on the essential question: What did you learn about examining instruction that you want to remember to discuss with the team? What did you learn about examining instruction that you want to remember to discuss with the team?

Reflect on Step 5 Objectives Participants will: Learn what a problem of practice is and practice identifying one Integrate the ACE Habits of Mind What did you learn about examining instruction that you want to remember to discuss with the team?

Future Learning Linked to Data Wise This is an overview of steps 3-7. The middle section identifies the outcome for each step and the last column identifies what the staff is responsible for. This should help staff take a look ahead toward future professional learning.

Agenda Revisiting Data Wise, ACE Habits of Mind, and the Ladder of Inference Overview of Step 5 Preparing to Observe Practicing Evidence Collection Defining a Problem of Practice Stoplight Protocol We will complete the Plus/Delta Protocol. ACE within Step 5 Plus/Delta Protocol

Plus /Delta Protocol What helped me to learn/interact with today’s meeting objectives? What changes are needed to help me to better learn/interact with today’s meeting objectives? Ask participants to list what went well about the meeting session (“Plusses”) and what they would have liked to change (“deltas”). We do not comment on pluses or deltas, but rather let them be a record of what happened. Effective facilitators use the last meeting’s plus/deltas to plan the next meeting, talking care to address the details.