1. Step 5 Module: Examine Instruction
Continuing our Journey toward Becoming Data Wise
The facilitator will introduce him/herself and begin by introducing step 5 of Data Wise which is focused on Examining Instruction.
Welcome to the 5th module on the Data Wise Improvement Process. This session is designed to help us to focus together on the importance of our practice as we consider improving the outcomes of our students.

2. Revisiting our Norms Take an inquiry stance Assume positive intentions
Ground statements in evidence
Stick to protocol and ensure all voices are heard
Be here now
Start and end on time
As always we want to ensure that we revisit our norms to set the stage for our collaborative work so that everyone is clear on the rules for engagement. Let’s take a moment to quickly review the norms. These should be familiar to everyone by now...can everyone commit to these norms today? (allow wait time for affirmative responses) Great! I also want to encourage you to hold both us, as facilitators, as well as one another, accountable to these norms throughout the session. Before we move on, we would like to ask...are there any norms that you’d like to add in order to enhance our collaborative work time? (allow wait time and add any agreed upon norms)

3. Narrowing the Scope of Inquiry
focus area
□ Relates to instruction. □ Narrows scope of inquiry while remaining broad enough to be relevant to many/most staff members.
[Insert your focus area here]
priority question
□ Arises from a collaborative process. □ Relates to instruction. □ Further narrows scope of inquiry. □ Is actionable. □ Is genuinely intriguing to staff.
[Insert your priority question here]
learner-centered problem
□ Is directly related to Priority Question. □ Is based on multiple data sources. □ Is within our control. □ Is a statement, not a question. □ Is specific and small.
[Insert your learner-centered problem here]
problem of practice
□ Is directly related to Priority Question. □ Is based on multiple data sources. □ Is within our control. □ Is a statement, not a question. □ Is specific and small.
[Insert your problem of practice here]
action plan
□ States specifically what teachers will do to address the problem of practice. □ Contains one or more research-based, evidence –based, high-leverage instructional strategies. □ Assigns responsibility to specific people. □ Is time-bound.
[Insert your action plan here]
plan to assess progress
□ Clarifies evidence that would show whether action plan is addressed the learner-centered problem. □ Includes short-term, medium-term, and long-term data sources. □ Includes specific and measurable student learning goals.
[Insert your plan to assess progress here]
We are continuing the building of a “funnel” of sorts, narrowing our inquiry focus more and more as we move through each step of the process.

4. Reflecting on Where We’ve Been
Learner- Centered Problem
Priority Question
Let’s quickly revisit where we have been:
We started our work together focused on step 1- Organizing for collaborative work. This is where we set our norms, we focused on building strong teams, learning work style preferences etc. (click)
As a reminder of our work after that : In module 2, teams/schools began to understand multiple forms of assessments, how results are reported, and how the results are responsibly used to inform decisions that impact instruction. With continued practice, teams will build their capacity and increase comfort. (click)
In module 3, our leadership team decided assessment data to display, how to display those data clearly and effectively, and how to start conversations around the displayed data. The outcome of the work of Step 3 was the generation of a priority question. (Facilitator may consider asking participants what they remember most about the work of Step 3 just to check for understanding and remembrances of the school data story. Some key ideas from step 3 include: ¾ of work belongs to leadership team and is backstage work, the full staff comes back into the process at 3.4 Make Sense of Data and Generate a Priority Question, and Criteria for Priority Questions) (click)
So by this point we have identified a priority question (the outcome of Step 3) as well as a Learner Centered Problem (LCP-the outcome of step 4). This means we are very clear about what “the problem of learning is” for students. Step 5 has 4 key tasks: They are: (Read verbatim from the slide) A key concept in this step that you will continue to hear about consistently is TRIANGULATION. Can someone remind us what that means? (Allow participants to respond). This focus on triangulation remains important as we continue to keep a lens on our Data Wise Norms. In particular : Ground Statements in Evidence. (click)

5. Step 5: Examine Instruction
Now comes the hard work. In step 5 as we Examine Instruction we will consider the role our practice plays in the problem that we have identified for our learners. This is BIG! Research tells us that the single most influential factor on whether students learn is what and how WE teach! We will delve deeply into the implications for this truth in Step 5. (click)

6. The Aim of Our Step 5 Work
As is the case with every step of the DWIP, Step 5 has an ultimate outcome. The aim of our work in Step 5 is to identify a problem of practice. This is, specifically, the answer to the question: “What might I/we be doing that is contributing to our learners problem?”
This is a tough question to consider since as practitioners we have heard many times comments like “These kids don’t do homework! That’s the reason they can’t pass!” or Families with no resources can’t help these kids so, the kids can’t help themselves. Research, however tells us that we as teachers are more influential than any other factor on whether students learn or not. Nothing else. (Facilitators should pause to see participant response)
As a result, the work of identifying the problem of practice is amongst the most important work in the DWIP.
You may remember from our work in Step 4 that we shared this “mock” learner centered problem with you. (Read from slide). What could be some potential problems of practice linked to this learner centered problem?
(facilitators should wait for responses. If no responses reframe the question by asking the following)
What could we as teachers be doing (or not doing) in our practice that could be contributing to the struggle our students are having with Tier 2 academic vocabulary use in their writing?
Allow time for responses. Be careful to highlight that responses should be linked to practice of teachers (because this is the only thing we can control). Once teachers have shared some responses/possibilities (at least 3) then ask the question: What instructional data could be examined in order to uncover this? (Allow wait time for participants to respond) .

7. Step 5: Examine Instruction
Key Tasks
5.1 Examine a wide range of instructional data
5.2 Get clear on the purpose of observation
5.3 Use protocols to stick to evidence
5.4 Identify a problem of practice
Problem of Practice
In order to achieve the work of Step 5 we must engage with 4 key tasks.
(Facilitator reads verbatim from the slide.)
So let’s take a closer look at what this really means. Starting with 5.1

8. Identifying the Problem of Practice (PoP)
5.1 Examine a wide range of instructional data
LESSON PLANS
In order to identify a problem of Practice in this step, we will follow some of the same steps that we did in Step 4. The Key to identifying the PoP is to remember the importance of triangulation of data sources.
So what are some examples of sources of instructional data?
(Facilitator should allow for adequate wait time to respond)
Review of lesson plans, Observation of Instructional Delivery, and “Other Qualitative Data”. What do you think that data might include?
Teacher feedback is a very viable option! Hearing what the teachers say they think they could be contributing to the learner centered problem is a key but often overlooked source of data.
OTHER QUALITATIVE DATA
INSTRUCTIONAL DELIVERY

9. Why Triangulate Data Sources?
Triangulation allows us to:
Use findings from multiple data sources to confirm, refute, or illuminate what we learned through initial analysis
Draw on different assessments or on assessments implemented at different intervals
Creates space for use of Student Work as a key source of data
You’ll recall that in Step 4 we talked about the concept of triangulation. Remember we said:
We sometimes underestimate the power and importance of Triangulation as we analyze data for improvement. Triangulation is a key activity that allows us to:
Use findings from multiple sources that will either confirm, dispute or illuminate what we found in a preliminary data set
Look at a variety of assessments including short cycle assessments and
Prompts us to examine student work as part of our analysis to inform our decision making
Data Wise, p. 99

10. 5.2 Get clear on the purpose of observation
How many of you have had observations by your principals/supervisors? We all have right?
How many of you have experience with being observed by your peers? (Wait for responses)
What would you say the difference is? (Wait for responses)
Something that is important about step 5 is that the identification of the problem of practice is “self” identified. That is, teachers name the problem in their practice with guidance from leaders in buildings. The data (observational, lesson plan, and feedback) helps the teachers identify their problem of practice. This in non-evaluative in nature and it is a form of collaborative reflection. But what is important about this is that we must all be on the same page about what it is we are looking for. Specifically, if we are for example trying to identify a problem of practice linked to student’s inability to use Tier 2 vocabulary in their writing, the first thing we have to look for is evidence of the learner centered problem first. Then we are interested in uncovering what trends we see in what teachers are doing or not doing when the learner’s problem is present.
SO what are we looking for? 2 Things- (1) The presence of the LCP and (2) What teachers are doing or not doing when the problem that the learners experienced is present.
What are we looking for?

11. 5.3 Use protocols to stick to evidence
In order to ensure success in the Peer observation, teachers have to have a clearly outlined plan for what happens during the observation.
While we assume positive intentions, we don’t want to assume that “people know”. Outlining specific details of “Do’s and Don’ts” could make the difference between success and failure in the part of our Data Wise work.
What do I do when I am looking ?

12. 5.3 Use protocols to stick to evidence
Keep in mind that the work of Step 1 is an integral part of the success of the work of Step 5. What specific key tasks do you think could help this work? Why?
Facilitator should allow responses and encourage the use of the swoosh as a guide. The facilitator should hope to hear dialogue about 1.2 Build a strong system of teams, and 1.6 Acknowledge work style preferences as well as 1.5 linked to norms.
Lean on your learning from Step 1

13. 5.3. Use protocols to stick to evidence
Judgmental
Objectivity
Descriptive
Specific and Judgmental
General and Judgmental
Specific and Descriptive
General and Descriptive
Another useful tool for teachers as they work to get clear on the appropriate way to engage in peer observations is the Objectivity/Specificity Matrix. It will help them to get clear on the type of the feedback and statements that they should capture as they are visiting classrooms.
The Objectivity/Specificity matrix is divided into 4 quadrants. (Facilitator Names them starting from General and Descriptive and moving counter clockwise)
Specific Specificity General
Adapted from Learning Walkthrough Guide, MA Dept. of Elementary & Secondary Education. See Data Wise p. 130 for more details.

14. 5.3. Use protocols to stick to evidence
Judgmental
Objectivity
Descriptive
Specific and Judgmental
General and Judgmental
Specific and Descriptive
General and Descriptive
Partner Work:
Define each quadrant by providing an example of evidence that could be collected during instruction.
Facilitator states:
Obviously we want to aim to be in the Specific and Descriptive Quadrant. It is however, to ensure that we are aware what is meant by each of the quadrants to ensure we steer clear of the other three.
Facilitator assigns partner work as outlined on the screen. Participants are given 7 minutes to complete this task.
Specific Specificity General
Adapted from Learning Walkthrough Guide, MA Dept. of Elementary & Secondary Education. See Data Wise p. 130 for more details.

15. 5.3. Use protocols to stick to evidence
Judgmental
Objectivity
Descriptive
Specific and Judgmental
General and Judgmental
Specific and Descriptive
General and Descriptive
The statement notes an overview of what occurred in the classroom and includes a bias statement.
The statement notes exactly what occurred in the classroom and includes a bias statement
The statement notes exactly what occurred in the classroom, capturing actions/statements like a video camera.
The statement notes exactly what occurred in the classroom, capturing actions/statements like a video camera.
Facilitator asks teams to share their interpretation and an example of what such a statement might sound like. Start with the top left quadrant and move around to the final box Specific and Descriptive. Facilitator should read what is on the slide verbatim.
Keeping these terms in mind as teachers visit classrooms will be useful in enhancing a teams evidence based focus as they visit.
Specific Specificity General
Adapted from Learning Walkthrough Guide, MA Dept. of Elementary & Secondary Education. See Data Wise p. 130 for more details.

16. 5.4. Identify a Problem of Practice
The final key task in this step is 5.4 Identify a problem of practice. We do that by collaborating to converge upon a theme. There are a variety of protocols that could help us but one of the most helpful is the affinity protocol. This may be used multiple times throughout the work of step 5. If you remember we talked about all of the sources of instructional data that could be leveraged in this step. What were they again?
Participants should say: Lesson plans, observations, feedback from teachers.
In order to ensure that we ground our identification of a problem of practice in evidence, we must review multiple lesson plans, observe multiple classes, and get feedback from a variety of teachers. Based on the collected evidence we use the affinity protocol. Here’s how it works:
Facilitator passes out outline of the affinity protocol and reviews the steps verbatim. Facilitator asks if participants have questions as well as the question why do you think this protocol might be successful as we name a problem of practice.
Remember that this work is not done alone

17. 5.4 Identify a problem of practice
A Problem of Practice is:
directly related to the learner-centered problem
based on evidence found when examining instruction
within our control
a statement about practice (not a question)
specific and small.
Data Wise, p. 110
based on evidence found when examining
instruction
Finally now that we have identified some appropriate protocols to use as we visit classrooms and look at data It is important to keep in mind that a problem of practice is not just what we think! It is an evidenced based issue that we define using the steps outlined in the key tasks of step 5 (5.1, 5.2. and 5.3). More specifically, a Problem is defined as (read verbatim from the slide)

18. Reflecting on what we have learned about inquiry...
Now let’s take a look back at what we have learned as we prepare to move into the ACT phase and start Step 6.
There is a lot of important work that has occurred (or that occurs) in Steps 4 and 5. Who is responsible for this work?
Facilitator should hear various responses
The work of these two steps belong to many people. As a leader, you need to consider your role and the role of others in each of these steps as a last stitch effort to create clarity for all.
Use the handout linked to the image.

19. Plus /Delta Protocol
What helped me to learn/interact with today’s meeting objectives?
What changes are needed to help me to better learn/interact with future meeting objectives?
Now that we are at the conclusion of our session, as always it is important for us to be reflective about successes in our professional learning experiences as well as opportunities for growth. Please share your pluses and deltas with us so that we can use them to improve our future work with you.