Data Wise Improvement Process Digging Into Student Data Module 4 Digging Into Student Data Source PGCPS: Ebony Cross Shields, Felice N. DeSouza, Donna Drakeford, Rotunda Floyd-Cooper, Tasheka Green, Paula Harris, Rhonda Hawkins, Trina Hayes, David Rease, Jr., Anthony Sims, Jennifer Williams

Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Role of Step 4 Examine a wide range of student data Use protocols to stick to evidence Understanding Student Thinking Identify a learner-centered problem

Triangulating Data Sources Module 4 Objectives: Provide a review of the Data Wise Improvement Process, ACE Habits of Mind, and Ladder of Inference. Understand and apply principles of Step 4: Dig into Student Data: Examine a wide range of student data Use Protocols to stick to evidence Identify learner-centered problem Triangulating Data Sources Facilitator: In the introduction module, teams/schools were given the opportunity to learn about Data Wise as an improvement process and the new way of thinking that is used when using data to drive inquiry and what that means for building a “data culture”. Provide a review of the Data Wise Improvement Process, ACE Habits of Mind, and the Ladder of Inference You engaged in tasks that covered: Click The primary outcome for Step 2 is to guide staff in understanding multiple forms of assessments, how the results are reported, and how the results are responsibly used to inform decisions that impact instruction. With regular practice, teams will build their capacity and increase comfort. We will look closely at the key tasks for step 2: Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use Also, we will... Highlight the integration of ACE within Step 2 Facilitators Note: It is important to emphasize that this is the second step in the Prepare Phase. Prepare phase comes before Inquire because when teams use data to inquire they often have difficulty if they are not prepared for the challenging work inquiry that is through team thinking and problem solving; structured within specifically formatted meetings and the consistent use of protocols. Much more will be learned about Data Wise meetings and the use of Protocols in Data Wise.

What is Data Wise? Data Wise is… A school improvement PROCESS that organizes and brings coherence to the work of improvement. A specific PROCESS that facilitates intentional thinking and utilizing a more disciplined way of looking at data as a collaborative group. A PROCESS that helps all educators in all positions to learn how to analyze data in a manner that contributes to improved instruction and increased student learning. Facilitator Notes: Read bullets as they come in.. Click Allow the participants to read the slide. Ask what they notice about the slide. If not mentioned, highlight that Data Wise is a process that is integrated into schools/systems to provide coherence to processes that influence practice and organizational learning. It is about bringing together a skilled team of educators that work together to implement a coherent instructional plan to identify the learning needs of every student, and to meet those needs. There are parts of the Data Wise process that informs all of our traditional meetings that we are accustomed to such as collaborative planning, data utilization, analyzing student work, etc.

Building Background Knowledge Individual Reading Select one section of the text to read silently at your table Share Create a 2 to minute overview to share with the table group. Consider : Significance of section to Step 4 Key terms Wonderings/Questions Chart wonderings or questions that arise from the readings at the table. Activity 1 – Developing Foundational Knowledge (25-26 min) Purpose – This activity is designed to provide participants with dedicated time to read source material pertaining to Step 4.   Materials – Photocopy excerpt of Step 4 from Data Wise book Leveraging your Learning Worksheet Facilitators will provide participants with photo copy of chapter 4- Dig into Student Data Facilitator will identify sections of the text (handout) to assign to each participant. Silent reading – Participants will engage in silent reading. The Leveraging your Learning worksheet can us used to capture notes. (10 min) Prepare – Participants will prepare a 2-minute overview of key points (e.g. significance to Step 4, key terms, wonderings/questions) from the text selection to share with table participants. (5 min work time) Sharing – Each participant will share the 2 min overview with their table group.

Triangulating Data Sources STEP 4 Dig into Student Data 4.1 Examine a wide range of student data 4.2 Use protocols to stick to evidence 4.3 Identify a learner-centered problem Triangulating Data Sources Click Facilitator: The primary outcome for Step 2 is to guide staff in understanding multiple forms of assessments, how the results are reported, and how the results are responsibly used to inform decisions that impact instruction. With regular practice, teams will build their capacity and increase comfort with reading and interpreting data. We will look closely at the key tasks for step 2: Study How Results are Reported Review Skills Tested Learn the Principles of Responsible Data Use

Reflection Making the information Relevant (5 min) Select a partner at your table Each person, recalls (1) problem of understanding something or problematic skill area (for yourself or class) where a change in instruction significantly increased learning or performance Share your reflection with you partner Activity 2 – Reflection Activity, Making the information Relevant (5 min) Purpose – This activity is designed to provide participants with an opportunity to personalize an experience of a learning problem and its resolution to connect the relevance learning challenges and adjustments to instructional practice.   Participants select a partner at their table Each partner, reflects on one prior experience of a problem of understanding a concept or problematic skill area (for yourself or class) where a change in instruction significantly improved learning or performance (2.5 min) Each partner briefly shares the reflection with their partner (2.5 min)

What is a “Learner Centered Problem” A problem of understanding or skill that underlies students’ performance on assessments The problem is about LEARNING, not that learners are the problem. Click The problem is about LEARNING, not that learners are the problem.

School leaders often skip this step Promise & Pitfalls The learner centered problem is common to many students and if solved, will positively impact learning and broader achievement goals but…… The process is messy, challenging, and requires dedicated time Click The problem is about LEARNING, not that learners are the problem. School leaders often skip this step thus

Facilitate Staff ownership Promise & Pitfalls Facilitate Staff ownership Plan time for staff to engage in data analyses as is allowing them to develop ownership by engaging in the work directly Deepen Understanding In-depth discussion enhances staff ability to understand and identify learning problems of learning Click The problem is about LEARNING, not that learners are the problem.

Narrowing the Scope of Inquiry Generates new questions Directs examination of data including new data sources Priority Question Prompts deeper investigation Single Source of Data Click Examining Multiple Data Sources Identifying a learner-centered problem

Selecting a Single Source of Data Essential Questions: Which data will help us to ask the questions we have about student learning? Which data will be the most compelling for school staff?

Delving into a Single Source of Data DEEPENS UNDERSTANDING OF STUDEINT THINKING CHALLENGES ASSUPSIONS ABOUT STUDENTS GENERATES NEW QUESTIONS

Understanding Student Thinking A response on an assessment item is the end product of the thinking process Self-Correction Response Communication Skills Background knowledge Problem Solving (Analysis, Synthesis) Skills Approach-Metacognition Analysis of data helps us to learn more about the path of student thinking.

What are the Skills Needed? Analysis Each table, will review an item/question to determine the skills students would need to successfully respond to the item. Identify specific terms. What differences in perspectives were noted in your group?   Activity 1 – Analysis of skills (10 min) Purpose – This activity is designed to engage participants in accurately discerning thinking/cognitive skills with task demands. Materials – Sample Test Items Participants at each table will review an item/question to determine the skills students must engage to successfully respond to the item. Participants will identify and chart specific terms, discuss commonalities and differences in perspectives. Lastly, participants will respond to the question “What differences in perspectives were noted in your group?” SOL items may be accessed from: http://www.pen.k12.va.us/testing/sol/released_tests/index.shtml

Identifying the “Right” Data Sources Priority questions: Are our students college and career ready? How do students with disabilities enrolled in co-taught classes perform in comparison to their non-disabled peers on SOL? How well are co-taught classes preparing students with disabilities perform in comparison to their non-disabled peers ? How well are special education resource service, preparing students with disabilities to perform in comparison to their non-disabled peers ? How well are students enrolled in AP classes performing on AP exams in students not scoring higher (4 and 5) on AP exams? What does variance in homework completion tell us about levels of student engagement/ How do homework completion rates vary by content, teacher and grade levels? Why are homework completion rates low? How are gap groups performing in comparison to each other?

Instructional Practice Assumptions Affect Instructional Practice Beliefs Instructional Practice Attitudes Training Values

Challenging Assumptions Identifying a Learner Centered Problem Challenging Assumptions Creating space to understand the significance of changing instructional practice Maintains a focus on the objective rather than the subjective Exposes beliefs about student ability

All Data Ladder of Inference I take: actions I draw: conclusions   I draw: conclusions   I add: interpretation   Another version of the Ladder of Inference: Continuation of previous slide. CLICK 7 times (for the entire slide) Facilitator Note: The star serves as a reminder of were we should start, on the first rung of the ladder. Unfortunately one of our practices has been that we jump to action as soon as we see a data report, we use this visual as it helps to always begin our observations of data by starting on the lowest rung of the ladder, keeping a focus on what we see. I select: some data   All Data

Triangulating Multiple Data Sources Analyzing other data sources to: illuminate What is learned through initial analysis Click dispute confirm

Triangulation Activity Each table reviews a set of data to: Learning by Doing – Examining Multiple Sources Triangulation Activity Each table reviews a set of data to: Delineate the skills and knowledge students must demonstrate to produce high quality work Practice formulating a Learner Centered Problem Activity 2 – Triangulation Activity The purpose of this activity is to provide an experiential learning opportunity for participants to engage in data triangulation.   Materials – Sample Data Sets (RELA, Math); Student Work Samples Each table group will review a set of data (at least three charts; or in the case of student work samples -10+) to practice: Delineating the skills and knowledge students must demonstrate to produce high quality work Practice formulating a Learner Centered Problem Participants will chart the information on poster paper

Evaluating the Learner-Centered Problem Is the LCP: Based on multiple forms of evidence found in examining data Specific and Small Within the school’s control Directly Related to the priority question As statement about student learning

Introducing Step 4 to a Colleague Constructing Knowledge At your table create a brief (min) PD activity that introduces the Step 4 to a group of colleagues including the role of Learner-Centered Problem Activity 4 – Constructing Knowledge (20 min) Purpose – This activity is designed help participants deepen their understanding of concepts learned and identify areas where additional review is needed to strengthen conceptual understanding by requiring participants to create a training activity based on Step 4.   Based on the learnings from today’s presentation, participants will work in pairs at their table to create a PD activity that introduces the concept of Step 4 and identifying a Learner Centered Problem.

Looking Ahead (5 minutes) Now that we have gained understanding of Step 2 and the key tasks, lets reflect upon your school teams current level of functioning with each of the three key tasks. We will do this through repeating the Stoplight Protocol. Reflect on your stoplight protocol thinking and where you placed the green, yellow, or red dot. Facilitator Note: Participants should see their dots from the first time they completed the Stoplight Protocol in the Introduction Module. Allow the participants to reflect on their dot and see if they still agree with their initial thinking. Review the Stoplight Protocol purpose and directions. Click to bring in each of the key tasks as the participants repeat the Stoplight Protocol

The ACE Habits of Mind Facilitator Note: Remind participants of the purpose behind the inclusion of the ACE Habits of Mind in BEING Data Wise instead of Doing data wise. Facilitator: The continual actions taken to apply the ACE Habits of Mind is the difference between BEING and DOING. These habits of mind also make the Data Wise improvement process more than just a way of looking at data. By applying these habits of mind, the thinking about data and its use shifts, thus supporting a culture of inquiry that surpasses normal jumping from data to action. When the culture of looking at data to inquire in a systematic way does not exist, subsequent action tends to fall flat. Purposeful preparation for inquiry is needed in order to avoid this common pitfall.

Plus /Delta Protocol What helped me to learn/interact with today’s meeting objectives? What changes are needed to help me to better learn/interact with today’s meeting objectives? Ask participants to list what went well about the meeting session (“Plus”) and what they would have liked to change (“Delta”). We do not comment on pluses or deltas, but rather let them be a record of what happened. Effective facilitators use the last meeting’s plus/delta to plan the next meeting, talking care to address the details. (5 minutes)

This presentations was modified from: PGCPS: Ebony Cross Shields, Felice N. DeSouza, Donna Drakeford, Rotunda Floyd-Cooper, Tasheka Green, Paula Harris, Rhonda Hawkins, Trina Hayes, David Rease, Jr., Anthony Sims, Jennifer Williams