Data Wise Improvement Process Module 2 Building Assessment Literacy Ebony Cross Shields, Felice N. DeSouza, Donna Drakeford, Rotunda Floyd-Cooper, Tasheka Green, Paula Harris, Rhonda Hawkins, Trina Hayes, David Rease, Jr., Anthony Sims , Jennifer Williams document was retrieved from PGCPS

Increase Comfort with Data Module 2 Objectives: Provide a review of the Data Wise Improvement Process, ACE Habits of Mind, and Ladder of Inference. Understand and apply principles of Step 2: Building Assessment Literacy through examining ways to: Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use Consider ACE within Step 2 Increase Comfort with Data Facilitator: In the introduction module, teams/schools were given the opportunity to learn about Data Wise as an improvement process and the new way of thinking that is used when using data to drive inquiry and what that means for building a “data culture”. Provide a review of the Data Wise Improvement Process, ACE Habits of Mind, and the Ladder of Inference You engaged in tasks that covered: Click The primary outcome for Step 2 is to guide staff in understanding multiple forms of assessments, how the results are reported, and how the results are responsibly used to inform decisions that impact instruction. With regular practice, teams will build their capacity and increase comfort data. We will look closely at the key tasks for step 2: Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use Also, we will... Highlight the integration of ACE within Step 2 Facilitators Note: It is important to emphasize that this is the second step in the Prepare Phase. Prepare phase comes before Inquire because when teams use data to inquire they often have difficulty if they are not prepared for the challenging work inquiry that is through team thinking and problem solving; structured within specifically formatted meetings and the consistent use of protocols. Much more will be learned about Data Wise meetings and the use of Protocols in Data Wise.

Assume the position of a learner Gentle Reminder… Assume the position of a learner Facilitator: As we take a moment to review the established norms, it is important to highlight this norm specifically as we will engage in learning to SEE together. CLICK Facilitator: Assume the position of a learner

Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Why of Step 2 Review Skills Tested Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 In module 1, teams/schools were given the opportunity to gather evidence or information to paint a complete picture of their school, and bring staff members together around organizing for collaborative work. These key tasks provided staff the opportunity to collectively learn how to regularly, build strong teams, see together and work together. Facilitator: Today we will engage in the collective work of building assessment literacy; before jumping into the why of Step 2, we will take a moment to revisit Data Wise, ACE Habits of Mind, and the Ladder of Inference.

What is Data Wise? Data Wise is… A school improvement PROCESS that organizes and brings coherence to the work of improvement. A specific PROCESS that facilitates intentional thinking and utilizing a more disciplined way of looking at data as a collaborative group. A PROCESS that helps all educators in all positions to learn how to analyze data in a manner that contributes to improved instruction and increased student learning. Facilitator Notes: Read bullets as they come in.. Click Allow the participants to read the slide. Ask what they notice about the slide. If not mentioned, highlight that Data Wise is a process that is integrated into schools/systems to provide coherence to processes that influence practice and organizational learning. It is about bringing together a skilled team of educators that work together to implement a coherent instructional plan to identify the learning needs of every student, and to meet those needs. There are parts of the Data Wise process that informs all of our traditional meetings that we are accustomed to such as collaborative planning, data utilization, analyzing student work, etc.

Is . . . Is Not What Data Wise IS What Data Wise is NOT Inquiry - questions drive learning and next steps Declarative - stating what has to be done Collaborative - all staff have a voice in determining what actions to take and what questions to explore Directive - one person decides what happens in isolation Designed to create spaces for learning about instruction, assessment, and practice for everyone A process that assumes skill and knowledge exist without having evidence of intentional knowledge and skill development. Facilitator Note: Data Wise is a broader framework Review the slide and allow the participants to share: What Data Wise is and is not? (2 minutes)

Is . . . Is Not What Data Wise IS What Data Wise IS NOT Means of organizing and bringing coherence to staff members’ collective efforts at improvement A program to implement A means to help school leaders organize the work of school improvement around a process that has specific, manageable steps A compliance measure or a checklist to be completed Cyclical; recursive A linear process that doesn’t allow for assessment and adjustment A series of yearlong coherent inquiry meetings A singular meeting (collaborative planning, data utilization, analyzing student work, etc.) Facilitator Note: Continuation of previous slide Data Wise is a broader framework Review the slide and allow the participants to share: What Data Wise is and is not? (2 minutes)

The ACE Habits of Mind Facilitator Note: Remind participants of the purpose behind the inclusion of the ACE Habits of Mind in BEING Data Wise instead of Doing data wise. Facilitator: The continual actions taken to apply the ACE Habits of Mind is the difference between BEING and DOING. These habits of mind also make the Data Wise improvement process more than just a way of looking at data. By applying these habits of mind, the thinking about data and its use shifts, thus supporting a culture of inquiry that surpasses normal jumping from data to action. When the culture of looking at data to inquire in a systematic way does not exist, subsequent action tends to fall flat. Purposeful preparation for inquiry is needed in order to avoid this common pitfall.

All Data Ladder of Inference I take: actions I draw: conclusions   I draw: conclusions   I add: interpretation   Another version of the Ladder of Inference: Continuation of previous slide. Facilitator Note: Unfortunately one of our practices has been that we jump to action as soon as we see a data report, we use this visual as it helps to always begin our observations of data by starting on the lowest rung of the ladder, keeping a focus on what we see. Let’s just to review the star serves as a reminder of were we should start, on the first rung of the ladder. CLICK 7 times (for the entire slide) I select: some data   All Data

Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Why of Step 2 Review Skills Tested Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 Facilitator: We will take a moment to discuss the WHY of Step 2 The primary outcome of Module 2 is to establish an understanding and build capacity around an essential step in the “Prepare” Phase of the Data Wise Improvement Process that will help faculty develop assessment literacy.   To build assessment literacy it is helpful to understand both the different types of assessments and the various scales scores that are used.  Participants in Module 2 will examine the inferences that may be drawn from these reports and specific differences in outcomes that can be considered meaningful. Additionally, participants will gain familiarity with key concepts such as reliability, validity, measurement error, and sampling error.  At the conclusion of Module 2, participants will be able to understand and apply the principles of Step 2: Build Assessment Literacy.

Increase Comfort with Data STEP 2 Build Assessment Literacy 2.1 Review Skills Tested 2.2 Study How the Results are Reported 2.3 Learn Principles of Responsible Data Use Increase Comfort with Data Click Facilitator: The primary outcome for Step 2 is to guide staff in understanding multiple forms of assessments, how the results are reported, and how the results are responsibly used to inform decisions that impact instruction. With regular practice, teams will build their capacity and increase comfort with reading and interpreting data. We will look closely at the key tasks for step 2: Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use

Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Why of Step 2 Review Skills Tested Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 Facilitator: As you see in our agenda that’s indicated by the red bar, we have skipped “the why of step 2” and key tasks “review skills tested” and “study how the results are reported“ to begin with learn principles of responsible data use. As educators we know that sometimes you have to be unconventional in order to be most effective. I can assure you that the rationale for this will be clear as we proceed through this work.

Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Why of Step 2 Learn Principles of Responsible Data Use Review Skills Tested Click 1 time Facilitator: As you view this agenda you will see that we should be in key task “review skills tested” followed by “study how results are reported” . However, by taking a look at these key tasks out of numerical order and addressing conceptual order, engaging in key task “learn principles of responsible data use” first will allow us to focus on our practice of using data responsibly. Facilitator Note: Be sure to reference that each team member has received and read the “Tips for Using Data Responsibly” to serve as a guide during group discussion of the vignette. Resources needed: Data Wise Book - Chapter 2 Tips for Using Data Responsibly Study How the Results are Reported ACE within Step 2

Using Data Responsibly Vignette During a collaborative teacher meeting, the team reviews the most recently scored assessment. After comparing students performance to one another’s (student to student and/or classroom to classroom), the team moves students into groups with scores ranging from high to low. Facilitator: We will engage in an activity that will allow us to focus on our practice and assess what we know about using data responsibly. Click Provide participants with Tips for Using Data Responsibly (8-10 minutes total) Allow participants to independently read Tips for Using Data Responsibly (3 minutes) After reading, participants will work in groups to play out the vignettes (5 minutes) While the participants are engaged, monitor each groups conversation to determine evidence of using data responsibly. As you move around the room to listen, listen for some of the explicit connections. Connections: Tips 1, 2, 3, 4, & 6. While working in group chart the evidence of responsible use of data CLICK After the completion of each vignette, allow the groups to share the evidence that the team used data responsibly? If not, how could the team use more of the ideas from the handout to use data more responsibly? Facilitator: Now that we have engaged in this activity, we are ready to move further into the next key tasks for building assessment literacy together.

Agenda The Why of Step 2 Review Skills Tested Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference Facilitator: Key tasks review skills tested and study how results are reported will be addressed as we engage in reviewing and interpreting how results are reported. This report is only 1 example of how data is presented through one of our data repositories.

Scavenger Hunt (6-8 minutes) Facilitator: T To understand the key tasks 2.1 and 2.2 we are now going to engage in the “Scavenger Hunt Protocol”. The purpose of this protocol is to help us learn how to SEE, READ and INTERPRET data together. As we engage and share what we see, we will build the knowledge capital of the team/school consequently building the capacity of the instructional staff around various data reports.

Agenda The Why of Step 2 Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 Review Skills Tested Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference

Stoplight Protocol 2.1 Review Skills Tested 2.2 Study How the Results are Reported 2.3 Learn Principles of Responsible Data Use (5 minutes) Now that we have gained understanding of Step 2 and the key tasks, lets reflect upon your school teams current level of functioning with each of the three key tasks. We will do this through repeating the Stoplight Protocol. Reflect on your stoplight protocol thinking and where you placed the green, yellow, or red dot. Facilitator Note: Participants should see their dots from the first time they completed the Stoplight Protocol in the Introduction Module. Allow the participants to reflect on their dot and see if they still agree with their initial thinking. Review the Stoplight Protocol purpose and directions. Click to bring in each of the key tasks as the participants repeat the Stoplight Protocol

Protocol/Learning Task Build Assessment Literacy: Evidence Key Task Protocol/Learning Task Artifact/Evidence Review skills tested Scavenger Hunt Agendas and notes from meetings focused on building assessment literacy Study how results are reported Slides from PowerPoint presentations in which mini-lessons around score reports and principles of responsible data use are featured Videos and pictures of staff members observed explaining the content or reporting format of assessments Learn principles of responsible data use Handout Vignette Placemat Activity Staff members observed holding one another accountable for using data responsibly As schools begin engaging in Data Wise, recording school improvement will be captured through the artifacts/evidence of actually engaging in doing the work within each step. The artifacts/evidence allow teams to maintain a relentless focus on evidence. Additionally, the artifacts/evidence provide a list of possible sources to support your claim about what stage of development your school is in with respect to a particular task. Review the chart. (5 minutes)

Agenda Revisit Data Wise, ACE Habits of Mind and the Ladder of Inference The Why of Step 2 Review Skills Tested Study How the Results are Reported Learn Principles of Responsible Data Use ACE within Step 2 Facilitator: As with the previous modules and those to come, ACE Habits of mind is the commitment of cultivating a way of thinking that prioritizes Action, Collaboration and Evidence. The next 3 slides will provide the concrete ideas for “how” to support your colleagues in developing these habits.

"A" in ACE Habits of Mind Don't take action on the basis of a single measure. Discussions should include: What skills did those particular test items assess? Facilitator: Let each team member read and share the meaning they glean from each slide.

"C" in ACE Habits of Mind Ensure multiple perspectives look at assessments together. A diverse group of people will ensure varied levels of expertise. Specialist and special educators have specific knowledge to specific learning disabilities and needs. The more diverse the group around the table, the greater the opportunity to allow each person to be both expert and learner. Facilitator: Let each team member read and share the meaning they glean from each slide.

"E" in ACE Habits of Mind Provide regular time for people to practice reading and interpreting data in varied forms and reports. This cannot be taught in isolation and should continue regularly through-out the process and year. Discussions should include: How are results reported? Are we looking for raw scores or scaled scores? Facilitator: Let each team member read and share the meaning they glean from each slide. CLICK

Objectives Provide a review of the Data Wise Improvement Process, ACE Habits of Mind, and Ladder of Inference. Understand and apply principles of Step 2: Building Assessment Literacy through examining ways to: Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use Consider ACE & Step 2 Facilitator: During today’s professional learning goal, we ... learned the primary outcome for Step 2, to guide staff in understanding multiple forms of assessments, how the results are reported, and how the results are responsibly used to inform decisions that impact instruction. With regular practice, teams will build their capacity and increase comfort. We looked closely at the key tasks for step 2: Click Review Skills Tested Study How Results are Reported Learn the Principles of Responsible Data Use And, we... Considered ACE within Step 2

Plus /Delta Protocol What helped me to learn/interact with today’s meeting objectives? What changes are needed to help me to better learn/interact with today’s meeting objectives? Ask participants to list what went well about the meeting session (“Plus”) and what they would have liked to change (“Delta”). We do not comment on pluses or deltas, but rather let them be a record of what happened. Effective facilitators use the last meeting’s plus/delta to plan the next meeting, talking care to address the details. (5 minutes)