DATA ANALYSIS Facilitator Talking Points:

DATA ANALYSIS Facilitator Talking Points:
As mentioned in the opening, the purpose of this training is not to review all required interventions, rather to focus on the most important aspect of the work: engaging in the TAIS continuous improvement process. Now we are going to take a look at the first quadrant of the continuous improvement process, data analysis.

Implementation & Monitoring
Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring The purpose of data analysis is to use multiple data sources to determine “what” factual insights are revealed about the strengths and needs of the campus. Facilitator Talking Points: We are going to first look at the Data Analysis quadrant. Data analysis is about carefully identifying WHAT the data is telling you. Data alone doesn’t carry meaning, so it must be interpreted to reveal factual insights to the strengths and needs of the campus/district system. Data analysis should utilize multiple sources of data over time throughout the continuous improvement process to identify trends and patterns that contribute to areas of low performance. It’s imperative that at this point in the process, the team remains objective, factual, and addresses only the WHAT. Facilitation Notes: Share with group the use of chevrons to indicate the progression of where we are at in the continuous improvement process. TEC (a)….Requires that a targeted improvement plan be developed, approved, and submitted that addresses the areas of low performance. This should be covered in your data analysis and a problem statement developed for each area of low performance.

GETTING STARTED WITH DATA ANALYSIS
Facilitator Talking Points: Data guru Victoria Bernhardt proposes that campus teams begin their end-of-year/planning for upcoming year data analysis process by addressing the following questions.

GETTING STARTED What is the vision for your school?
What are the roadblocks (i.e., people, products, reasons) to collecting data at your campus? What are the roadblocks to analyzing data at your campus? What are the roadblocks to using data at your campus? How can you eliminate the roadblocks identified so data can be used on a systemic and continuous basis? Bernhardt, pg 17 Facilitator Talking Points: As you review these questions, keep in mind Dr. Laura Lipton’s recommendations about the importance of a culture of inquiry and honesty. Although it is essential to revisit your vision, we are going to reserve that discussion for conversations back on the campus. For the next 5-10 minutes, discuss your roadblocks with your team member(s).

Steps for Data Analysis Work
Create an inventory of data sources Analyze trends and patterns within the data Identify the problem area that resulted in the IR rating based on trends and patterns in the data Create a problem statement Verify problem statement against the Problem Statement Checklist Facilitator Talking Points: The following is a list of high level steps that can be taken when navigating the data analysis process. In this section, we will briefly discuss specific actions that can be taken to fulfill each of these steps.

Facilitator Talking Points:
In terms of data, most campuses tend to focus their energy on STAAR results and other data found on TAPR reports and other state reports. Please keep in mind, your accountability data only serves as a signal to your campus that there is a problem that must be addressed. It should be viewed simply as a “siren” to signal that action must be taken - not an indication of the source of the problem. What does your accountability rating mean as a signal for your campus? In addition to state assessment data, what data sources give you a deeper understanding about the source of the problem? Facilitation Notes: Question for discussion: Table talk (5 min) Last point setting up the Margaret Heritage clip Use questions to get teams thinking about Margaret heritage explain where to begin with the data analysis process.

taisresources.net http://www.taisresources.net/data-analysis/
Facilitator Talking Points: Let’s take a look at what Dr. Heritage has to say around data. As you watch, be thinking about the prompt that precedes the video. Facilitation Notes: Go to --locate the data analysis section and find Margaret Heritage, “Preparing for Data Analysis”

SHARE YOUR THOUGHTS Facilitator Talking Points: OPTIONAL
After watching the video ask the participants to share the keys points from this video clip and insights gained.

Create an inventory of data sources Record all data available when looking through the lens of the CSFs Facilitator Talking Points: By creating an inventory of data sources, we will be able to identify why the alarm is sounding. Consider looking for data sources through the lens of each of the critical success factors. It is important to consider all data points that your campus/district has access to in addition to state assessment data. Facilitation Notes: Remind campuses that this is a suggested process for data analysis. Encourage teams to see how these steps align to processes that they already may have in place.

Critical Success Factors
7 Critical Success Factors Facilitator Talking Points: The CSFs represent the elements of a campus system that are critical to success. Analyzing data around each CSF provides a true picture of the functions and systems within a school that leads to the student achievement outcomes. We are going to take a deeper look at the kinds of data your team can draw from in order to get this holistic pictures. We are about to get into a tool that will organize these possible data sources. Facilitation Notes: Emphasize the importance of data analysis through the CSF lens is to gain a clear picture of the effectiveness of each CSF system and/or process that is in place.

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Critical Success Factors CSF 1: Academic Performance CSF 2: Use of Quality Data to Dive Instruction CSF 3: Leadership Effectiveness CSF 4: Increase Learning Time CSF 5: Family and Community Engagement CSF 6: School Climate CSF 7: Teacher Quality List all data sources used for each CSF NOTES Facilitator Talking Points: The purpose of this activity is to collaboratively identify the data sources campuses/districts are already collecting around each CSF. Step 1. Access CSF placemat Step 2. Ask teams to begin to fill out the chart with data sources teams use to analyze data that would fit into each CSF. (Note: Encourage teams to fill out more than one or two columns and to consider CSFs other than Academic Performance). Data analysis is continuous, therefore revisiting this chart each quarter will not only help strengthen data discussions, but be able to help the team identify what data sources need to be collected to ensure effective implementation of reconstitution planning/implementation. Facilitation Notes: Set up the activity and review the steps teams will work through Give teams 5-7 min working time Check in with teams around 6 min to add a 3-4 more min Handout 5: CSF Placemat blank

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: This is a CSF data sort example. This is a great resource if team seems a little stuck or needs extra assistance. Share the examples as a means to continue conversations and make them aware of an additional resource. Consider how you might use this overall activity across your campus to allow all stakeholders to be a part of the process and see the importance of their role. Facilitation Notes: Share the examples as a means to continue conversations and make them aware of an additional resource. Handout 6: CSF Placemat Data Examples

DATA ANALYSIS What data do you have? How do you use it?
Aside from state assessment data, what other data will you need to analyze to develop a plan for continuous improvement? Facilitator Talking Points: So, as Margaret stated - a good starting point is gathering and looking at what data you have. Once you have the big picture it will reveal to you what you need to investigate further. Facilitation Notes: If time permits have teams reflect individually on these questions Slides set up the next section of data analysis, which is analyzing trends.

DATA ANALYSIS How much are teachers engaged in data analysis?
How could they become more engaged? What does Dr. Heritage mean when she refers to the “opportunity to learn?” Facilitator Talking Points: Think about the various data sources as you navigate through the continuous improvement process. Because data analysis lives throughout the continuous improvement process, data sources and HOW you use those data sources will vary throughout the process. With that in mind, consider the following questions.

TYPES OF DATA Demographic School Process Perception Student Learning
Facilitator Talking Points: Once you have taken an inventory of the data you currently have available, it’s important that you also understand the different types of data, or the “multiple measures of data.” An awareness of the types of data, helps to clarify what information we should expect to glean from the various sources of data we collect. For example, perception data, the most overlooked source of data, over time will tell whether school climate and culture are improving. Moreover, looking at perception data by demographic will tell us which student groups are positively and negatively affected by the climate and culture. To apply this to the TAIS framework: Looking at perception data by demographic for the CSF teacher quality would lead us to discover which students are positively and negatively impacted by our teacher’s classroom environment OPTIONAL ACTIVITY: STEP 1: Allow participants 2-3 minutes to review this handout. STEP 2: Allow participant 5-7 minutes to discuss the insights gained from this handout. STEP 3: Allow participants 5 minutes to discuss how to use the CSF inventory and the multiple measures handout to deepen their data analysis conversations.

• After thinking about all of the data available to you, you may be feeling a little like this…. • To get a clear picture of what the data says, it’s imperative to analyze data sources to identify trends and patterns that contribute to areas of low performance. • As data is analyzed, gaps based on the data emerge and teams will need to determine what the problem areas are. • Often when gaps are identified, an immediate reaction is to assign the reason and/or solutions for the gap. However, before taking action, problem statements will be developed in order to launch into a root cause analysis to discover WHY the problem exists.

Create an inventory of data sources Analyze trends and patterns within the data Identify the problem area that resulted in the IR rating based on trends and patterns in the data Facilitator Talking Points: Our next step is to analyze the trends and patterns seen within the data. After doing so, only then can we truly identify the problem area(s) that resulted in the IR rating. Remember we are looking to see what factual evidence the data is revealing. Margaret Heritage shared some valuable tips around data analysis: (ask teams to recall the tips Dr. Heritage shared). Facilitation Notes: Have teams reflect individually (or as teams) on how they practice analyzing trends and patterns. Get teams to stretch their thinking by being specific about the actions or questions they rely on to do so.

The first four questions in the Data Analysis tab of the targeted improvement plan template will refer to your index data. These questions are designed to help you dig deeper in and around the index data to pinpoint specific student groups with the largest gaps in their outcomes. Index 1 focuses on Student Achievement. The question is, “which student group is in greatest need of improvement?” You will identify by student group and content area that had the most impact on your Index 1 score. System safeguards, which give data by student group and tested content areas, is tool that helps you with this analysis. This will help you narrow your focus in identifying gaps your in data. Facilitator Note: You may want to have participants open a version of the template and follow along with you while going through these slides. You may choose to wait until a work time to have participants access a version of the template. See the OPTIONAL opening slide that may help facilitate accessing the template. Handouts can also be accessed on the PMI website or from the TCDSS website.

Index 2 focuses on student growth and these questions help to narrow the focus on specific student groups impacting this index.

The first question here asks you to identify the two ethnic/race student groups that you were held accountable for on your 2015 State Accountability ratings. The second question allows us to be proactive in thinking about ratings that will come out in August of 2016. The two ethnic/race student groups that you will be held accountable for in August 2016 will be the two lowest performing ethnic/race groups according to 2015 Index 1 calculations. You can use your Index 1 calculation pages once accountability ratings are released to help you identify these student groups. By doing, this we can keep these student groups in mind as we plan for the upcoming year.

si Facilitator Talking Points:
For Index 4, the first question is just yes/no if you met the index target. If you miss index 4, the follow-up question asks you to focus in on which component(s) contributed to missing Index 4. This question is really for high school campuses. For other campuses, only the STAAR component figures into Index 4. Remember, the STAAR component here focuses on students meeting the final passing rate, not phase-in.

CSFs Organized by Data Sources
Facilitator Talking Points: As mentioned previously, it is important that your campus team uses multiple data sources to validate the problem. Data is used throughout the continuous improvement planning process. It is critical that your team uses multiple data sources beyond the accountability reports while working in this process. This section in the data analysis tab of the template, allows your team to capture the work that was introduced earlier in this section wherein you completed a data inventory by CSFs. Throughout the needs assessment process, this data will be used to validate whether your hunches about your root cause(s) are verifiable. This will be especially helpful as you begin to validate the root cause(s) identified when conducting your root cause analysis. Facilitation Notes: Connect this section to systems thinking as it leads into the root cause analysis process piece Reference back to inventory….this section gets at what CSF data is “contributing” IR problem area

Section III - For the campus tab – they will need to think through the PBMAS lens – did your campus contribute to the PBMAS district report? Section IV – This section is only applicable for priority campuses. All other campuses will skip if you are not priority. This is basically identifying student groups that contributed to them being a priority campus.

Create an inventory of data sources Analyze trends and patterns within the data Identify the problem area that resulted in the IR rating based on trends and patterns in the data Create a problem statement Verify problem statement against the Problem Statement Checklist Facilitator Talking Points: The next steps we are going to walk through are how to create a problem statement based on the data and verifying the problem statement with a check list resource. Facilitation Notes: Emphasize the importance of staying in the “what” of data and not the “why”. The team further examines the causal factor for the problem statement during the need assessment process a root cause analysis

CURRENT PERFORMANCE Facilitator Talking Points:
The main purpose of conducting a data analysis is to develop a clear picture of the campus’s current performance

CURRENT PERFORMANCE DESIRED PERFORMANCE Facilitator Talking Points:
Once your team has a clear picture of your current performance, you will create 2-5 problems statements based on high leverage areas for which a significant gap between current performance and future is identified. A problem statement objectively describes WHAT the data indicates about the campus’s current performance. The team further examines the causal factor for the problem statement, or WHY the problem is occurring, during the needs assessment process at which time the campus conducts a root cause analysis. Problem statements will also very likely be rooted in student performance data, however it is not required that all problem statements are rooted in STAAR data. It’s important to note if a gap exists, based on the data, in culture or climate on a campus. A problem statement can reflect this gap as well. Let’s take a closer look at the element of a well written problem statement Facilitation Notes: Emphasize that the intention isn’t to complete all of the work around problem statements today. But as a team, begin practicing the process of developing a problem statement.

Well written problem statements are…
Criteria Y/N Substantiated by facts/data Written objectively Uses concise language Includes specific details (who, what, when, where) Focuses on a single, manageable issue Has relevance to our campus Avoids causation or assigning solutions Facilitator Talking Points: To help support the development of a problem statement, the following check list may also be used with your team. Notice that the criteria maintain objective and factual. Again, at this point, we are only exploring what the problems are. Being that the requirement to develop a targeted improvement plan is to focus on the areas of low performance, it’s possible that teams may only identify 2-3 problem statements. Keep in mind that, the rest of the plan will be developed further once the problem statements have been generated. At this point in the process you are bringing clarity to what issues need to be addressed in your plan. Refer to the Problem Statement Checklist handout. This is a resource that you can use in the development of your problems statements. With this criteria in mind, let’s review a sample problem statement to see if it is well written. Facilitation Notes: Have participants access the problem statement checklist Reiterate that this practice is to become very clear about WHAT the problem are is, not why or how to address it. Give teams time to review each criteria Handout 7: Problem Statement Checklist

PROBLEM STATEMENT QUALITY CHECK
ELLs have a 50% pass rate in reading due to a lack of parental involvement Facilitator Talking Points: Explain that as a group, you will assess the above problem statement according to the criteria in the tool. Ask that the group help as you assess the problem statement according to each piece of the criteria by giving a “thumbs up,” “thumbs down,” or something in between. Tell them that the end goal will be to produce an even better problem statement.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Facilitator Talking Points: Ask the group if the problem statement is substantiated by data. Ask some individual participants to share out why they think it was or was not.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Written objectively? Facilitator Talking Points: Ask the group if the problem statement is written objectively. Ask some individual participants to share out why they think it was or was not written objectively. If the point is not raised by the group explain that since the parental involvement piece is not supported by data that it is an opinion, and thus not written objectively.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Written objectively? Uses concise language? Facilitator Talking Points: Ask the group if the problem statement uses concise language. Ask some individual participants to share out why they thought it was or was not.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Written objectively? Uses concise language? Includes specific details (i.e., who, what, when, where) Facilitator Talking Points: Ask the group if the problem statement contains specific details such as who, what, when, where. Again encourage sharing out, if appropriate. Mention that it could possibly be more specific, however it does contain specific enough information to know who, what, when we are talking about, etc.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Written objectively? Uses concise language? Includes specific details (i.e., who, what, when, where) Focuses on a single issue? Facilitator Talking Points: Ask the group if the problem statement focuses on a single issue. Explain that we have already discussed that there are 2 issues contained in the problem statement: ELL pass rate in reading and parental involvement.

ELLs have a 50% pass rate in reading due to a lack of parental involvement Substantiated by data? Written objectively? Uses concise language? Includes specific details (i.e., who, what, when, where) Focuses on a single issue? Relevant to our campus? Avoids assigning causation or including solutions. Facilitator Talking Points: Ask the group if the problem statement appears relevant to the campus and avoids causation. Again encourage sharing out, if appropriate. If the group does not raise the point, explain that when we see the words “due to”, we are assigning blame. OPTIONAL ACTIVITY Perhaps give the group some time to re-write the problem statement using the lessons learned from the criteria tool. Have them share out what they came up with. Allow the group to assess each one that is shared.

ELLs have a 50% pass rate in reading
PROBLEM STATEMENT QUALITY CHECK ELLs have a 50% pass rate in reading due to a lack of parental involvement ELLs have a 50% pass rate in reading REVISED PROBLEM STATEMENT Facilitator Talking Points: Here is an example of a revised problem statement that addresses the criteria that were not followed previously. Perhaps quickly run the new problem statement through the criteria to illustrate – especially highlight no causation and one manageable issue. Facilitation Notes: Encourage sharing out, if appropriate. If the group does not raise the point, explain that when we see the words “due to,” we are assigning blame. OPTIONAL ACTIVITY Perhaps give the group some time to re-write the problem statement using the lessons learned from the criteria tool. Have them share out what they came up with. Allow the group to assess each one that is shared. (7-10min)

REMEMBER the problem statements identified and recorded within the template should be concise and objective statements that reflect the current state according to the data. These statements do not assign causation as to why a gap in the data exists or solutions to the problems. These statements clearly articulate the campus’s current performance. Essentially, problem statements capture “where you are” compared to “where you want to be.” Before beginning any planning or needs assessment, it is important to have a true picture of what the problems are as the foundation for your planning.

PRACTICE Facilitator Talking Points: Now it’s your turn to practice.
At your tables, there are posters/chart paper where teams will record the work from the day. For this work section, only focus on the problem statement. Explain the poster activity steps below. Poster Activity Steps: Have participants take piece of chart paper and fold it into quarters. Have participants label top left quadrant “Problem Statement.” Once teams have worked through identifying a gap in the data and creating a problem statement, have them record their problem statement in that top, left quadrant. Have teams post their posters close to their tables. Teams will return to this poster throughout training as they continue building these connections. Facilitation Notes: You may want to have a model ready to show participants. You can use the components from the training example to build out your poster. This activity was used during our summer TTIPS training. We found it very useful in being able to check for understanding for each component of these foundational connections. If a campus was off track, we were able to use this poster and the quality checks (Problem Statement Checklist) as a way to start a conversation to help guide campuses to a clearer understanding.

Create an inventory of data sources Analyze trends and patterns within the data Identify the problem area that resulted in the IR rating based on trends and patterns in the data Create a problem statement Verify problem statement against the Problem Statement Checklist Facilitator Talking Points: As a team, or individual, you are going reflect on the gaps based on data that caused the IR rating. Once a gap has been identified, know that more time and data will be needed. For the purposes of today, we are modeling and practicing the process so that it can be replicated back at your campus. Practice developing a problem statement using the problem statement checklist. When trying to determine which gap to focus on, consider which gaps (if addressed through a root cause analysis and plan) may have a positive impact on more than one CSF. Team work time: start with 13 minutes and check in; allow for a couple more minutes if needed. Once teams are done, have them record their problem statement on the poster. Facilitation Notes: Ask teams to bring data while going through this training, or make campus accountability reports available during training. OPTIONAL – You could have participants try to enter problem statement on template during this time. This might depend on whether or not you had introduced the template yet and/or how many colleagues you have available to assist.

Well written problem statements are…
Criteria Y/N Substantiated by facts/data Written objectively Uses concise language Includes specific details (who, what, when, where) Focuses on a single, manageable issue Has relevance to our campus Avoids causation or assigning solutions Facilitator Talking Points: To help support the development of a problem statement, the following checklist may also be used with your team. Notice that the criteria maintain objective and factual. Again, at this point we are only exploring what the problems are. Being that the requirement to develop a targeted improvement plan is to focus on the areas of low performance, it’s possible that teams may only identify 2-3 problem statements. Keep in mind, the rest of the plan will be developed once the problem statements have been generated. At this point in the process, you are bringing clarity to what issues needs to be addressed in your plan. Refer to the Problem Statement Checklist handout. This is a resource you can use in the development of your problems statements.

“A problem properly stated is half solved.”
John Dewey Facilitator Talking Points: Provide the participants time to read this quote. Then inform them that as we move through each section, especially data analysis and needs assessment, the underlying message in this quote w section think about what this quote means to each step in the continuous improvement process. Facilitation Notes: Provide the participants time to read this quote.

DATA ANALYSIS QUALITY CHECK
Has a thorough data analysis been conducted on the indexes and system safeguards that were missed or are potential areas of concern? Have clear problem statements been identified and created? Do problem statements meet the criteria in the problem statement checklist? Facilitator Talking Points: These questions have been developed for a quality check to be conducted before moving on to the needs assessment. This will help guide your work once you are back at your campus working with the rest of the team. The data analysis guidance document is available as well to support this process.

ENTER PROBLEM STATEMENT HERE
IMPROVEMENT PLAN TEMPLATE ENTER PROBLEM STATEMENT HERE REQUIRED OPTIONAL Facilitator Talking Points: After you have conducted a thorough data analysis and developed a problem statement, you will enter your problem statement(s) on the targeted improvement plan. This part of the template indicates where you will place your identified problem statements. You will relate your problem statement to the missed index. Based on district request, we have added the ability to connect your problem statement to a missed system safeguard. This is not a state requirement for DCSIs. If you choose locally for them to make that decision, you now have the ability.

DATA ANALYSIS NEXT STEPS
Identify trends and patterns within the data Consider how the team will capture the findings and develop problem statements Create and verify problem statement(s) Determine what additional data sources need to be collected Determine how the data will be organized for review Facilitator Talking Points: These questions have been developed for a quality check to be conducted before moving on to the needs assessment. This will help guide your work once you are back at your campus working with the rest of the team. The data analysis guidance document is available as well to support this process. Facilitation Notes: Allow teams to make note of steps. Emphasize that these steps don’t trump the process teams may already practice. Encourage them to use what works for them.

taisresources.net http://www.taisresources.net/needs-assessment/
Facilitator Talking Points: As we transition into the needs assessment section, we’re going to watch a short video from Laura Lipton about launching a data pursuit. This video emphasizes the need to continuously refer back to your data as you begin to explore WHY (i.e., the root cause) your problems are occurring. Facilitation Notes: The purpose of this video is to be a “teaser” for the next section, needs assessment. Play Video If there is time, follow up questions with teams

TABLE DISCUSSION How does the process Dr. Laura Lipton describes compare to the one you currently use to identify priorities and surface multiple causal theories? Once you have a list of possible causes for a problem, how do you validate and select the one that likely has the most impact? Facilitator Talking Points: Share out practices that you are already utilizing to analyze data. Share what you think you’ll add to your process based on what we have discussed so far today.

BREAK Facilitation Notes: 10 min break

NEEDS ASSESSMENT Facilitation Notes:
Transition slide for next step in the process.

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring The purpose of the needs assessment is to determine “why” gaps based on data exist by conducting a root cause analysis. Facilitator Talking Points: Often times when a problem is discovered, an action or intervention is immediately applied to the problem in an attempt to resolve it. The danger in action planning before drilling down to the root cause may only allow for the treatment of the symptoms of a bigger problem. Recall that the data analysis process is an attempt to identify WHAT the problem is. The jump from WHAT the problem is to HOW it can be fixed overlooks the most important step, which is determining WHY the problem exists. In order to align actions with areas of need, the root cause must be identified. Facilitation Notes: Remind participants that this section moves into the needs assessment (green quadrant) of the continuous improvement process

taisresources.net http://www.taisresources.net/needs-assessment/
Facilitator Talking Points: As you watch Dr. Heritage, consider the process your campus/district currently uses to narrow the focus in order to achieve desired results. Listen for the key points Dr. Heritage outlines that are important to consider as we transition from data analysis to needs assessment. Play video Coming back from video: How can data support the needs assessment process? Data corroborates our hunches and helps to narrow the focus and dig deeper by conducting a root cause analysis. Facilitation Notes: Go to select continuous improvement process and click on needs assessment

Steps for Needs Assessment Work
Conduct a root cause analysis Review additional data sources Select the root cause(s) Facilitator Talking Points: The following is a list of high level steps that can be taken when navigating the needs assessment process. As we go through this section, keep in mind the ways your campus/district may already practice examining the WHY of this work. These steps will be modeled in a way to be replicated with your teams to continue the work. 1. Once problem statements have been developed and prioritized, the next step is to determine WHY the problem(s) exists by conducting a root cause analysis. This is a three-step process that we will review later in this section. 2. As teams move through each step of the process, additional data may need to be gathered and/or reviewed to help corroborate/substantiate suggested reasons why a problem exists. Lastly, teams will narrow down possible causes to each problem statement and arrive at a consensus on possible root cause(s). Facilitation Notes: Remind groups that this is a suggested list of high level steps. As we go through this section, have teams think about the ways campuses/districts may already practice examining the WHY of this work. Share that these steps will be modeled in a way for teams to be able to replicate the work with their teams.

WHY? Facilitator Talking Points:
The purpose of the root cause analysis is to identify possible reasons WHY the problem is occurring. Digging beneath the surface of the common assumptions that are often made when attempting to fix the problems we see. Understanding the root cause will lead to a more effective plan that addresses the real issues and is not based on assumptions. Once we have analyzed the data and gained a clear understanding of what the data says through the development of problem statements, we can effectively begin to drill into WHY these problems exist. The needs assessment, or root cause analysis, will help us understand WHY the problems, or gaps in data, exist. The root cause is significant because it helps answer WHY the problem (or gap based on data) exists. Determining a reason WHY a gap exists, especially one that isn’t an obvious reason why, gives us insight as to HOW best to address the problem in the planning phase of the process. The root cause is a hypothesis for the problem statement. This hypothesis sets the context for us to treat the deeper issues, below the surface of the problem statement. This hypothesis is a data grounded, proposed reason why this problem is occurring. However, there are no absolutes when it comes to determining whether or not the “right” root cause has been identified. The root cause analysis process allows one to drill deep below the surface reasons why. Drilling deep enough helps better identify the root cause, thus allowing us to select the best possible strategy later in the process. Facilitator Note: Acknowledge those who may already work with a root cause analysis process.

WHY PROBLEM STATEMENT ROOT CAUSE
ELLs have a 50% pass rate in reading due to a lack of parental involvement Lack of teacher understanding for the instructional strategies needed to meet linguistic needs of ELLs WHY Facilitator Talking Points: If the problem statement we identified was “ELLs have a 50% pass rate in reading” once we conducted our root cause analysis process, we determined a possible reason why the ELL passing rate is this low. In this example, after a root cause analysis was conducted it was determined that the ELL passing rate is 50% because of a “Lack of teacher understanding for instructional strategies needed to meet linguistic needs of ELLs” Notice there is no call to action, proposed solutions, interventions or data listed within this reason. In this case, once a closer look was taken into why, it was revealed that it’s possibly due to a lack of knowledge and skills. Now, let’s talk through how we got there.

Conduct a root cause analysis Facilitator Talking Points: The first step of the needs assessment process is to conduct a root cause analysis.

Conduct a root cause analysis ACTION 1.1: Conduct the 10, 5, 5 process ACTION 1.2: Use the TWO CIRCLES process ACTION 1.3: Conduct the 5 WHYS process Facilitator Talking Points: There are 3 actions teams can take to address the first step. We’ll go through each action and allow time to practice. By the end of this section, a root cause will be identified and recorded on team posters.

ROOT CAUSE ANALYSIS PROCESS
PROBLEM STATEMENT ACTION 1: 10, 5, 5 ACTION 2: 2 Circles ACTION 3: 5 Why ROOT CAUSE Facilitator Talking Points: The root cause analysis process is designed to explore the reason why the problem(s) exists. Therefore, starting with the problem statement, our first ACTION uses the 10, 5, 5 process to generate a possible list of reasons WHY the problem exists. As we brainstorm this list we must use the data that we collected to validate whether this possible list is simply unsubstantiated claims or potential root causes. Our second ACTION is to use the 2 circle process to narrow our list to a set of potential root causes that are within our realm of control. Finally we use the 5 WHY process to dig as deeply as possible to identify the root cause that the team feels most certain is creating the problem on our campus. For those of you who aren’t familiar with a root cause analysis, we have a suggested process and associated protocol to share with you. Beginning with the problem statement, the following root cause analysis can be utilized to help drill down into why the gaps in data are occurring. As we discuss each phase of the process, time will be dedicated for you to practice the protocol starting with the problem statement you developed previously. Refer to the Root Cause Analysis handout. Inside the packets are the materials you can use to work through this process. Facilitation Notes: When PSPs aren’t present or able to be there, consider having teams available to facilitate this process during the training. Make note to participants that this is a suggested root cause analysis process Handout: 10_Root Cause Analysis

Before we practice this process, it important to understand that this process is designed to be a funnel. We start with a broad understanding of the problem based on the comprehensive data analysis and we gradually and intentionally narrow the focus of our conversation to a previously undiscovered underlying issue that is contributing to the problem identified.

PROBLEM STATEMENT ACTION 1: 10, 5, 5 ACTION 2: 2 Circles ACTION 3: 5 Why ROOT CAUSE Facilitator Talking Points: The first phase of this analysis is called the 10, 5, 5 10, 5, 5---This is a group brainstorming process meant to begin conversations around the possible reasons WHY the problems or gaps exist. This process establishes multiple hypothesis to a given problem statement. The purpose of this activity is to stretch brainstorming beyond the surface level or common reasons “why.” In a minute, starting with the drafted problem statement written on your poster, you’re going to be given time individually to come up with a list of reasons why the problem is occurring AND THEN YOU WILL BE GIVEN TIME TO WORK WHOLE GROUP TO COMBINE YOUR POSSIBLE REASONS WHY THE GAP EXISTS. There are no wrong answers so record whatever possible reasons come to mind that problem is occurring. However, it’s important to note that data will need to be referenced in order to avoid subjectivity and opinion based reasons moving forward. Handout 8: Root Cause Analysis activity

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: You have a handout in the root cause analysis process packet that looks like this. Please do not work ahead of the group. It’s imperative to following the protocol as facilitated in order to get the most out of the process.

10 5 5 REASONS WHY MORE MORE Facilitator Talking Points:
Explain and follow activity steps below. Activity Steps: Step 1: Allow teams 5-7 minutes to come up with 10 reasons why the problem exists individually (don’t advance the slides until it’s time for the next step). Let teams know when time is up and if they haven’t gotten to 10 reasons it’s ok. If they feel compelled to write more than 10 reasons, that’s ok too. Step 2: Click to next step when time is up. Now come up with 5 more reasons why the problem exists. Mention to them that it may be a little more difficult to come up with 5 more reasons. Encourage them to stretch their thinking and try to come with 5 more. 3-5 minutes Step 3: Click to next step. Now come up with 5 more reasons why the problem exists. This may be getting more difficult which is the intention. Allow 3 more minutes for this part. Acknowledge with teams that this may be hard to do after coming up with 15 reasons. The intention is to move past the obvious reasons and possible “finger pointing” that takes place when we think about why something isn’t working. Encourage them to do their best, there isn’t a right or wrong. Now, allowing teams to work whole group, have them share their reasons with one another and record the reasons on one piece of paper. Step 4: Ask the participants to take note of the reasons for which they currently have data to support their belief and which one’s need additional data. Facilitation Notes: Explain and follow activity steps Emphasize the importance of following the timing within the protocol

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: Your handout is probably starting to look something like this Next, we need to narrow down the list. In order to do so we will get into the next phase called “2 Circles.” Facilitation Notes: Ask teams if it became more challenging to come up with responses as time lapsed. Encourage them and let them know this is supposed to get more difficult. The idea is to dig deep.

PROBLEM STATEMENT ACTION 1: 10, 5, 5 ACTION 2: 2 Circles ACTION 3: 5 Why ROOT CAUSE Facilitator Talking Points: Often times we expend energy in solving issues that are beyond our direct control but perhaps we still have influence over. In order to make urgent changes that have a strong impact, we want to focus our attention on what we can directly control. So, in this next phase, you will sort through the list of 20 reasons and begin to decide which of those ideas can we control and which might we only have an influence over.

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: Instructions for 2 Circles For this activity, assign a scribe to record responses. Have each team member share one response and move to the next team member while the scribe records responses (click for animation). The scribe will place check marks next to repeated responses (click for animation). The team needs to swiftly decide which column or circle the reason belongs in: Control or Influence. To reiterate, “control” is defined by how directly a campus can act based on that reason. For example, if transportation schedule was a reason, in most cases, a campus is unable to make a change to the bus schedule without having to make some calls or have a meeting with district level team members. However, in some cases, campuses do have a direct impact on making that kinds of change without having to schedule meetings with outside team members. These decisions can take place within the walls of the campus. Either graphic organizer will work. It’s up to you/team to decide. You may also use one of the 10, 5, 5, lists recorded from a team member and place an “I” or “C” next to each. BEFORE MOVING ON Ask the participants to take note of the reasons for which they currently have data to support their belief and which one’s need additional data. Facilitator Note: Take minutes for this phase. Check in to see if more time can be allotted. Get as far as you can in this activity.

TEAM CONSENSUS Facilitator Talking Points:
Now that we have narrowed the list by determining what you think is in “control” vs “influence,” the next step is to select 1 reason listed in your circle of control. To do this, the team will need to come to a consensus as to which reason they would like to take through the next phase of the needs assessment process (5 Whys, which we’ll discuss in a minute). Possible practice is to utilize “hot dots” which is a voting protocol that can be easily and quickly used when faced with coming to a consensus. Have each team member use a sticker, post it, marker etc...and mark their top 2-3 possible reasons why the problem exists. The one with the most “dots” or indicators will represent the team consensus. An important point to make here is that while it most likely isn’t available to you today, it’s important to verify these reasons with data. This will help in deciding which reason to go with. Consider which of those reasons listed in the circle of control would have a high impact if you tried to resolve it. Also consider if there is data backing up this reason why or if this is just a stated opinion or feeling. Please know that the time provided today isn’t enough time to thoroughly conduct this analysis, however the intention is to begin to model the process so that you can take this back to your team when you have the right stakeholders, time, and data with you. Take 3-5 minutes and vote as a team on which reason listed within your control you’d like to move on with. Write that consensus at the bottom of your handout.

PROBLEM STATEMENT ACTION 1: 10, 5, 5 ACTION 2: 2 Circles ACTION 3: 5 Whys ROOT CAUSE Facilitator Talking Points: The last phase to this process is called 5 Whys. Once a consensus has been reached, the 5 Whys will provide an opportunity for teams to dig deeper into why the problem is occurring. The last answer to the last question becomes the possible root cause. Know that 5 is not a magic number, it’s possible your team may want to explore up to 7 “whys” and in other cases, only 3. Let’s revisit our example...

Problem Statement: ELLs have a 50% pass rate in reading
Team Consensus: school is boring to the students Why are the students bored at school? Because the students aren’t engage Why aren’t the students engaged? Because the aren’t connecting with the material Why aren’t the students connecting with the material? Facilitator Talking Points: In the example, the problem statement started with “ELLs have a 50% pass rate in reading.” The consensus was to look at “School is boring.” (Click) To begin the 5 Whys process, take the possible reason the team had a consensus on and turn it into the first question, “Why is school boring?” In the handout provided, the scribe can record responses. (Click) Once an answer is settled upon, write it down on the handout and turn the response into question 2. (Click) Each answer to “Why” becomes the next question until the team feels like they have landed on a root cause that Is still within the team’s control Has some sort of data to verify it’s a cause, again the purpose is to avoid opinions and subjectivity Is not stated in a way that takes action or describes how to solve the problem. It’s also important to avoid finger-pointing and blaming. It’s important to stay in the circle of control. Facilitation Notes: This slide is to model the first questions of this process

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: This process is lengthy and should generate a lot of discussion. We’ll start with giving teams minutes to conduct the 5 Whys and check in along the way. If your team gets stuck, determine if you have moved your way out of the “circle of control.” It’s possible that your final root cause may appear vastly different than where your team started. This is ok. This is a result of digging deeper and going beyond the obvious reason of why the gap exists. Record the root cause on the poster at your table.

Conduct a root cause analysis Review additional data sources ACTION 2.1: corroborate possible root causes with data ACTION 2.2: Eliminate hypotheses not supported by data and revisit brainstormed list Facilitator Talking Points: The actions around step two (which should be considered throughout the needs assessment process) are to review additional data. Once a root cause has been selected through the root cause process, determine if there are data to support any evidence of the hypothesis. Data will help reduce generalizations and personal opinions when hypothesizing WHY. Therefore if any possible reasons WHY aren’t supported by data, it would be beneficial for the team to revisit the brainstormed list of possible reasons WHY with data in mind. Possible example: 1. Why are the students bored at school? --because classes are boring for the students. Data to corroborate this answer: when conducting walkthroughs, data was collected indicating a high number of disengaged students. When interviewing a focus group of students regarding disengagement, students shared that they felt bored in class. 2. Why are students bored with their classes? --because students aren't connecting with the material. Data to corroborate this answer: When discussing the issue of student disengagement, many teachers felt that students were not connecting with the materials being taught.

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Facilitator Talking Points: Remember the many data sources available as you ensure the root cause being considered is supported by data. The CSF data sources document was introduced during the data analysis section of the training. Use the completed CSF data sources document to ensure all data sources that may support the proposed root cause are considered.

Data Analysis Needs Assessment Improvement Planning Implementation & Monitoring Critical Success Factors CSF 1: Academic Performance CSF 2: Use of Quality Data to Dive Instruction CSF 3: Leadership Effectiveness CSF 4: Increase Learning Time CSF 5: Family and Community Engagement CSF 6: School Climate CSF 7: Teacher Quality List all data sources used for each CSF NOTES Facilitator Talking Points: Provided to you in a handout form is a blank tool for your team to talk through what kind of data you already collect and analyze around the CSFs. Consider, does any of this data provide information or insight into the root cause identified? Not only does this data help with the root cause identification, but it will become important during the improvement planning development. Specifically, when it comes to determining the data that will be used to track quarterly interventions. (Time permitting, otherwise, it’s a tool to take back to their campuses and walk through with their PSPs) Facilitation Notes: Optional handout: 12_CSF Placemat

Conduct a root cause analysis Review additional data sources Select the root cause(s) ACTION 3.1: Review data if necessary ACTION 3.2: Come to consensus on the root cause Facilitator Talking Points: Step three will lead to the selection of the root cause for each problem statement. It’s important to remember that there are no absolutes in identifying the root cause, but a hypothesis should be supported with data.

Problem Statement: ELLs have a 50% pass rate in reading
Potential Root Cause: Teachers lack of understanding of instructional strategies needed to meet the linguistic needs of ELLs. Supporting Data: Classroom walkthrough data and teacher interviews reveal an absence of instructional strategies to support ELLs. Facilitator Talking Points: Before finalizing the root cause, ensure it is supported by data. In this example, the root cause is supported by classroom walkthrough data and teacher interviews.

NEEDS ASSESSMENT QUALITY CHECK
The root cause(s) identified: WHY the problem(s) is/are occurring Is/are validated by data Do(es) not contain solutions or suggested actions Is/are focused on what the team can control Facilitator Talking Points: Remember that to keep the following list in mind when arriving at the root cause. This quality check list can be used to determine as a team if the root cause “passes”

TARGETED IMPROVEMENT TEMPLATE
Average parent attendance/participation at our quarterly parent night is 25% ELLs have a 50% pass rate in reading Teachers lack of understanding the instructional strategies needed to meet the linguistic needs of ELLs There is a lack of a streamlined communication process and protocol with our external providers Facilitator Talking Points: Another quality check that is built into the template is located on the Needs Assessment tab of the template. Problem statements carry over from the data tab. Once the root cause is entered, be sure to read aloud the problem statement and root cause all at once. When read together, the way the root cause is written should sound like it’s answering why the problem exists, such as this example here. Read aloud to the group, specifically using the “because of” language connecting the problem statement and root cause. Facilitation Notes:

TARGETED IMPROVEMENT TEMPLATE
Facilitator Talking Points: Access the targeted improvement plan template. Familiarize yourselves with the flow of the tool and if the root cause is ready to be entered, do so. It is possible, the team will need to revise the root causes developed today so placing it in the tool is not required at this moment.

NEEDS ASSESSMENT NEXT STEPS
Identify necessary stakeholders to be a part of the needs assessment process Through the process, determine what data is needed for corroboration/validation of the root cause Revisit for resources on the needs assessment process Facilitator Talking Points: Review next steps. Optional Activity: If there is time, allow for teams to take a closer look at the root cause analysis protocol. Review the data for one student outcome that interests the team. What conditions in the school or district environment might be influencing this outcome? Develop some hypotheses regarding possible causation. What data could you collect to test these hypotheses? Create a brief action plan to collect that data and come back together to determine if you have appropriately identified the root cause(s) and narrowed the focus of improvement efforts. Facilitation Notes: Review the tips with the group and provide time for questions.

BREAK Facilitation Notes: 10 min break

DATA ANALYSIS Facilitator Talking Points:

Similar presentations

Presentation on theme: "DATA ANALYSIS Facilitator Talking Points:"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

DATA ANALYSIS Facilitator Talking Points:

Similar presentations

Presentation on theme: "DATA ANALYSIS Facilitator Talking Points:"— Presentation transcript:

Similar presentations

About project

Feedback