Using State and Local Data to Improve Results DAC Data Analytics Using State and Local Data to Improve Results

DAC GOAL Form partnerships in states that join state and local agencies in the use of data to drive improved results

Using Data is an Iterative Process! Premises Data Use involves: Working through a Collaborative Team approach. Engaging Team in a Continuous Improvement Process. Relating the Data to specific Problem/Issue. Using Data is an Iterative Process!

Why Data Analysis? To identify strategies for continual improvement To be responsive when issues arise To avoid surprises during monitoring Script Notes Data analysis takes time and everyone in early intervention is busy. It is important we pause a minute and think about why we should take the time to engage in data analysis? As you see, it gives you the information you need to decide what steps are needed to keep moving forward and progress; It allows you to respond when issues or problems arise; and if used on an ongoing basis, it allows you to be continually aware of the status of your program. 5

Proactive Versus Reactive A process to review existing data to select priorities for program improvement. A process to determine program compliance and effectiveness. Proactive A process used to respond to a state identified problem. A process used to respond to a locally identified problem. Reactive Script Notes There are two primary purposes for data analysis. Both can be very important to use in program and system improvement planning. Both are positive. 6

Preparation Phase Step 1: Identify Relevant Data Module 1 - Identify relevant data to define/refine problem Provide overview of data quality standards Demonstrate how to visually represent data Review available data Identify relevant data based on defined/refined problem

Inquiry Phase Step 2: Conduct Data Analysis Module 2 – Conduct data analysis to generate hypothesis Review relevant data Define hypothesis Analyze data Develop analysis plan

Inquiry Phase Step 3: Test Hypothesis Module 3 – Test hypothesis to determine root cause(s) Triangulate data Discuss how to test hypothesis Determine root cause(s)

Action Phase Step 4: Plan for Improvement Module 4 – Improvement planning Discuss goal setting Review basic components of an improvement plan Give examples of good v. unacceptable components of improvement plan Develop improvement plan

Action Phase Step 5: Evaluate Progress Module 5 – Evaluate progress Discuss evaluation types, performance data & measures Differentiate between efforts v. effects

Infant & Toddler Connection of Virginia Data Analysis for System Improvement Analytic Model PREPARATION Define the purpose and the people Identify the people and the process Identify relevant Data INQUIRY Analyze the data Generate and Test Hypotheses Determine Actionable Causes ACTION Develop and Implement Improvement Plans Evaluate Progress Rebecka – Can you please make the center oval say “Analysis Process”? Thanks Script Notes As a summary, here is a visual of the three phases together- Preparation, Inquiry and Action

Data Quality Standards Question: How do we know if we are using high quality data? Answer: Data collected, submitted, analyzed, and reported must be: Useful Timely Accurate (Reliable and valid) Secure Rebecka: Flow the callouts in one at a time at a 1-2 second interval. Script Notes Making sure data are timely, accurate, secure and useful are critical to ensuring the success of your efforts. Let’s talk about what this means. 13

Steps in Preparation What is the issue? Who needs to be involved and how will they interact? How long will it take? What data are available and how will it be displayed? Taking the time to prepare carefully is important to success. In the next slides we will describe each preparation step and include an example to assist you in understanding how to prepare for the analysis process. 14

Collecting the Available Relevant Data What data are available related to the issue? State? Local? Other data sources? How many years of trend data should be reviewed? What data should be included for comparison purposes? State ? Other locality data? How can the data be disaggregated? By age of child? By service coordinator? By a time period? Do we have qualitative as well as quantitative data that relate to the issue? First, gather the state and local data that are available and relate to this issue. Then consider other data. Trend data across time might be helpful to review. Comparison data across localities can also be helpful. Disaggregating the data by factors such as individual service providers or referral sources or age of the child can also offer clues to help in your analysis.

For easy analysis how should the data be displayed? Possible ways to display data include: Column charts Bar charts Line charts Pie charts Maps There are multiple ways to display the data. How it is displayed effects how easily the team members can understand what the data mean and use it for analysis. Some displays work better than others depending on multiple factors including the type of data and the expertise of the team. 16

Begin with Setting Guidelines Remind the Team Ensure that your data are valid Ensure that your data are reliable Remind the team Follow the data where it leads you Only make data-based decisions Leave Inquiry when you have determined Actionable Cause (s) Fix 17

Generating a Hypotheses What actions in practice have contributed to local results in these areas: Infrastructure Policies and Procedures Daily Practices Monitoring and Supervision Resources

Generating Hypotheses Only consider the factors involved in the inquiry phase (i.e., relevant data) Document ideas and/or preliminary observations about why something is or is not working Write the hypothesis including the who, what, when, where and why Note: Hypothesis are tentative statements, and might be re-written several times as they are tested and re-tested. Source: hypothesis generating and testing (n.d.) Retrieved from http://www.angelfire.com/ca5/anthro/hypo.html on November 11, 2010 (SPIFF UP) 19

Hypothesis Testing Guiding Questions What procedures might have contributed to current performance? Is there any information that would lead us to reject or accept each hypothesis? Given our data picture, are there any other possible explanations we might pose? What data if any, do we still need to collect to determine our actionable causes. 20

Analysis involves organizing and understanding data based on criteria you develop; it is useful when you want to find some trend or pattern. Per Alan, See Wikipedia definition Analysis is: organizing and understanding Source: Purdue Online Writing Lab 21

Determining the Actionable Causes: Helps resolve the issue Eliminates meaningless effort Conserves resources Identifies efficient and effective strategies Leads to Improvement

The Data Analysis Process Preparation Define the purpose and the issue Identify the people and the process Identify relevant data Script Notes First is the Preparation phase in which you define the purpose of the work and carefully define the issue that will be addressed. You will decide who will be involved in the process and how the work will be accomplished. Finally, you will identify the data available to look at the issue.

The Data Analysis Process Inquiry Analyze the data Generate and Test Hypotheses Determine Actionable Cause(s) Script Notes The Inquiry phase takes you through the major analysis work including reviewing the data to develop hypotheses about the cause of current performance. This includes the gathering and consideration of more data to test these hypotheses that allows you to determine the actionable causes that need to be address for improvement. In other words – you will follow the data where it leads until you identify the major cause or causes of the current performance.

The Data Analysis Process Action Develop and Implement Improvement Plans Evaluate Progress Script Notes The third and final phase is Action in which an improvement plan is developed. This includes designing strategies based on what the data tell you. The implementation plan must include an evaluation process so that you know if the strategies being used are actually working.

“I’ll pause for a moment so you can let this information sink in.”

Discussion Questions What approaches or methods are you using at the state level to analyze your child and family outcome data to improve results for children and families? Can you share any lessons learned? Are you working with local programs to assist them in using outcome data to consider how quality practices impact child and family outcome results? If so, describe the process or tools you use to help locals examine data and practices to make changes for improvement.

Discussion Questions How might the DAC data analytic model or the Relationship of Quality Practices to Child and Family Outcome Measurement Results tool help you with these efforts? What additional information or technical assistance would be beneficial?