Integrating Outcomes Learning Community June 12, 2013 Using Child Outcomes Data

Using Data for Improvement Linking outcomes data to other data Helping locals look at data Examining practices and how they relate to outcomes 2

Linking Outcomes Data to other Data Linking child outcomes data to... Child/Family characteristics, e.g. Race/ethnicity, income, disability type Service data, e.g. Amount of services, type of services provided Program data Geographic location or specific program characteristics What kinds of questions might these answer to inform program improvement efforts? Does our program serve some children more effectively than others? E.g. are there differences by race/ethnicity, income, disability type, home language, other? And, if so, why? That would be the next level – interpreting the data Are outcomes better for children who receive more services? Certain types of services? Do outcomes differ across local programs? E.g. a comparison across local programs vs. a comparison of programs with different characteristics What kind of characteristics might you compare? programs with higher quality? (any quality data? programs with more experienced staff? programs with more or certain training (e.g. TACSEI) caseload size turnover teaming approach other? 3

Child/Family Characteristics In your state, what child/family characteristics data are you linking to your child outcomes data? What have you learned? What could you potentially link?

Child/Family Characteristics In your state, what child/family characteristics data are you linking to your child outcomes data? What have you learned? What could you potentially link?

Service Data In your state, what service data are you linking to your child outcomes data? What have you learned? What could you potentially link?

Program Data In your state, what program data are you linking to your child outcomes data? What have you learned? What could you potentially link?

What other data are you linking to your child outcomes data? What are your data telling you?

Helping Local Programs Look at Child Outcomes Data Examples of states... What efforts/initiatives are going on in your state to help local programs look at their child outcomes data? 9

Tools linking outcomes and practices Local Contributing Factors Tool Relationship of Quality Practices to Child and Family Outcome Measurement Results 10

Local Contributing Factors Tool What do you like about the tool? What do you think is useful about the tool? How might you use the tool in your state? 11

Relationships What do you like about the tool? What do you think is useful about the tool? How might you use the tool in your state? 12

REMEMBER! You want to trust your data BEFORE You draw conclusions and take action based on that data! 13

High Quality Child Outcomes Data Looking at statewide data quality: Completeness of the data (% of children in the data) Patterns in the data 14

Completeness of the Data Denominator Children exiting in the reporting period, with at least 6 months of service Numerator Of the children in the denominator, the children with entry and exit outcomes data GOAL: 100% What % is your state currently reporting? 15

Completeness of the Data ECO National Analysis (proxy) Part C: outcomes data available on at least 40% of children who exit Part C Annually, more and more states are meeting the standard. 16

Looking for patterns in the data ECO National Analysis Odd patterns in the data E.g. State % in ‘a’ is not overly high (> 5%) State % in ‘e’ is not overly low (< 5%) or high (> 65%) Annually, more and more states are meeting the standard. 17

Looking for patterns in the data What might be other ‘odd’ patterns in the a-e data? State % in ‘b’ is not overly low (< 5%) or high (> 50%) State % in ‘c’ is not overly low (< 5%) or high (> 50%) State % in ‘d’ is not overly low (< 5%) or high (> 50%) Does your state have any ‘odd’ patterns in the data? 18

Looking for patterns in the data States’ capacity to do more pattern checking than nationally available E.g. By entry scores By exit scores By disability By geography By age at entry Other 19

Looking for patterns in the data The quality of the child outcomes data are established by a series of analyses that demonstrate the data are showing predictable patterns. Pattern Checking Table http://www.fpg.unc.edu/~eco/assets/pdfs/Pattern_Checking_Table.pdf Pattern checking is a good strategy for checking data quality. ECO resource document available on the web. Provides information about outcomes data patterns that a state (or local program) might want to analyze to look at their data.

High Quality Child Outcomes Data Looking at LOCAL data: Completeness of the data (% of children in the data) Patterns in the data 21

Data Quality Efforts? What are you doing related to checking data quality? What strategies are you planning to use to ensure data quality over time? How could integrating outcomes into the IFSP or IEP help with data quality efforts?

Improving Data Quality Training - e.g. initial/orientation, refresher, regular opportunities ECO resources – professional development self-directed learning TA – e.g. ongoing support, mentoring, supervision with feedback Guidance materials – e.g. manuals, early learning guidelines 23

Improving Data Quality Data analysis and use – e.g. pattern checking at state and local levels ECO resources Pattern checking document (and new one coming!) PD – “Looking at Data” Calculators and Graphing templates 24

How is my state’s data with regard to... Completeness of the outcomes data? Do we have the ability to identify the true denominator for this calculation? What % of exiters with 6 mo service have child outcomes data? Are we looking at ‘missing data’ issues for local programs/districts? What, if anything, do we need to work on? What strategies have we used to reduce missing data? 25

How is my state’s data with regard to... Data patterns for a-e? Do we have ‘a’ greater than 5%? Do we have ‘e’ greater than 65%? What other ‘odd’ patterns in the data have we examined? Are we looking at ‘data patterns’ for local programs/districts? What, if anything, do we need to work on in my state? What strategies have we used to identify and investigate odd patterns in the data? 26

Additional Questions/Comments?