1. Integrating Outcomes Learning Community June 12, 2013
Using Child Outcomes Data

2. Using Data for Improvement
Linking outcomes data to other data
Helping locals look at data
Examining practices and how they relate to outcomes 2

3. Linking Outcomes Data to other Data
Linking child outcomes data to...
Child/Family characteristics, e.g.
Race/ethnicity, income, disability type
Service data, e.g.
Amount of services, type of services provided
Program data
Geographic location or specific program characteristics
What kinds of questions might these answer to inform program improvement efforts?
Does our program serve some children more effectively than others?
E.g. are there differences by race/ethnicity, income, disability type, home language, other?
And, if so, why? That would be the next level – interpreting the data
Are outcomes better for children who receive more services? Certain types of services?
Do outcomes differ across local programs?
E.g. a comparison across local programs vs. a comparison of programs with different characteristics
What kind of characteristics might you compare?
programs with higher quality? (any quality data? programs with more experienced staff?
programs with more or certain training (e.g. TACSEI)
caseload size
turnover
teaming approach
other? 3

4. Child/Family Characteristics
In your state, what child/family characteristics data are you linking to your child outcomes data?
What have you learned?
What could you potentially link?

5. Child/Family Characteristics
In your state, what child/family characteristics data are you linking to your child outcomes data?
What have you learned?
What could you potentially link?

6. Service Data
In your state, what service data are you linking to your child outcomes data?
What have you learned?
What could you potentially link?

7. Program Data
In your state, what program data are you linking to your child outcomes data?
What have you learned?
What could you potentially link?

8. What other data are you linking to your child outcomes data?
What are your data telling you?

9. Helping Local Programs Look at Child Outcomes Data
Examples of states...
What efforts/initiatives are going on in your state to help local programs look at their child outcomes data? 9

10. Tools linking outcomes and practices
Local Contributing Factors Tool
Relationship of Quality Practices to Child and Family Outcome Measurement Results 10

11. Local Contributing Factors Tool
What do you like about the tool?
What do you think is useful about the tool?
How might you use the tool in your state? 11

12. Relationships What do you like about the tool?
What do you think is useful about the tool?
How might you use the tool in your state? 12

13. REMEMBER! You want to trust your data BEFORE
You draw conclusions and take action based on that data! 13

14. High Quality Child Outcomes Data
Looking at statewide data quality:
Completeness of the data (% of children in the data)
Patterns in the data 14

15. Completeness of the Data
Denominator
Children exiting in the reporting period, with at least 6 months of service
Numerator
Of the children in the denominator, the children with entry and exit outcomes data
GOAL: 100%
What % is your state currently reporting? 15

16. Completeness of the Data
ECO National Analysis (proxy)
Part C: outcomes data available on at least 40% of children who exit Part C
Annually, more and more states are meeting the standard. 16

17. Looking for patterns in the data
ECO National Analysis
Odd patterns in the data
E.g.
State % in ‘a’ is not overly high (> 5%)
State % in ‘e’ is not overly low (< 5%) or high (> 65%)
Annually, more and more states are meeting the standard. 17

18. Looking for patterns in the data
What might be other ‘odd’ patterns in the a-e data?
State % in ‘b’ is not overly low (< 5%) or high (> 50%)
State % in ‘c’ is not overly low (< 5%) or high (> 50%)
State % in ‘d’ is not overly low (< 5%) or high (> 50%)
Does your state have any ‘odd’ patterns in the data? 18

19. Looking for patterns in the data
States’ capacity to do more pattern checking than nationally available
E.g.
By entry scores
By exit scores
By disability
By geography
By age at entry
Other 19

20. Looking for patterns in the data
The quality of the child outcomes data are established by a series of analyses that demonstrate the data are showing predictable patterns.
Pattern Checking Table
Pattern checking is a good strategy for checking data quality.
ECO resource document available on the web. Provides information about outcomes data patterns that a state (or local program) might want to analyze to look at their data.

21. High Quality Child Outcomes Data
Looking at LOCAL data:
Completeness of the data (% of children in the data)
Patterns in the data 21

22. Data Quality Efforts?
What are you doing related to checking data quality?
What strategies are you planning to use to ensure data quality over time?
How could integrating outcomes into the IFSP or IEP help with data quality efforts?

23. Improving Data Quality
Training - e.g. initial/orientation, refresher, regular opportunities
ECO resources –
professional development
self-directed learning
TA – e.g. ongoing support, mentoring, supervision with feedback
Guidance materials – e.g. manuals, early learning guidelines 23

24. Improving Data Quality
Data analysis and use – e.g. pattern checking at state and local levels
ECO resources
Pattern checking document (and new one coming!)
PD – “Looking at Data”
Calculators and Graphing templates 24

25. How is my state’s data with regard to...
Completeness of the outcomes data?
Do we have the ability to identify the true denominator for this calculation?
What % of exiters with 6 mo service have child outcomes data?
Are we looking at ‘missing data’ issues for local programs/districts?
What, if anything, do we need to work on?
What strategies have we used to reduce missing data? 25

26. How is my state’s data with regard to...
Data patterns for a-e?
Do we have ‘a’ greater than 5%?
Do we have ‘e’ greater than 65%?
What other ‘odd’ patterns in the data have we examined?
Are we looking at ‘data patterns’ for local programs/districts?
What, if anything, do we need to work on in my state?
What strategies have we used to identify and investigate odd patterns in the data? 26

27. Additional Questions/Comments?