1. Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags Donna Noyes, New York Part C Program Lauren Barton, ECO at SRI International Cornelia Taylor, ECO at SRI International

2. Session Overview Why do pattern checking? What are we looking for? Example: New York Part C Data Activity: Getting started looking for patterns 2Early Childhood Outcomes Center

3. What is Pattern Checking? A process of slicing and displaying your child outcomes data in different ways Reveals relationships between variables Does this data look similar to or different from –Data for various subgroups –Overall data observed in other states –The relationship or pattern you would expect to see 3Early Childhood Outcomes Center

4. Searching for Red Flags Searching for patterns - sometimes Warning signs… Look more closely, can this be right? –Missing data? –Data quality? –Data analysis? –Program itself? Early Childhood Outcomes Center4

5. 5 Available on the ECO website www.the-eco-center.orgwww.the-eco-center.org, ECO resources, Quality Assurance, Data Quality (called the Pattern Checking Table)

6. Why Undertake Pattern Checking? Makes a case that your data are valid (or not yet) Uncovers information about the quality of data – confidence and/or concern Spotlight on areas for further investigation Provides clues about where to target resources –Quality improvement –Program improvement 6Early Childhood Outcomes Center

7. Outcomes Measurement Approach Pattern checking is a process It works with any outcomes measurement approach Ways you split the data may change Still looking for data to show the same relationships What you do with the red flags you find may differ Early Childhood Outcomes Center7

8. Caveat!! Remember this is only weighing the pig Weighing the pig does not make it fatter Need to take what you learn from the analysis and do something with it 8Early Childhood Outcomes Center

9. Using Data E vidence I nference A ction The numbers, facts What they mean from thinking about the facts in context Implications, what to do Early Childhood Outcomes Center9

10. State Example: New York Part C Program 10Early Childhood Outcomes Center

11. Pattern Checking Activity 11Early Childhood Outcomes Center

12. Example: Distribution of Progress Categories in Two States Early Childhood Outcomes Center12

13. Distribution of COSF Ratings at Entry in Two States Early Childhood Outcomes Center13

14. Progress Categories by Gender 14Early Childhood Outcomes Center ABCDE Males (n=312) 1.6%28.4%29%23%18% Females (n=299) 1.0%14%26.5%31%27.5%

15. Crosstab of Entry and Exit Ratings on Outcome 1 (could use scores) 15Early Childhood Outcomes Center Exit Outcome 1 Entry Outcome 1 1234567 17110002 2201641200 33336265810 416283914124 5226272402498 6346310664932611 720395844 3114

16. Early Childhood Outcomes Center16