Looking for Patterns in Child Outcome Data – Examples from NYS New York State Department of Health Bureau of Early Intervention

July 30, 2010Measuring Child & Family Outcomes2 New York’s Child & Family Outcomes System New York’s Early Intervention Program (NYSEIP) is one of the largest in the nation – Provides services to nearly 75,000 children and families each year – 58 local programs administered by 57 county government agencies and New York City Given size and scope of the NYSEIP, a sampling approach has been implemented for collection of child and family outcome data

July 30, 2010Measuring Child & Family Outcomes3 New York’s Child & Family Outcomes System Child outcomes data are collected using the COSF Each year, local programs enroll up to a designated number of children in child outcomes samples COSF forms are completed at entry and exit for children enrolled in child outcomes samples

July 30, 2010Measuring Child & Family Outcomes4 New York’s Child & Family Outcomes System A memorandum of understanding with the University at Buffalo, School of Public Health and Health Professions, Population Health Observatory, to partner with BEI in implementing New York’s child and family outcomes system – Responsible for data entry, quality control, management, and analyses – Maintains a suballocation with the University At Binghamton, Institute for Child Development, for materials development, training and technical assistance, and annual family survey

July 30, 2010Measuring Child & Family Outcomes5 New York’s Child & Family Outcomes System Special thanks to University at Buffalo, PHO staff who collaborated on the analyses and presentation – Li Yan, Ph.D., M.A., Data Manager – Amy Barczykowski, M.S.Ed., Project Coordinator – Randy Carter, Ph.D., Director of PHO Our partners at University at Binghamton staff are – Ray Romanczyk, Ph.D., Director, Institute for Child Development – Emily Huber-Callahan – Jamie Silverio

July 30, 2010Measuring Child & Family Outcomes6 Pattern 1a Children will differ from one another in their entry scores in reasonable ways (few scores at the high and low ends and more in the middle)

July 30, 2010Measuring Child & Family Outcomes7 Pattern 1a Entry Scores Across All 3 Outcome Areas Outcome 1 Positive Social Emotion Skills Outcome 2 Acquire and Use Knowledge and Skills Outcome 3 Take Appropriate Action to Meet Needs

July 30, 2010Measuring Child & Family Outcomes8 Pattern 1b Children will differ from one another in their exit scores in reasonable ways

July 30, 2010Measuring Child & Family Outcomes9 Pattern 1b Exit Scores Across all 3 Outcome Areas Outcome 1 Positive Social Emotion Skills Outcome 2 Acquire and Use Knowledge and Skills Outcome 3 Take Appropriate Action to Meet Needs

July 30, 2010Measuring Child & Family Outcomes10 Pattern 1c Children will differ from one another in their OSEP Progress Categories in reasonable ways

July 30, 2010Measuring Child & Family Outcomes11 Pattern 1c OSEP Categories Across all 3 Outcome Areas aChildren who did not improve functioning b Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers cChildren who improved functioning to a level nearer to same-aged peers but did not reach it dChildren who improved functioning to reach a level comparable to same-aged peers eChildren who maintained functioning at a level comparable to same-aged peers

July 30, 2010Measuring Child & Family Outcomes12 Pattern 2 Functioning in one outcome area will be related to functioning in other outcome areas

July 30, 2010Measuring Child & Family Outcomes13 Pattern 2 Comparisons Between Outcome 1 and Outcome 2 Entry Outcome 1 x Outcome 2 Outcome 1Outcome TOTAL TOTAL

July 30, 2010Measuring Child & Family Outcomes14 Pattern 2 Comparisons Between Outcome 1 and Outcome 2 Exit Outcome 1 x Outcome 2 Outcome 1Outcome TOTAL TOTAL

July 30, 2010Measuring Child & Family Outcomes15 Pattern 2 Comparisons Between Outcome 1 and Outcome 2 OSEP Categories Outcome 1 and Outcome 2 OSEP Scores Outcome 1 x Outcome 2 Outcome 1 Outcome 2 abcdeTOTAL a b c d e TOTAL

July 30, 2010Measuring Child & Family Outcomes16 Pattern 3 Functioning at entry within one outcome area will be related to functioning at exit (children who have higher functioning at entry in an area will be the ones who are high functioning at exit in that area)

July 30, 2010Measuring Child & Family Outcomes17 Pattern 3 Comparisons Between Entry Outcome 2 and Exit Outcome 2 Entry and Exit Entry Outcome 2 by Exit Outcome 2 Entry Outcome 2 Exit Outcome TOTAL TOTAL

July 30, 2010Measuring Child & Family Outcomes18 Pattern 4 Most children will either hold their developmental trajectories or improve their trajectories from entry to exit

July 30, 2010Measuring Child & Family Outcomes19 Pattern 4 Outcome 2 Exit Score-Entry Score Outcome 2 Acquire and Use Knowledge and Skills

July 30, 2010Measuring Child & Family Outcomes20 Pattern 5 Entry, exit, and OSEP progress category distributions should be similar from year to year

July 30, 2010Measuring Child & Family Outcomes21 Pattern 5 Outcome 2 Entry Scores Across 4 Cohort Years Outcome 2 Acquire and Use Knowledge and Skills

July 30, 2010Measuring Child & Family Outcomes22 Pattern 5 Outcome 2 Exit Scores Across 3 Exiting Periods Outcome 2 Acquire and Use Knowledge and Skills

July 30, 2010Measuring Child & Family Outcomes23 Pattern 5 Outcome 2 OSEP Categories Across 3 Entry Cohort Years aChildren who did not improve functioning b Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers cChildren who improved functioning to a level nearer to same-aged peers but did not reach it dChildren who improved functioning to reach a level comparable to same-aged peers eChildren who maintained functioning at a level comparable to same-aged peers

July 30, 2010Measuring Child & Family Outcomes24 Pattern 7 Entry and exit scores and OSEP categories should be related to the nature of the child’s disability

July 30, 2010Measuring Child & Family Outcomes25 Pattern 7 Children with Only a Communication Delay ENTRY Scores for 3 Outcome Areas Outcome 1 Positive Social Emotion Skills Outcome 2 Acquire and Use Knowledge and Skills Outcome 3 Take Appropriate Action to Meet Needs

July 30, 2010Measuring Child & Family Outcomes26 Pattern 7 Children with Only a Communication Delay EXIT Scores for 3 Outcome Areas Outcome 1 Positive Social Emotion Skills Outcome 2 Acquire and Use Knowledge and Skills Outcome 3 Take Appropriate Action to Meet Needs

July 30, 2010Measuring Child & Family Outcomes27 Pattern 7 OSEP Categories for Children with Only a Communication Delay aChildren who did not improve functioning b Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers cChildren who improved functioning to a level nearer to same-aged peers but did not reach it dChildren who improved functioning to reach a level comparable to same-aged peers eChildren who maintained functioning at a level comparable to same-aged peers

July 30, 2010Measuring Child & Family Outcomes28 Pattern 8 Scores at entry and exit should not be related to certain characteristics

July 30, 2010Measuring Child & Family Outcomes29 Pattern 8 Outcome 2 Percentage of Male and Female Entry Scores Outcome 2 Acquire and Use Knowledge and Skills

July 30, 2010Measuring Child & Family Outcomes30 Pattern 8 Outcome 2 Medicaid Status and Entry Scores Outcome 2 Acquire and Use Knowledge and Skills

July 30, 2010Measuring Child & Family Outcomes31 Conclusion/Next Steps Pattern checking is a useful way to begin the data quality assurance/quality improvement process Analyzing the data in these ways has helped NY target areas for improvement Planning/designing a webinar to share data with local programs Identifying other strategies to provide training and technical assistance to programs, providers, and families