Trends in Child Outcomes (C-3 / B-7) and Family Outcomes (C-4) Analysis and Summary Report of All States’ 2008-2009 Annual Performance Reports Christina Kasprzak, Robin Rooney, Siobhan Colgan Lynne Kahn, Kathy Hebbeler (NECTAC / ECO) November 30, 2010 4:00 PM EST

Webinar Focus 1. National data on child and family outcomes (C3/B7 and C4) 2. Challenges related to collecting and reporting on this indicator 3. Improvement activities

Who is joining us on the call today? Quick Poll 1 Who is joining us on the call today?

Child Outcomes Data Summary of 2010 APR Data

Part C State Approaches (n=56) to Measuring Child Outcomes Child Outcomes Summary Form (COSF) 41 (73%) states Single assessment statewide 7 (13%) states Publishers’ online assessment systems 3 (5%) states Other approaches 5 (9%) states

State Approaches to Child Outcomes Measurement Part C Program Early Childhood Outcomes Center –August 2010 MP GU Legend: COSF Publishers’ on-line systems One tool statewide Other AS HI

619 State Approaches (n=59) to Measuring Child Outcomes Child Outcomes Summary Form (COSF) 36 (61%) states Single assessment statewide 9 (15%) states Publishers’ online assessment systems 6 (10%) states Other approaches 7 (12%) states 7

State Approaches to Child Outcomes Measurement Section 619 Programs Early Childhood Outcomes Center –August 2010 MP GU MH Legend: COSF Publishers’ on-line systems One tool statewide Other PW FM AS HI

The number of Part C children with outcome data is increasing! Part C Total Number of Children States Included in Progress Data # of children reported # of States FFY07 FFY08 99 or less 13 6 100-499 25 16 500-999 1000-1999 9 11 2000+ 3 10 Range = 5-6452 Range = 11-7998

The number of 619 children with outcome data is increasing! 619 Total Number of Children States Included in Progress Data # of children reported # of States FFY07 FFY08 99 or less 12 8 100-499 14 6 500-999 10 1000-1999 7 2000-2999 5 3000+ 9 15 Range= 3-10,157 Range= 3-9,967

Number of children in the data Quick Poll 2 Number of children in the data

Missing Data ECO additional analysis State efforts to identify missing data State efforts to reduce missing data

National Conference Call on Data Quality – Coming Soon What do you know the quality of your state’s outcomes data? Do you know how much missing data you have? How much is reasonable? Missing data is still a major problem for many states. Join us to learn about how much progress has been made and how your state compares to the national numbers.

Part C Progress data trends FFY07 FFY08 Part C Progress data trends

619 Progress data trends FFY07 FFY08

Quick Poll 3 Progress Data Trends

Part C - Category ‘e’ by % served

FFY2008 Summary Statements Baseline FFY08 Part C 619 SS1 SS2 Outcome 1 63% 76% 61% Outcome 2 68% 54% 53% Outcome 3 69% 62% 75% 65%

Part C - Summary Statement 2 by % Served

What States are Doing for Improvement Continuing training and TA on data collection system Enhancing data systems Developing data analysis Identifying and addressing data quality issues Identifying areas for program improvement

Open Discussion Questions? Comments?

Family Outcomes Data Summary of 2010 APR Data

State Approaches to Family Outcomes Measurement Part C Program Early Childhood Outcomes Center – August 2010 HI GU AS MP Legend: ECO Family Outcomes Survey State-developed survey NCSEAM survey

Survey timing and family population Total All families Families with ≥6 months* Point in time (or time period) 25 12 37 Based on child participation 8 9 17 -at exit (3) (6) -at IFSP meeting(s) (2) -IFSP and/or exit (1) 33 21 54 *One State used ≥9 months, and one State used ≥12 months

Quick Poll 4 Survey distribution

Response rate variables

Response rate by state size

Representativeness: Comparison data Thirty-nine states (70%) reported the source of data used: Part C population/ 618 data: 31 states Program population data: 3 states Target population: 3 states State data (not specified): 2 states Remaining 17 states did not specify

Addressing representativeness Quick Poll 5 Addressing representativeness

Criteria used for evaluating representativeness Forty-six states (89%) reported the criteria they used for determining representativeness Race/ ethnicity: 73% (41 states) Geography (district, county, region): 50% (28 states) Sex: 21% (12 states) Child’s age: 20% (11 states) Disability/ eligibility category: 9% (5 states) Length of time in services: 9% (5 states) Program size : 9% (5 states) Previous years: 2009: 44/56 (78.6%) and 2008: 37/56 (66.1%) reported criteria used Mean number of criteria used this year: 2.7 Previous years: 2009: 2.6 criteria and 2008: 2 criteria.

Performance and trends Early intervention has helped… Families know their rights: 84% +3% from last year Families effectively communicate children’s needs: 85% +2% from last year Families help their children develop and learn: 90% +2% from last year

Performance by Survey Type

Performance by scoring criteria

Performance by state size

Performance by percent served

Were data representative? Forty-four states reported whether their data were representative (79%) Yes, some data provided: 36% (20 states) Yes, no data provided: 14% (8 states) No: 11% (6 states) Varied results: 18% (10 states) No conclusions re: representativeness reported among the remaining 12 states (21%)

State Highlights Analyzing and reporting outcomes among subgroups Improvement activities based on detailed analysis (e.g. by subgroups of families) Data collection improvement activities Partnering w/parent organizations

Suggested Formats for February 2011 APR Reporting http://www.fpg.unc.edu/~eco/pages/ fed_req.cfm#ECOSuggestedFormats (this link is also available from the webinar series page)

Open Discussion Questions? Comments?

Full APR analysis Reports are Available Online Part C: http://www.nectac.org/~pdfs/calls/2010/partcapr/2010_spp_partc_report.pdf Part B: (these links are also available from the webinar series page)

Trends in Child Outcomes (C-3 / B-7) and Family Outcomes (C-4) Thank you for participating. Presentations from this series and their related resources are made available on the NECTAC website at: http://www.nectac.org/~calls/2010/partcapr/partcapr.asp