1. OSEP Project Directors Meeting
Child Outcomes for Early Intervention and Preschool Special Education: The National Picture
Kathy Hebbeler
ECO at SRI
Lynne Kahn
ECO at FPG
Presented at
OSEP Project Directors Meeting
Washington, DC
July, 2012 1

2. Early Childhood Outcomes Center
Overview
National numbers and the quality of the state data
The findings for
Findings for the last 3 years
State variation and data quality
Early Childhood Outcomes Center

3. OSEP Reporting Requirements: Child Outcomes
Positive social emotional skills (including positive social relationships)
Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])
Use of appropriate behaviors to meet their needs
Early Childhood Outcomes Center

4. Early Childhood Outcomes Center
If you have not seen it….
Early Childhood Outcomes Center

5. State Approaches to Outcomes Data, 2010-11
Part C
(56 states/jurisdictions)
Preschool
(59 states/jurisdictions)
COS* 7 pt. scale
43/56 (77%)
36/59 (61%)
One tool statewide
7/56 (13%)
9/59 (15%)
Publishers’ online analysis
3/56 (5%)
6/59 (10%)
Other
5/56 (9%)
8/59 (14%)
Part C, one tool: four are using the Battelle Developmental Inventory (BDI)/Battelle Developmental Inventory, Second Edition (BDI-2), two are using the Assessment, Planning, and Evaluation System (AEPS), and one is using the Oregon. One state (2%) is using publishers’ online analysis and reporting systems where local programs choose from three assessments: High Scope, Creative Curriculum, or AEPSi (although they are moving toward just one assessment system, GOLD).
Part B, one tool: four reported the use of the Battelle Developmental Inventory, Second Edition (BDI-2), one state named the Assessment, Evaluation, and Planning System (AEPS), one state uses the Work Sampling System (WSS), and one uses selected subtests of the Brigance Inventory of Early Development II. Two states have developed their own assessment tools.
*Child Outcomes Summary Rating

6. Child Outcomes: What we know
States are in various stages with regard to building their child outcomes measurement systems
Some phrased in
Some changed approaches
Early Childhood Outcomes Center

7. 3 Methods Methods for Calculating National Estimates
States with the highest quality data weighted by child count to represent all states*
All states weighted by child count
All states averaged (each state weighted as 1)
*The data we will be presenting for the national picture
Early Childhood Outcomes Center

8. Identifying the States with Highly Questionable Data
Criteria for eliminating states:
Not reporting data on enough children
Odd patterns in the data
Review of method reported suggested the data were of questionable quality
Early Childhood Outcomes Center

9. Continued increase in the number of states included
Number of States that Met Criteria for Inclusion in the National Analysis
08-09
09-10
10-11
Pt. C 19 29 39
Part B Preschool 15 33 36
Early Childhood Outcomes Center

10. Early Childhood Outcomes Center
And now the data….
And now the data….
Early Childhood Outcomes Center

11. OSEP Reporting Categories
Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers
3 outcomes x 5 “measures” = 15 numbers
Early Childhood Outcomes Center

13. *Based on 39 states with highest quality data
The number of children reported by states ranged from 22 to 10,274.
*Based on 39 states with highest quality data

14. *Based on 36 States with the highest quality data.

15. The Summary Statements
Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program.
The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program.
States set targets on the Summary Statements
Early Childhood Outcomes Center

16. *Based on 39 states with highest quality data

17. *Based on 36 States with the highest quality data.

18. Part C: Trends over Time
Early Childhood Outcomes Center

19. Part C: Trends over Time
Early Childhood Outcomes Center

20. Part B Preschool: Trends over Time
Early Childhood Outcomes Center

21. Part B Preschool: Trends over Time
Early Childhood Outcomes Center

22. Early Childhood Outcomes Center
Looking for Patterns
Early Childhood Outcomes Center

23. Early Childhood Outcomes Center

24. Early Childhood Outcomes Center%
Early Childhood Outcomes Center

25. Early Childhood Outcomes Center

26. Early Childhood Outcomes Center
Building the national capacity to report and use child outcomes data is a long term endeavor.
Early Childhood Outcomes Center

27. Part C: States continue to report on more children
Total Number of Children States Included in the Child Outcomes Data
Number of children reported
Number of States and Jurisdictions
(N=56)
(N=54)
99 or less 13 6 5 4 25 16 9 10 7
1,000-1,999 11
2,000+ 3 20 22
the number of children included in the data ranged from 17 to 16,639 children
Early Childhood Outcomes Center

28. Early Childhood Outcomes Center

29. Part B Preschool: States continue to report on more children
Total Number of Children Included in Progress Data
Number of children reported
Number of States and Jurisdictions
FFY 2007
(N=58)
FFY 2008
FFY 2009
FFY 2010 (N=59)
10 or fewer 1
10-99 11 7 6
100–499 14 4 2 10 12 8
1000–1999 9 5
2000–2999
3000–4999 13 3
9000+
Early Childhood Outcomes Center

30. Early Childhood Outcomes Center

31. Early Childhood Outcomes Center%
Early Childhood Outcomes Center

32. Early Childhood Outcomes Center

33. Early Childhood Outcomes Center
Conclusions
The data show good outcomes for many children who attend Part C and Part B, Preschool.
The data quality continues to get better.
The data will be a powerful tool for program improvement when
The data are consistently good at the local level
States and local programs consistently use it.
Early Childhood Outcomes Center

34. Early Childhood Outcomes Center
For more information….
Early Childhood Outcomes Center

35. Early Childhood Outcomes Center

36. Early Childhood Outcomes Center
Identifying States with Quality Data
Early Childhood Outcomes Center

37. Calculating Missing Data for Part C
Proxy for missing data = Number with data for C3/ Exiting Data (618)
Do not expect this number to be 100%
..but we don’t expect it to be 10% either
Early Childhood Outcomes Center

38. Percentage of Exiters Reported by Part C Continues to Increase
Early Childhood Outcomes Center

39. Calculating Missing Data for 619
Proxy for missing data = Number with data for B7/ Child count
Do not expect this number to be 100%
..but we don’t expect it to be 10% either
Early Childhood Outcomes Center

40. Early Childhood Outcomes Center
Percentage of Child Count Reported by Part B Preschool Continues to Increase
Early Childhood Outcomes Center

41. Many states reporting a low percentage for “a” but still have a few outliers.
Number of States by % Reported in “a,”
Part C
Part B Preschool
<2% 32 30
2 to <5% 11
5 to <7% 5 3
7 to <13%
28 to 49%
Early Childhood Outcomes Center

42. Early Childhood Outcomes Center
Reason Part C state was excluded
Number excluded
Percent
(n=51)
State is sampling 3 6
No outcomes data reported
Reported outcomes data on less than 28% of reported exiters
Had at least one outcome with category a greater than 10% or category e greater than 65% 4 8
AND 1 2
Questionable data quality based on review of SPP/APR and knowledge gained through TA
States included in the analysis 39 76
Early Childhood Outcomes Center

43. Early Childhood Outcomes Center
Reason Part B state was excluded
Number excluded
Percent (n=51)
State is sampling 4 8
No outcomes data reported 1 2
Reported outcomes data on less than 12% of child count
Had at least one outcome with category a greater than 10% or category e greater than 65%
Reported outcomes data on less than 12% of child count
AND
Questionable data quality based on review of SPP/APR and knowledge gained through TA
No child count data available
Reported Summary statements but no progress categories
States included in the analysis 36 71
Early Childhood Outcomes Center

44. Early Childhood Outcomes Center
Producing National Estimates Using the States with the Highest Quality Data
Divide the states into those with large, medium, and small child counts.
Identify the states that appear to have the best data.
Weight by child count and calculate the overall OSEP percentages using the “best data” states in each category.
Use the OSEP percents and total child count for each category to get the national estimates.
Early Childhood Outcomes Center