1. Researchers as Partners with State Part C and Preschool Special Education Agencies in Collecting Data on Child Outcomes
Kathy Hebbeler, ECO at SRI International
Bill Brown, University of South Carolina
Beth Rous, University of Kentucky
Conference on Research Innovations in Early Intervention
San Diego, 2008

2. What we will do
Describe the federal data collection requirements for child outcome (the what and the why)
Describe how states are responding
Present examples of how researchers are working with state agencies
Early Childhood Outcomes Center

3. The pretest
Do you know the name of your state’s Part C or 619 coordinator?
Have you ever had a conversation with either of them?
Do you know that states are required to submit data on child outcomes to the federal government?
Do you know what states are required to submit?
Do you know the significance of February 3?
Early Childhood Outcomes Center

4. What we hope to accomplish
Acquaint the research community with the states’ need for good data and meaningful analysis
Identify possible roles for researchers in helping states build their capacity to collect and use data to develop better programs for children and families
Early Childhood Outcomes Center

5. Early Childhood Outcomes Center
The What and Why of What is Happening in State Agencies Related to Child Outcomes
Kathy Hebbeler
Early Childhood Outcomes Center
at SRI International

6. Federal Reporting Requirements
Early Childhood Outcomes Center

7. States are required by IDEA to:
Submit an State Performance Plan (SPP)
Applies for 6 years
Includes a variety of indicators (measures related to compliance and performance)
Indicators differ for Part C and 619 Preschool
Must be posted on state’s web site
Submit an Annual Performance Report
Contains data on performance toward SPP targets
Early Childhood Outcomes Center

8. Required data: One indicator addresses child outcomes
#3 for Part C
#7 for Part B preschool
Can access state requirements at:
Early Childhood Outcomes Center

9. Child Outcomes
Positive social emotional skills (including positive social relationships)
Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])
Use of appropriate behaviors to meet their needs
Early Childhood Outcomes Center

10. OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged peers but did not reach it
d. Improved functioning to reach a level comparable to same-aged peers
e. Maintained functioning at a level comparable to same-aged peers
3 outcomes x 5 “measures” = 15 numbers
Early Childhood Outcomes Center

11. Reporting Schedule Reported February 2007
Entry information: Age expected? Yes, No
One time requirement
Reported for children entering between July 1, 2005, and June 30, 2006
Due February 3, 2008
Data in reporting categories at exit for all children who have been in the program for at least 6 months
Must be reported for the year beginning July 1, 2006
Repeat with next year’s data in 2009, etc.
Early Childhood Outcomes Center

12. Also States are required to: Make public data reported to OSEP
Analyze state data by program (i.e., compute a through e for each program)
Make the data public by program
Early Childhood Outcomes Center

13. Why are states required to submit data on child outcomes?
Early Childhood Outcomes Center

14. Driving Force for Data on Child Outcomes Comes from the Federal Level
Government Performance and Results Act (GPRA)
Program Assessment Rating Tool (PART)
Individuals with Disabilities Education Act (IDEA)
Early Childhood Outcomes Center

15. Government Performance and Results Act (GPRA) passed in 1993
Requires goals and indicators be established for IDEA
Indicators and data collection further along for outcomes for school age population progressed faster than for EC
Previously, for early childhood data had been collected on:
Number of children served (Part C)
Settings (both Part C and 619)
Notes from OSEP’s slides: Lou
GPRA:
Passed in 1993
Applies to all federal agencies
Requires that a set of indicators be established in order to judge the effectiveness of programs meeting their goals
OSEP has established GPRA indicators for all IDEA programs, including Parts B, C, and D that are reported to Congress annually.
For the Part C indicators, OSEP has collected data on the total number of children served and the number of children served under the age of one, and on settings where services are provided to eligible children
For 619, OSEP has collected data on settings where children receive special education and related services
Thus far OSEP has not yet collected and reported data on the functional abilities of children and family outcomes
Early Childhood Outcomes Center

16. OSEP: PART evaluation results (2002)
130 programs examined in 2002; 50% programs had no performance data
Programs looking at inputs, not results
Part C and Section 619 findings:
No long-term child outcome goals or data
Need to develop a strategy to collect annual performance data in a timely manner
During the 2003 budget period, the Administration rated approximately 130 federal programs on their effectiveness using part
50% of the these programs had no performance data
Programs tend to use inputs or process data, rather than results or outcome data
Specifically for Part C and Section 619: did not do well in the area of accountability/results because there were no long-term child outcome goals; no child outcome data. The recommendation from the PART is that OSEP needs to develop a strategy to collect annual performance data (child and family outcomes) in a timely manner
Early Childhood Outcomes Center

20. Federal Funding in Millions for EI and ECSE
Early Intervention
Early Childhood Special Education
2004
$444
$387
2005
$441
$385
2006
$437
$381
2007
2008 (est)
$436
$374
Early Childhood Outcomes Center

21. President’s Commission on Excellence in Special Education (2002)
Major Recommendation 1: Focus on results – not process
“IDEA will only fulfill its intended purpose if it raises expectations for students and becomes result-oriented—not driven by process, litigation, regulation and confrontation. In short, the system must be judged by the opportunities it provides and the outcomes achieved for each child.”
Early Childhood Outcomes Center

22. Individuals with Disabilities Education Act
SEC <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.-…..…..
``(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- ``(A) improving educational results and functional outcomes for all children with disabilities;
Early Childhood Outcomes Center

23. State decisions and activities
Where are the states:
State decisions and activities
Early Childhood Outcomes Center

24. Why does a state want data on child outcomes?
Purpose
To meet provider/teacher, local and/or state need for outcome information
and
to respond to federal reporting requirements
To respond to federal reporting requirements
Early Childhood Outcomes Center

25. State Vision: Using Data as a Tool for Program Improvement
Have quality data available on an ongoing basis about multiple components of the system
Outcomes for children and families
Services provided
Personnel (types, qualifications, etc.)
Etc.
Early Childhood Outcomes Center

26. State approaches
Most states have embraced outcomes measurement and are collecting outcomes data for their own purposes.
Many states are building bigger systems than needed to produce the federal data.
Go to for more information about what other states are doing
Early Childhood Outcomes Center

27. How are states collecting child outcomes data?
Possible state approaches to collection of child data
Child Outcomes Summary Form (COSF)
Publisher’s online assessment system
Single assessment statewide
Other approaches
Early Childhood Outcomes Center

28. State approaches to measurement for Part C child outcomes
40 states using the ECO Child Outcomes Summary Form (COSF)
8 states using 1 assessment tool statewide
3 states using on-line assessment systems with the capacity to report OSEP data reports
5 states using other unique approaches
Early Childhood Outcomes Center

30. State approaches to measurement for preschool (Section 619) child outcomes
34 states using the ECO Child Outcome Summary Form
11 states using 1 assessment tool statewide
5 states using on-line assessment systems with the capacity to report OSEP data reports
7 states using other unique approaches
Early Childhood Outcomes Center

32. Where do states go from here?
Early Childhood Outcomes Center

33. Future activity for child outcomes data collection
States (and federal government) need good data on child outcomes
States cannot use the data to improve services until confident that data are valid
Right now, states need to improve data collection process
Early Childhood Outcomes Center

34. State Capacity Issue Is there a role for EI and ECSE researchers?
Need to examine data for validity
Need to analyze and interpret data for program improvement
Many states do not have capacity to analyze, interpret, and use data
Is there a role for EI and ECSE researchers?
Early Childhood Outcomes Center