1. Researchers as Partners with State Part C and Preschool Special Education Agencies in Collecting Data on Child Outcomes Kathy Hebbeler, ECO at SRI International Bill Brown, University of South Carolina Beth Rous, University of Kentucky Conference on Research Innovations in Early Intervention San Diego, 2008

2. Early Childhood Outcomes Center2 What we will do Describe the federal data collection requirements for child outcome (the what and the why) Describe how states are responding Present examples of how researchers are working with state agencies

3. Early Childhood Outcomes Center3 The pretest 1.Do you know the name of your state’s Part C or 619 coordinator? 2.Have you ever had a conversation with either of them? 3.Do you know that states are required to submit data on child outcomes to the federal government? 4.Do you know what states are required to submit? 5.Do you know the significance of February 3?

4. Early Childhood Outcomes Center4 What we hope to accomplish Acquaint the research community with the states’ need for good data and meaningful analysis Identify possible roles for researchers in helping states build their capacity to collect and use data to develop better programs for children and families

5. The What and Why of What is Happening in State Agencies Related to Child Outcomes Kathy Hebbeler Early Childhood Outcomes Center at SRI International

6. Early Childhood Outcomes Center6 Federal Reporting Requirements

7. Early Childhood Outcomes Center7 States are required by IDEA to: Submit an State Performance Plan (SPP) Applies for 6 years Includes a variety of indicators (measures related to compliance and performance) Indicators differ for Part C and 619 Preschool Must be posted on state’s web site Submit an Annual Performance Report Contains data on performance toward SPP targets

8. Early Childhood Outcomes Center8 Required data: One indicator addresses child outcomes #3 for Part C #7 for Part B preschool Can access state requirements at: http://www.rrfcnetwork.org/content/view/ 248/358/

9. Early Childhood Outcomes Center9 Child Outcomes Positive social emotional skills (including positive social relationships) Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) Use of appropriate behaviors to meet their needs

10. Early Childhood Outcomes Center10 OSEP Reporting Categories Percentage of children who: a.Did not improve functioning b.Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c.Improved functioning to a level nearer to same- aged peers but did not reach it d.Improved functioning to reach a level comparable to same-aged peers e.Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers

11. Early Childhood Outcomes Center11 Reporting Schedule Reported February 2007 Entry information: Age expected? Yes, No One time requirement Reported for children entering between July 1, 2005, and June 30, 2006 Due February 3, 2008 Data in reporting categories at exit for all children who have been in the program for at least 6 months Must be reported for the year beginning July 1, 2006 Repeat with next year’s data in 2009, etc.

12. Early Childhood Outcomes Center12 Also States are required to: Make public data reported to OSEP Analyze state data by program (i.e., compute a through e for each program) Make the data public by program

13. Early Childhood Outcomes Center13 Why are states required to submit data on child outcomes?

14. Early Childhood Outcomes Center14 Driving Force for Data on Child Outcomes Comes from the Federal Level Government Performance and Results Act (GPRA) Government Performance and Results Act (GPRA) Program Assessment Rating Tool (PART) Program Assessment Rating Tool (PART) Individuals with Disabilities Education Act (IDEA) Individuals with Disabilities Education Act (IDEA)

15. Early Childhood Outcomes Center15 Requires goals and indicators be established for IDEA Indicators and data collection further along for outcomes for school age population progressed faster than for EC Previously, for early childhood data had been collected on: Number of children served (Part C) Settings (both Part C and 619) Government Performance and Results Act (GPRA) passed in 1993

16. Early Childhood Outcomes Center16 130 programs examined in 2002; 50% programs had no performance data Programs looking at inputs, not results Part C and Section 619 findings:  No long-term child outcome goals or data  Need to develop a strategy to collect annual performance data in a timely manner OSEP: PART evaluation results (2002)

20. Early Childhood Outcomes Center20 Federal Funding in Millions for EI and ECSE Early Intervention Early Childhood Special Education 2004$444$387 2005$441$385 2006$437$381 2007$437$381 2008 (est) $436$374

21. Early Childhood Outcomes Center21 President’s Commission on Excellence in Special Education (2002) Major Recommendation 1: Focus on results – not process “IDEA will only fulfill its intended purpose if it raises expectations for students and becomes result-oriented—not driven by process, litigation, regulation and confrontation. In short, the system must be judged by the opportunities it provides and the outcomes achieved for each child.”

22. Early Childhood Outcomes Center22 SEC. 616. > MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.- …..….. ``(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- ``(A) improving educational results and functional outcomes for all children with disabilities; Individuals with Disabilities Education Act

23. Early Childhood Outcomes Center23 Where are the states: State decisions and activities

24. Early Childhood Outcomes Center24 To respond to federal reporting requirements To meet provider/teacher, local and/or state need for outcome information and to respond to federal reporting requirements Purpose Why does a state want data on child outcomes?

25. Early Childhood Outcomes Center25 State Vision: Using Data as a Tool for Program Improvement Have quality data available on an ongoing basis about multiple components of the system Outcomes for children and families Services provided Personnel (types, qualifications, etc.) Etc.

26. Early Childhood Outcomes Center26 State approaches Most states have embraced outcomes measurement and are collecting outcomes data for their own purposes. Many states are building bigger systems than needed to produce the federal data. Go to www.the-eco-center.org for more information about what other states are doing

27. Early Childhood Outcomes Center27 How are states collecting child outcomes data? Possible state approaches to collection of child data Child Outcomes Summary Form (COSF) Publisher’s online assessment system Single assessment statewide Other approaches

28. Early Childhood Outcomes Center28 State approaches to measurement for Part C child outcomes 40 states using the ECO Child Outcomes Summary Form (COSF) 40 states using the ECO Child Outcomes Summary Form (COSF) 8 states using 1 assessment tool statewide 8 states using 1 assessment tool statewide 3 states using on-line assessment systems with the capacity to report OSEP data reports 3 states using on-line assessment systems with the capacity to report OSEP data reports 5 states using other unique approaches 5 states using other unique approaches

30. Early Childhood Outcomes Center30 State approaches to measurement for preschool (Section 619) child outcomes 34 states using the ECO Child Outcome Summary Form 34 states using the ECO Child Outcome Summary Form 11 states using 1 assessment tool statewide 11 states using 1 assessment tool statewide 5 states using on-line assessment systems with the capacity to report OSEP data reports 5 states using on-line assessment systems with the capacity to report OSEP data reports 7 states using other unique approaches 7 states using other unique approaches

32. Early Childhood Outcomes Center32 Where do states go from here?

33. Early Childhood Outcomes Center33 Future activity for child outcomes data collection States (and federal government) need good data on child outcomes States cannot use the data to improve services until confident that data are valid Right now, states need to improve data collection process

34. Early Childhood Outcomes Center34 State Capacity Issue Need to examine data for validity Need to analyze and interpret data for program improvement Many states do not have capacity to analyze, interpret, and use data Is there a role for EI and ECSE researchers?