1 Early Childhood and Accountability OSEP’s Project Director’s Meeting August 2006

2 Jennifer Tschantz, OSEP Lou Danielson, OSEP Kathleen Hebbeler, SRI International Corry Robinson, University of CO Mary McLean, University of WI-Milwaukee Beth Rous, University of Kentucky Pat Trohanis, University of NC, NECTAC

3 Objectives Provide background to OSEP’s early childhood outcomes work Hear diverse perspectives on critical issues related to early childhood assessment and accountability Facilitate a dialogue on these critical issues

4 Why the focus on early childhood outcomes? Improve results for young children with disabilities and their families Meeting a need in the field Development of outcomes for general early childhood programs Address GPRA, PART, and IDEA 2004

5 Approach Short-term: obtain data from the States regarding child and family outcome GPRA indicators used to assess program performance at the Federal Level Long-term: support the development of State early childhood outcome data collection used for program improvement by States, local programs and service providers On-going stakeholder involvement

6 OSEP’s Key Investments Early Childhood Outcomes (ECO) Center – Funded Fall 2003 – Provides leadership and technical guidance General Supervision Enhancement Grants (GSEGs) – 2004 = 18 awards focused on ec outcomes – 2006 = 9 award focused on ec outcomes National Early Childhood Technical Assistance Center (NECTAC) – Provision of TA and 2 national conferences

7 State Performance Plan (SPP) and Annual Performance Report(APR) Each state required to submit SPP in December 2005 and an APR beginning Feb Part C SPP has 14 indicators Part B SPP (includes preschool) has 20 indicators Early childhood outcomes are part of the SPP/APR

8 Critical Events Spring 2005 – Public input on what should be collected with regard to child and family outcomes Summer 2005 – OSEP released the reporting requirements December 2005 – States submitted their plans for outcome data collection in their State Performance Plan Spring 2006 – States begin collecting data February 2007 – Status and entry data due February 2008 – first progress data due

9 OSEP Reporting Requirements: Part C and Preschool Child Outcomes Percent of children who demonstrate improved: Positive social emotional skills (including positive social relationships) Acquisition and use of knowledge and skills (including early language/communication [and early literacy]) Use of appropriate behaviors to meet their needs

10 Reporting Categories a. % of children who reach or maintain functioning at a level comparable to same-age peers b. % of children who improve functioning but do not achieve functioning comparable to same age peers c. % of children who do not improve functioning 3 outcomes x 3 percentages = 9 numbers

11 Proposed Change to Reporting Categories CURRENT a. % of children who reach or maintain functioning at a level comparable to same-aged peers b. % of children who improve functioning (not in a) c. % of children who did not improve functioning PROPOSED CHANGE a. % of children who maintain functioning at a level comparable to same-aged peers b. % of children who improved functioning to reach a level comparable to same-aged peers c. % of children who improve functioning but did not reach a level comparable to same-aged peers d. % of children who did not improve functioning

12 Additional information about child indicators Child must be in program at least 6 months Outcomes are broad and functional All outcome areas apply to all children regardless of area receiving services These progress indicators compare entry to exit data for each child, requiring a minimum of two data points Not mandating assessment tools, States have flexibility to choose own tool or tools.

13 Analysis of State Performance Plans ECO summarized State Plans for collecting child outcome data Based on SPPs submitted in December 2005 Many States doing more than what they sent in and many state plans have evolved since December

14 SPP Analysis: Part C Outcomes Data Sources Data Source #% Formal assessment instruments 4580% Parent report2545% Observation1425% Clinical opinion1018% IFSP goals & objectives611% Record review47% Not reported611%

15 SPP Analysis: Preschool Outcomes Data Sources Data Source #% Formal assessment instruments 4580% Observation1221% Parent report1119% Teacher/provider report814% IEP goals & objectives12% Clinical opinion12% Not reported1017%

16 Commonly Reported Assessment Instruments: Part C Of 28 states who listed specific assessment instruments: – HELP- 15 states – BDI/BDI states – AEPS- 11 states – Creative Curriculum- 6 states – ELAP- 6 states Not yet determined- 23 states

17 Commonly Reported Assessment Instruments: Preschool Of 31 states who listed specific assessment instruments: – BDI/BDI-2- 9 states – Creative Curriculum- 8 states – Brigance- 7 states – High Scope COR- 6 states – AEPS- 5 states – State developed assessments- 7 states Not yet determined or not reported- 27 states

18 State approaches to assessments One assessment selected by state List of assessments developed by state; programs pick Programs can use whatever they have been using

19 ECO Child Outcomes Summary Form (COSF) States need to be able to aggregate data across tools –need a “common metric” to which data from different assessments can be converted COSF – Provides “scores” directly on each of the 3 outcomes – Allows different assessment data to be transformed to a common metric (1 to 7 scale) – Allows for increments of significant progress to be tracked over time – Allows for any degree of progress to be tracked over time

21 Use of the form ECO envisioned the form as the final step in a team process where the child’s functioning was discussed and consensus was reached Alternatives: One professional completing, team members completing the form individually “Behind the Scenes” Alternative: Converting online assessment data directly to the 7-point scale

22 Current Activities States currently focused on training Development of guidance materials on the COSF Reliability studies on the COSF Encouraging states to start thinking about how they will use these data

23 Summary All States are required to submit data on 3 outcomes for all children participating in Part C and Part B Preschool programs. States implementing a variety of approaches to produce these data Information from assessments are critical – Single assessment statewide – Small set of approved assessments – Any assessment in use

24 Summary States are making critical decisions related to assessment right now.

25 Questions for the Panel

26 Assessment Tools / Use of Data Is it professionally acceptable (and will the data be valid) to use instruments designed for screening and eligibility, progress monitoring, or other purposes for accountability?

27 Assessment Tools / Use of Data Can and should the same assessment data be used for accountability, program improvement and individual progress monitoring purposes? Can data providers are collecting for other purposes (i.e., progress monitoring) be credible for accountability?

28 Training What is your sense of the current level of expertise of the field with regard to early childhood assessment? What has to be done to increase the overall level of knowledge of providers?

29 Families What is the role of the family in early childhood assessment for accountability purposes? How do we include families in a meaningful way?

30 Assessment Practices How can we ensure that accountability has a positive impact on assessment practices in early childhood and promotes authentic assessment?