Gathering Input for the Summary Statements Kathy Hebbeler Lynne Kahn TA Call, August 2008 Early Childhood Outcomes Center

OSEP asked ECO to help Should OSEP reduce the number of targets associated with the progress data in the SPP/APR? And, if so, what would be the most meaningful subset or summary of progress data on which to set targets? Early Childhood Outcomes Center

Summary Statement: Single or combination of OSEP categories that will be used to describe child progress and for which targets will be set. Target: Numerical goal related to the summary statement, e.g, 65% in 2011; increase by 1% each year.

ECO’s charge Collect stakeholder input on what the summary statement(s) should be Same for 3 outcomes? Same for B and C? Same for SPP/APR and OSEP reporting to OMB? Early Childhood Outcomes Center

Stakeholder input Nov 2007: 2 teleconferences with 619 and Part C state staff December 2007: 2 sessions at the OSEP Early Childhood Meeting January 2008: ECO Advisory Board Meeting January 2008: Posted the input on the ECO web site for public comment Early Childhood Outcomes Center

What was presented See Setting Targets for Child Outcomes for more information http://www.fpg.unc.edu/~eco/pdfs/summary_of_target_setting.pdf Early Childhood Outcomes Center

What was presented Early childhood examples from the PART review Possible summary statements and what they captured or communicated and associated strengths and weaknesses Early Childhood Outcomes Center

3. Sum of two or more categories: Percent of children in D+ E, i. e 3. Sum of two or more categories: Percent of children in D+ E, i.e., the percentage of children who exited the program functioning like same age peers. Example: 40% of children left the Part C/619 program performing at age expectations Pros Cons For 619, compatible with focus on school readiness since it presents children who achieved same age functioning at the end of the program Compatible with other outcome summary statement that describe the % of children who achieve a certain level of performance. The implication of 100 minus the number is accurate (e.g., for the example above, 60% of the children did not meet age expectations when they left the program) Does not represent the other kinds of progress that children made. Focusing only on this summary statement suggests the purpose of the program is so children can catch up which is not a reasonable expectation for many children.

7. Relationship between two or more categories 7. Relationship between two or more categories. Of the children who entered the program below age expectations in an outcome area, the percentage who made greater than expected progress by exit (C+D)/(A+B+C+D). Example: 46% of children who entered services below age expectations related to acquiring knowledge and skills made significant gains in this outcome by exit. Pros Cons Captures two types of progress being made by children in program Puts the focus on children who made greater than expected progress (i.e., the value added by the program) Controls for variation across states in the number of children who did not have needs in an area (E) by taking them out of the denominator thus leading to more comparable numbers across states. The implication for 100 minus the reported number is accurate and can be implied (e.g., for the example above, 54% of those who were below age expectations did not make significant gains). Somewhat cumbersome to describe the calculation. Does not show the progress of children in category E.

Lots of insightful comments and good discussion Stakeholder input Lots of insightful comments and good discussion No clear consensus on statement(s) Agreement on some principles, e.g., needed statement for which the targets could move. If the number starts too high, program not likely to show progress. Early Childhood Outcomes Center

ECO Recommendations Should OSEP require states to submit data on the same summary statement(s)? Weak yes. There should be at least one statement for which all states are expected to report data and set targets. Early Childhood Outcomes Center

ECO Recommendations What should the summary statement(s) be? Early Childhood Outcomes Center

Of those children who entered the program below age expectations in the outcome, the percentage who substantially increased their rate of growth by the time they exited (C+D)/(A+B+C+D) Reflects value added by the program Removes ‘e’; comparable across states Can be expected to changed over time.

The percentage of children who are functioning within age expectations by the end of the program (D + E) Reflects emphasis on school readiness; prevention of disability Reflects value added by the program Ready explainable Could serve as an incentive to states to broaden eligibility and serve more children