1. Delaware Birth to Three Early Intervention System Evaluation: Child Outcomes July 15, 2004 Conference Call Series: Measuring Child Outcomes “Examples of using changes in growth rates to measure child outcomes in Part C” Rosanne Griff-Cabelli, Delaware Part C rosanne.griff-cabelli@state.de.us

2. Delaware Birth to Three Child Outcomes Evaluation Primary purposes: 1. To document changes in the children served by Part C Programs 2. To collect data, systematically so that programs can communicate their impact on children and families to various audiences.

3. The State Budget Office, as part of the Interagency Management Resource Committee (IRMC), requested child change data from federal and state funded programs. Part C’s evaluation was linked through the IRMC to early childhood programs, such as Head Start and Part B 619. The IRMC Annual Report includes Part C evaluation results, distributed to Department Secretaries and Legislative Joint Finance Committee. Part C formed an Ongoing Program Evaluation Committee (OPEC) to coordinate, review and update components of child outcome evaluation. Results are shared annually at ICC and Part C regional staff meetings. Impetus for Delaware’s Child Outcome Evaluation

4. Outcome Areas: Focus and Scope Outcome Measures: Describe changes in eligible children’s development and functioning over time, while active in Part C programs by:  Developmental Domains Motor Cognitive  Play and Functional Changes

5. Outcome Areas: Focus and Scope Demographic Variables: Describe socio-economic factors of family Describe demographic information such as gender, race/ethnicity, county of residence, age child started program, primary language spoken

6. Design/Method Population to Sample: Children selected are Part C eligible, have their first MDA completed, and not older than two years and six months Random sample of every 15 th child in New Castle County and every 10 th in Kent and Sussex Counties Children will have at least two points of evaluation, approximately one year apart No service coordinator will have more than three children actively involved in Child Change at any given point

7. Design/Method Measurement Instruments: Bayley Scales of Infant Development II, administered as part of MDA Fewell Play Assessment Scale (PAS) with adaptations for children with more severe disabilities Demographic Information Form, administered by service coordinators HOME [Home Observation for Measurement of the Environment] Inventory – Birth to Three, administered by service coordinators

8. Design/Method All assessments are administered by Part C program staff. The PAS is a valuable addition to the battery of assessments conducted with children. The PAS can be implemented by Part C program personnel but additional resources are needed for annual training. A contract is in place with the University of Delaware, Center for Disabilities Studies for data analysis and reports.

9. Analysis Using Proportional Change Index (PCI) Delaware’s Part C Child Change Evaluation is analyzed through a contract with the University of Delaware, Center for Disabilities Studies. The statistical procedure used is PCI. The PCI is a method devised by Mark Wolery of the University of Kentucky that “compares children’s rate of development at pretesting to their rate of development during intervention.” 1 The following link is to a paper on measuring developmental change. It refers to the PCI and its strengths and drawbacks. http://www.aare.edu.au/94pap/konzd94260.txt 1 Mark Wolery, Exceptional Children, Volume 50, Number 2, page 168, 1983

10. Developmental Gain / Pretest Developmental Age Time in Intervention / Pretest Chronological Age Divide second rate of development by first rate of development to calculate PCI. –Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0 –Children whose rates of development are slower during intervention will receive a PCI of less than 1.0 –Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0 Analysis Using Proportional Change Index (PCI) Design/Method

11. Developmental Gain / Pretest Developmental Age Time in Intervention / Pretest Chronological Age Divide second rate of development by first rate of development to calculate PCI. –Children who continue to develop at the same rate during intervention as they did prior to intervention will have a PCI of 1.0 –Children whose rates of development are slower during intervention will receive a PCI of less than 1.0 –Children whose rates of development accelerated during intervention will receive a PCI greater than 1.0 Analysis Using Proportional Change Index (PCI) Design/Method

12. Strengths of the PCI It is not a measure solely of the actual number of months gained in intervention, but takes into account the number of months in intervention and the child’s rate of development at pretesting. Comparisons can be made between children at various levels of delay, chronological age, and developmental age. Drawbacks of the PCI Assumes children’s rates of development at pretesting would be stable in the absence of intervention. Larger sample and three measures for each child would allow use of Growth Curve Analysis. Analysis Using Proportional Change Index (PCI)

13. Major Findings (sample in child change as of January 2004) The Children and Families:  43 children tracked  22 Northern CDW  21 Southern CDW  23 male; 20 female  Disabilities:  8 VLBW  11 developmental delay  24 VLBW and developmental delay  Ethnicity  15 African American, 18 Caucasian, 6 Hispanic, 4 bi-racial

14. Major Findings (as of January 2004) The Children and Families:  Family SES  15 families below poverty level  20 families above poverty level  8 unknown  Education Level of Mothers  2 less than 8th grade  4 less than high school  7 high school graduate/GED  15 some college or technical school  9 college degree  2 graduate degree  4 unknown

15. Major Findings (as of January 2004) Play Skills Development: 32 children with two PAS scores; 11 with three scores  Time between PASs (M=12.9 months)  Play skill changes (M=11.2 months; this is a.87 rate of development)  Expected rate of development (M=.72 or 8.64 months in 12 months time)  1.21 times the rate of development expected for this group of children  There was no significant difference in rates of development for girls or boys or children living in poverty and those not living in poverty

16. Major Findings (as of January 2004) Mental Skills Development: 32 children with 2 Bayley Mental scores; 11 with three scores  Time between Bayleys (M=12.3 months)  Mental score changes (M=10.6 months; this is a.86 rate of development)  Expected rate of development (M=.67 or 8.0 months in 12 months time)  1.28 times the rate of development expected for this group of children  There remains no significant difference in rates of cognitive development for girls and boys or children living in poverty and those not living in poverty

17. Major Findings (as of January 2004) Motor Skills Development: 32 children with 2 Bayley Motor scores; 11 with three scores  Time between Bayleys (M=12.3)  Motor score changes (M=10.3 months; this is a.84 rate of development)  Expected rate of development (M=.69 or 8.3 months in 12 months time)  1.22 times the rate of development expected for this group of children  No differences in rates of development for children living in poverty and those not; no differences between boys and girls

18. Major Findings (as of January 2004) Additional Findings:  Of the 43 children, 36 had rates of development greater than 1.0 for at least a one year period of time in both motor and mental skills  Seven children had rates of development equal or greater to 1.50 for at least a one year period of time in at least one domain of development  Seven children had rates of development below 1.0 for at least a one year period of time  After at least one year of services, only two children had rates of development lower than that which they entered the program (usually attributed to a progressive or degenerative disorder)

19. Lessons Learned Early intervention seems to be having a positive effect on children’s play, and mental and motor skill development across demographic groups. Child Change assessments are conducted systematically and program staff are able to implement the Bayley assessments as part of MDA annual timelines. Child Change results are reported in context of Birth to Three Evaluation Plan. Recommendations –Report the trend of children’s outcomes in three year intervals so that a long-term trend can be determined; the cohort should always be between 40-60 children. –Efforts are in place to increase the number of males in the evaluation to 55-60% (currently at 54%). –Continued efforts need to be in place to encourage families to participate in the evaluation since a sampling method is used.