1. Illinois October, 2008 Early Childhood Outcomes: Using Data for Program Improvement Kathy Hebbeler ECO at SRI International Robin Rooney and Christina Kasprzak ECO at Frank Porter Graham Institute

2. Early Childhood Outcomes Center2 Objectives Understand the purposes of the child outcomes data collection Be familiar with key considerations related to accurately completing the Child Outcomes Summary Form Be able to evaluate child outcomes data for accuracy as part of supervision Understand the use of child outcomes data for program improvement, including for instruction, resource allocation, and professional development

3. Early Childhood Outcomes Center3 Why are we doing this?

4. Early Childhood Outcomes Center4 Keeping our eye on the prize: High quality services for children and families that will lead to good outcomes.

5. Early Childhood Outcomes Center5 Goal of early childhood special education … To enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community. (from Early Childhood Outcomes Center, http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf)

6. Early Childhood Outcomes Center6 High Quality Data on Outcomes Data are a piece of a system that helps to achieve overarching goals for children and families Data yield Evidence that allows you to make an inference that should lead to specific actions to improve the system.

7. Profl Development Preservice Inservice System for Producing Good Child and Family Outcomes Good Federal policies and programs Good State policies and programs High quality services and supports for children 0-5 and their families Good outcomes for children and families Good Local policies and programs Adequate funding Strong Leadership

8. Early Childhood Outcomes Center8 The Vision: Using Data as a Tool for Program Improvement Illinois will have quality data available on an ongoing basis about multiple components of the system Outcomes for children and families Programs and services provided Personnel (types, qualifications, etc.) Etc.

9. Early Childhood Outcomes Center9 Driving Force for Data on Child Goals Comes from the Federal Level Government Performance and Results Act (GPRA) Government Performance and Results Act (GPRA) Program Assessment Rating Tool (PART) Program Assessment Rating Tool (PART) Individuals with Disabilities Education Act (IDEA) Individuals with Disabilities Education Act (IDEA)

10. Early Childhood Outcomes Center10 Requires goals and indicators be established for IDEA Indicators and data collection further along for school age population than for EC Previously, for early childhood data had been collected on: Number of children served Settings Government Performance and Results Act (GPRA) passed in 1993

11. Early Childhood Outcomes Center11 130 programs examined in 2002; 50% programs had no performance data Programs looking at inputs, not results Part C and Section 619 No long-term child outcome goals or data Need to develop a strategy to collect annual performance data in a timely manner PART evaluation results (2002)

14. Early Childhood Outcomes Center14 Federal Funding for Preschool Special Education Total U.S.Illinois 2004$387,699,000$17,943,956 2005$381,385,691$17,812,936 2006$380,751,030$17,650,453

15. Early Childhood Outcomes Center15

16. Early Childhood Outcomes Center16 SEC. 616. > MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.- ….. ….. ``(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- ``(A) improving educational results and functional outcomes for all children with disabilities; Individuals with Disabilities Education Act

17. Early Childhood Outcomes Center17 How Office of Special Education (OSEP) responded Required states to submit outcome data in their Annual Performance Report (APR) Funded the Early Childhood Outcomes (ECO) Center to do research, make recommendations, and assist states

18. Early Childhood Outcomes Center18 Where are we now: Federal reporting requirements

19. Early Childhood Outcomes Center19 OSEP Reporting Requirements: the Outcomes Positive social emotional skills (including positive social relationships) Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) Use of appropriate behaviors to meet their needs

20. Early Childhood Outcomes Center20 OSEP Reporting Categories Percentage of children who: a.Did not improve functioning b.Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c.Improved functioning to a level nearer to same- aged peers but did not reach it d.Improved functioning to reach a level comparable to same-aged peers e.Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 measures = 15 numbers

21. Early Childhood Outcomes Center21 Reporting Schedule Due February 2008 Data in reporting categories at exit for all children who have been in the program for at least 6 months Must be reported for the year beginning July 1, 2006 Repeat with next years data in 2009, etc. 2010 States must set targets Summary statements for targets recommended; not yet official* States must report data to public by school districts for these target numbers * See http://www.fpg.unc.edu/~ECO/meetings.cfm for more information on recommended targetshttp://www.fpg.unc.edu/~ECO/meetings.cfm

22. Early Childhood Outcomes Center22 Where are we now: State decisions and activities

23. Early Childhood Outcomes Center23 To respond to federal reporting requirements To have data for program improvement and to respond to federal reporting requirements Purpose WHY? (State Version)

24. Early Childhood Outcomes Center24 To provide data to the state To have data for program improvement and to provide data to the state Purpose WHY? (Local Version)

25. Early Childhood Outcomes Center25 State approaches Most states have embraced outcomes measurement and are collecting outcomes data for their own purposes. Many states are building bigger systems than needed to produce the federal data. Go to www.the-eco-center.org for more information about what other states are doing

26. Early Childhood Outcomes Center26 How are states collecting child outcomes data? Possible state approaches to collection of child data Child Outcomes Summary Form (COSF) Publishers online assessment system Single assessment statewide Other approaches

27. Approach Part C (56 states) Preschool (59 states) One tool statewide 8/56 (14%) 13/59 (22%) Multiple Publishers online tools 2/56 (4%) 3/59 (5%) COSF 7 pt. scale 40/56 (71%) 36/59 (61%) Other 6/56 (11%) 7/59 (12%) State Approaches to Measuring Child Outcomes

28. Early Childhood Outcomes Center28 Checking in – How is it going? Implementing the Child Outcomes Summary Form Process What is going well? What is not going well? Are there other issues or concerns that need to be addressed about the process?

29. Early Childhood Outcomes Center29 Lets Review: Child Outcome Summary Form Child Outcome Summary Form

30. Early Childhood Outcomes Center30 Learning from Each Other 1.Did you attend a training on the COSF? 2.Have you participated in a COSF team rating process? For more than 10 children? 20? 30? 3.Have you reviewed COSF forms completed by others? 4.Have you provided training on the COSF? 5.How confident are you in your knowledge of the COSF process?

31. Early Childhood Outcomes Center31 Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

32. Early Childhood Outcomes Center32 Important point It is not necessary that all team members be knowledgeable in all 5 areas Especially, no expectation that parents understand the rating scale or typical child development But the professionals have to!

33. Early Childhood Outcomes Center33 Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

34. Early Childhood Outcomes Center34 1. Know about the childs functioning across settings and situations How we learn about the childs functioning across settings and situations:Goodassessment

35. Early Childhood Outcomes Center35 DEC* recommended practices for assessment Involve multiple sources Examples: family members, professional team members, service providers, caregivers Involve multiple measures Examples: observations, criterion- or curriculum- based instruments, interviews, norm-referenced scales, informed clinical opinion, work samples *Division for Early Childhood

36. Early Childhood Outcomes Center36 Assessment practices appropriate for outcomes measurement: ASHA* ASHA recommended practices: Gather information from families, teachers, other service providers Collect child-centered, contextualized, descriptive, functional information (*American Speech-Language-Hearing Association)

37. Early Childhood Outcomes Center37 Assessment instruments Assessment the tool vs. assessment the process Assessment tools can inform us about childrens functioning in each of the three outcome areas Challenge: There is no assessment tool that assesses the three outcomes directly

38. Early Childhood Outcomes Center38 Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

39. Early Childhood Outcomes Center39 Resources for understanding age- expected child development ECO link http://www.fpg.unc.edu/~eco/pdfs/Age-expected_child_dev_9-5-07.pdf (under ECO Tools) New course coming soon Watch ECO web sitewww.the-eco-center.org

40. Early Childhood Outcomes Center40 Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

41. Early Childhood Outcomes Center41 Outcomes Jeopardy Pointing to the cabinet for cereal Reading the letter S on the Stop sign Washes hands before lunch Biting Plays by himself in the classroom Plays with rhyming words Building a castle from blocks with a friend Problems sleeping Sharing a cookie at lunchtime $100 $200 $100 $300 $200 $300 $200 $100 $300

42. Early Childhood Outcomes Center42 Children have positive social relationships Involves: Relating with adults Relating with other children For older children, following rules related to groups or interacting with others Includes areas like: Attachment/separation/autonomy Expressing emotions and feelings Learning rules and expectations Social interactions and play

43. Early Childhood Outcomes Center43 Children acquire and use knowledge and skills Involves Thinking Reasoning Remembering Problem solving Using symbols and language Understanding physical and social worlds Includes: Early conceptssymbols, pictures, numbers Imitation Object permanence Expressive language and communication Early literacy

44. Early Childhood Outcomes Center44 Children take appropriate action to meet their needs Involves: Taking care of basic needs Getting from place to place Using tools (e.g., fork, toothbrush, crayon) In older children, contributing to their own health and safety Includes: Integrating motor skills to complete tasks Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility) Acting on the world to get what one wants

45. Early Childhood Outcomes Center45 Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

46. Early Childhood Outcomes Center46 The two COSF questions a. To what extent does this child show age- appropriate functioning, across a variety of settings and situations, on this outcome? (Rating: 1-7) b. Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No)

47. Early Childhood Outcomes Center47 7 – Completely The child shows behaviors and skills expected in all or almost all everyday situations that are part of the childs life Home, store, park, child care, with strangers, etc. The childs functioning is considered appropriate for his/her age No one has significant concerns about the childs functioning in this outcome area

48. Early Childhood Outcomes Center48 6 – Between completely and somewhat The childs functioning generally is considered appropriate for his or her age but there are some significant concerns about the childs functioning in this outcome area

49. Early Childhood Outcomes Center49 5 – Somewhat The child shows functioning expected for his/her age some of the time and/or in some situations The childs functioning is a mix of age- appropriate and not appropriate functioning The childs functioning might be described as like that of a slightly younger child

50. Early Childhood Outcomes Center50 4 – Between a 5 and a 3 Child shows some age appropriate functioning some of the time or in some situations or settings but most of the childs functioning would be described as not yet age appropriate. The childs functioning might be described as like that of a younger child

51. Early Childhood Outcomes Center51 3 – Emerging The child does not yet show functioning expected of a child his/her age in any situation The childs behaviors and skills include immediate foundational skills on which to build age-appropriate functioning The childs functioning might be described as like that of a younger child

52. Early Childhood Outcomes Center52 2 – Between 3 and 1 The child does not yet show functioning expected of a child his/her age in any situation The childs behaviors and skills does have some the immediate foundational skills on which to build age-appropriate functioning but these are not displayed very often The childs functioning might be described as like that of a younger or even much younger child

53. Early Childhood Outcomes Center53 1 – Not yet The child does not yet show functioning expected of a child his/her age in any situation The childs skills and behaviors also do not yet include any immediate foundational skills on which to build age-appropriate functioning The childs functioning might be described as like that of a much younger child Children with 1 ratings still have skills, just not yet at an immediate foundational level

54. Early Childhood Outcomes Center54 Rating Scale Jeopardy Age appropriate functioning – no concerns Mix of age appropriate and not age appropriate functioning No age appropriate functioning – not yet showing immediate foundational skills Some age appropriate functioning but very little No age appropriate functioning – lots of immediate foundational skills Age appropriate functioning – some concerns Rarely shows age appropriate functioning No age appropriate functioning – some immediate foundational skills Age appropriate functioning $100 $200 $100 $300 $200 $300 $200 $100 $300

55. Early Childhood Outcomes Center55

56. Early Childhood Outcomes Center56 Essential Knowledge for Completing the Child Goals Outcome Form Between them, team members must: 1.Know about the childs functioning across settings and situations 2.Understand age-expected child development 3.Understand the content of the three child outcomes 4.Know how to use the rating scale 5.Understand age expectations for child functioning within the childs culture

57. Early Childhood Outcomes Center57 Point of clarification Process is NOT about comparing groups of children – it IS about asking how close children are to being able to do what is expected at their age Early learning guidelines Kindergarten and access to the general curriculum

58. Early Childhood Outcomes Center58 What we are learning nationally The process of training for child outcomes data collection has uncovered other areas of significant need related to professional development.

59. Early Childhood Outcomes Center59 Providers need to know more about: Assessment How to gather assessment data to reflect functioning across settings and situations, especially how to gather child functioning information from families Understanding the results of the assessment Sharing assessment results sensitively and honestly with families

60. Early Childhood Outcomes Center60 Providers need to know more about: Functional outcomes What are they? How do they differ from outcomes organized around domains? What do they mean for how professionals from different disciplines operate as a team? Typical child development What are the functional expectations for children at different ages with regard to each of the 3 goal statements?

61. Early Childhood Outcomes Center61 Lets meet Ava