1. Understanding and Using Early Childhood Outcome (ECO) Data for Program Improvement TASN – KITS Fall 2012 Webinar August 31 st, 2012 Tiffany Smith Phoebe Rinkel Chelie Nelson 1

2. Tiffany Smith KSDE, ECSE Program Consultant tsmith@ksde.org Phoebe Rinkel KITS, Part B, TA provider prinkel@ku.edu Chelie Nelson KITS, Part B, TA provider chelie.nelson@ku.edu 2

3. Online Resources www.kskits.org 3

4. Agenda Overview of the Early Childhood Outcome (ECO) Data Kansas Data Drill Down Guide Case Study Examining Policies and Procedures Examining APR Reports Examining ECO Addendum Reports Examining Data Verification Examining Child Level Data in OWS 4

5. Early Childhood Outcomes OSEP required states to submit outcome data in their State Performance Plan (SPP) and Annual Performance Report (APR) 2010 – 2011 (Federal Fiscal Year 2009) first year Districts and Part C Networks were compared to State targets 5

6. The Three Early Childhood Outcomes 1. Positive social-emotional skills (including social relationships) 2. Acquisition and use of knowledge and skills (including early language/communication [and early literacy*]) 3. Use of appropriate behaviors to meet their needs *for 3-5 6

7. How Kansas Early Childhood Outcome Data is Reported 7

8. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same-age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 8

9. Entry Kasprzak & Rooney (2010) 9

10. EntryExit Kasprzak & Rooney (2010) 10

11. EntryExit Kasprzak & Rooney (2010) 11

12. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same-age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 12

13. EntryExit a Kasprzak & Rooney (2010) 13

14. EntryExit a Kasprzak & Rooney (2010) 14

15. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same-age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 15

16. EntryExit b Kasprzak & Rooney (2010) 16

17. EntryExit b Kasprzak & Rooney (2010) 17

18. EntryExit b Kasprzak & Rooney (2010) 18

19. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same- age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 19

20. EntryExit c Kasprzak & Rooney (2010) 20

21. EntryExit c Kasprzak & Rooney (2010) 21

22. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same-age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 22

23. EntryExit d Kasprzak & Rooney (2010) 23

24. States Report Data in these categories Percentage of children who: a) Did not improve functioning b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers c) Improved functioning to a level nearer to same-age peers, but did not reach it d) Improved functioning to reach a level comparable to same-age peers e) Maintained functioning at a level comparable to same-age peers. 24

25. EntryExit e Kasprzak & Rooney (2010) 25

26. EntryExit e Kasprzak & Rooney (2010) 26

27. EntryExit e Kasprzak & Rooney (2010) 27

28. Summary Statements For Reporting Progress on Targets Required Summary Statement 1: Of those children who entered or exited the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they exited the program. c+d __ a+b+c+d Required Summary Statement 2: The percent of children who were functioning within age expectations in each Outcome by the time they exited the program. d+e __ a+b+c+d+e 28

29. State ECO Targets FY 2010 (Reported on March 2012) Outcome 1Outcome 2Outcome 3 Summary Statement 1 % of children who moved closer to same age peers Part C = 57.53% Part B = 86.43% Part C = 61.14% Part B = 86.88% Part C = 66.99% Part B = 86.74% Summary Statement 2 % of children who exited at age level Part C = 56.33% Part B = 65.66% Part C = 47.44% Part B = 64.10% Part C = 63.44% Part B = 77.29% 29 State targets change each year, always be sure to use the most current data for your data drill down

30. Purpose Developed as a tool for local Part B Preschool Special Education Programs To identify components of a high quality system To evaluate their existing Indicator 7 Data To encourage decision making that will support program improvement efforts 30

31. 5 Sections A. Local Policies and Procedures for Data Reporting B. District APR Data C. Addendum Report Data D. Data Verification E. Child Level Data from OWS 31

32. Each Section includes; Information about the data to be examined and where it can be found Questions to Guide your Review Process Action Planning Form 32

33. Action Plan 33

34. Suggested Use Local Implementation Team Part of an ongoing strategic planning process May be completed in total or in sections Reassess periodically 34

35. Many Steps for Ensuring Quality Data Good Data Collection/Training Good data system and data entry Ongoing supervision of implementation Feedback to implementers Refresher training Review of COSF Records Data Analyses for validity checks 35 Kasprzak & Rooney (2010) Section A: Examine Local Policies and Procedures for Data Reporting

36. Administrator Quality Rating Checklist Data Entry Quality Rating Checklist Direct Service Provider Quality Rating Checklist Questions to Guide the Review Process (pg. 4 Data Drill Down Guide) 36

37. ECO City Example – HO#6 37 Section A: Examine Local Policies and Procedures for Data Reporting

38. Section B: Locating and Examining District APR Data

39. Section B: ECO City APR Data – HO#7

40. Comparing ECO City Data with State Data 40

41. Section C: Locating and Examining Addendum Reports 41

42. Section C: Examining Addendum Reports for ECO City – HO#8 42

43. Section C: Examining ECO City Addendum Reports – HO#9 43 Progress and Slippage Reports

44. Section C: Locating and Examining Addendum Reports 44

45. Section D: Data Verification Data Verification occurs each August 1 st – 31 st 45

46. Section D: ECO City Data Verification HO#10 46

47. Section E: Examining Child Level Data in OWS 47

48. Section E: Examining Child Level Data in OWS 48

49. Section E: Parameterized Data Report 49

50. Section E: No Permanent Exit 50

51. Section E: Permanent Exit Report 51

52. Section E: Summary Statement Report 52

53. Section E: ECO Report 53

54. Sharing Your ECO Data Sample messages that can be gleaned from your ECO data: Data show that children are making progress from entry to exit in the program Many children are catching up or getting closer to same age peers Point out how programs are contributing to school readiness Link message to broader EC issues (i.e. cost effectiveness of high quality EC programs) 54

55. What the data look like: Nationally 55

56. Part C and Preschool Average Percentage of Children in Each Category Outcome 1: Social/Emotional Kasprzak & Rooney (2010) 56

57. Part C and Preschool Average Percentage of Children in Each Category Outcome 2: Knowledge/Skills Kasprzak & Rooney (2010) 57

58. 58 Part C and Preschool Average Percentage of Children in Each Category Outcome 3: Getting Needs Met Kasprzak & Rooney (2010)

59. 59

60. Kasprzak & Rooney (2010) 60

61. Keeping our eye on the prize: High quality services for children and families that will lead to good outcomes. Kasprzak & Rooney (2010) 61

62. Questions? 62

63. References Hebbeler, K., Kahn, L., Taylor, C. & Bailey, A. (2011). Data Workshop: Analyzing and Interpreting Data. Presented at the Measuring and Improving Child and Family Outcomes Conference, New Orleans, LA. Kasprzak & Rooney (2010, March). Measuring Child Outcomes, Presentation for Delaware; ECO Center & NECTAC. Retrieved 10/3/11 from: http://www.fpg.unc.edu/~eco/assets/ppt/DE%20COSF %20training%20slides%20for%20web%204-12-10.ppt http://www.fpg.unc.edu/~eco/assets/ppt/DE%20COSF %20training%20slides%20for%20web%204-12-10.ppt 63