1. Presented at Measuring Child and Family Outcomes Meeting Crystal City, VA July, 2010 Building a Child Outcomes Measurement System Kathy Hebbeler, ECO at SRI International Janna Gregory, Maine Office of Child Development Services Jim Lesko, Delaware Department of Education

2. Objectives Introduce the framework and self assessment Provide an opportunity to work with the self assessment Learn how 2 states have used the self assessment 2 Early Childhood Outcomes Center

3. System for Producing Good Child and Family Outcomes Good Federal policies and programs Good State policies and programs High quality services and supports for children 0-5 and their families Good outcomes for children and families Good Local policies and programs Information infrastructure Evidence Based Practice Adequate funding Strong Leadership Prof’l Development Preservice Inservice

4. A good information infrastructure is an essential tool to providing high quality services and supports. 4 Early Childhood Outcomes Center

5. "Success should be judged by results, and data is a powerful tool to determine results. We can't ignore facts. We can't ignore data." President Barack Obama July 24, 2009 5Early Childhood Outcomes Center

6. 6 Early Childhood Information Infrastructure: Data Needed for Program Improvement WHOSERVICES COSTPERSONNEL OUTCOMES

7. Early Childhood Outcomes Center 7 Where is your state (or program) now? WHOSERVICES OUTCOMES What do you have? How much is linked?

8. What you may have noticed… Moving to data-based decision-making requires planning, resources, and time (years!). There are lots of activities involved. 8 Early Childhood Outcomes Center

9. Have you been wondering? What else do we need to do with our outcomes data? How do we know if we are heading in the right direction? How good is our outcomes measurement system? How can I find resources related to using data for program improvement? 9 Early Childhood Outcomes Center

10. Framework for a Child Outcomes Measurement System 10 Early Childhood Outcomes Center Introducing…..

11. Child Outcomes Measurement System The set of components a state needs to have in place to make full use of child outcomes data. NOT just a data system or a data collection method. 11 Early Childhood Outcomes Center

12. Purpose of the Framework Provide a common language for ECO and other TA providers to use in discussing COMSs with states. Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS. Serve as the organizing structure for the self assessment 12 Early Childhood Outcomes Center

13. Framework and Self-Assessment F RAMEWORK –Set of components and quality indicators –Provides the structure for the self-assessment S ELF - ASSESSMENT –Scale that provides criteria for levels of implementation within each quality indicator –Rating assigned based on level of implementation within each indicator 13 Early Childhood Outcomes Center

14. Process for Framework Development Built off what we had learned from ECO work with states and previous ECO conceptual framework Literature review Repeated discussion and review internally and with 7 Partner States 14 Early Childhood Outcomes Center

15. Framework Partner States StatePart C619 CaliforniaX ColoradoXX DelawareXX MaineXX MinnesotaXX New YorkX OhioXX 15 Early Childhood Outcomes Center

16. Some People We Appreciate California Meredith Cathcart Anne Kuschner Patty Salcedo Greg Sampson-Gruener Mary McLean Steve Lohrer Colorado Nan Vendegna Nick Ortiz Christy Scott Delaware Rosanne Griff-Cabelli Susan Campbell Jim Lesko Maine Debra Hannigan Janna Gregory Lisa Cooper Minnesota Lisa Backer Mary Hunt Shivani Pandit Hope Brandt Early Childhood Outcomes Center16 New York Donna Noyes Li Yan Amy Barzykowski Randy Carter Ohio Kim Carlson Ellen Gow Stephanie Siddens

17. Early Childhood Outcomes Center 17

18. Early Childhood Outcomes Center 18 State has articulated purpose(s) of COMS.

19. Early Childhood Outcomes Center 19 The state has effective procedures for collecting, storing, and transmitting data to the state.

20. Early Childhood Outcomes Center 20 The state agency and local programs generate outcomes questions and analyze data for accountability and program improvement.

21. Early Childhood Outcomes Center 21 State has developed and implemented a plan for interpreting data, reporting, and communicating information about child outcomes.

22. Early Childhood Outcomes Center 22 The state agency and local programs use child outcomes data for accountability and program improvement.

23. Early Childhood Outcomes Center 23 Evaluation of the COMS is in place to ensure relevance and timeliness of procedures.

24. Early Childhood Outcomes Center 24 State coordinates child outcomes measurement and data use across EC systems.

25. Quality Indicator Provides additional detail as to what constitutes quality implementation of the component. 18 quality indicators across the 7 components 25 Early Childhood Outcomes Center

26. 26 The state has effective procedures for collecting, storing, and transmitting data to the state.

27. 3 Quality Indicators for Data Collection and Transmission Data collection procedures are carried out efficiently and effectively. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. State's method for entering, transmitting, and storing data is effective and efficient. 27 Early Childhood Outcomes Center

28. The Self Assessment 28 Early Childhood Outcomes Center

29. Purpose of the Self Assessment Assist states in setting priorities for improving their COMS Provide information to assist states in advocating for resources for systems development 29 Provide guidance to states on what constitutes a high quality child outcomes measurement system. Early Childhood Outcomes Center

30. The Scale for the Quality Indicators 1 = No or minimal implementation of the quality indicator 3 = Some implementation 5 = Substantial implementation 7 = Full implementation 30 Early Childhood Outcomes Center

31. 31 Early Childhood Outcomes Center

32. 32 Early Childhood Outcomes Center

33. Working with the Self Assessment 33 Early Childhood Outcomes Center

34. Where would a state start? 34 Early Childhood Outcomes Center Questions (Analysis) Purpose

35. Early Childhood Outcomes Center 35 Data on Outcomes: A critical piece but only one piece of the information infrastructure WHOSERVICES COSTPERSONNEL OUTCOMES

36. Next steps for ECO Populate the COMS framework with resources related to the components Revise the self assessment based on feedback Develop a framework for a Family Outcomes Measurement System 36Early Childhood Outcomes Center

37. Feedback Self assessment is a working draft Your questions and comments will help us make it better Don’t understand something? Disagree with the scaling? We want to hear from you! E-mail comments to staff@the-eco-center.org 37Early Childhood Outcomes Center

38. 38

39. For more information For updates to the framework and the self- assessment and resources to support the quality indicators: www.the-eco-center.org 39 Early Childhood Outcomes Center