Presented at Measuring Child and Family Outcomes Meeting Crystal City, VA July, 2010 Building a Child Outcomes Measurement System Kathy Hebbeler, ECO at SRI International Janna Gregory, Maine Office of Child Development Services Jim Lesko, Delaware Department of Education

Objectives Introduce the framework and self assessment Provide an opportunity to work with the self assessment Learn how 2 states have used the self assessment 2 Early Childhood Outcomes Center

System for Producing Good Child and Family Outcomes Good Federal policies and programs Good State policies and programs High quality services and supports for children 0-5 and their families Good outcomes for children and families Good Local policies and programs Information infrastructure Evidence Based Practice Adequate funding Strong Leadership Prof’l Development Preservice Inservice

A good information infrastructure is an essential tool to providing high quality services and supports. 4 Early Childhood Outcomes Center

"Success should be judged by results, and data is a powerful tool to determine results. We can't ignore facts. We can't ignore data." President Barack Obama July 24, Early Childhood Outcomes Center

6 Early Childhood Information Infrastructure: Data Needed for Program Improvement WHOSERVICES COSTPERSONNEL OUTCOMES

Early Childhood Outcomes Center 7 Where is your state (or program) now? WHOSERVICES OUTCOMES What do you have? How much is linked?

What you may have noticed… Moving to data-based decision-making requires planning, resources, and time (years!). There are lots of activities involved. 8 Early Childhood Outcomes Center

Have you been wondering? What else do we need to do with our outcomes data? How do we know if we are heading in the right direction? How good is our outcomes measurement system? How can I find resources related to using data for program improvement? 9 Early Childhood Outcomes Center

Framework for a Child Outcomes Measurement System 10 Early Childhood Outcomes Center Introducing…..

Child Outcomes Measurement System The set of components a state needs to have in place to make full use of child outcomes data. NOT just a data system or a data collection method. 11 Early Childhood Outcomes Center

Purpose of the Framework Provide a common language for ECO and other TA providers to use in discussing COMSs with states. Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS. Serve as the organizing structure for the self assessment 12 Early Childhood Outcomes Center

Framework and Self-Assessment F RAMEWORK –Set of components and quality indicators –Provides the structure for the self-assessment S ELF - ASSESSMENT –Scale that provides criteria for levels of implementation within each quality indicator –Rating assigned based on level of implementation within each indicator 13 Early Childhood Outcomes Center

Process for Framework Development Built off what we had learned from ECO work with states and previous ECO conceptual framework Literature review Repeated discussion and review internally and with 7 Partner States 14 Early Childhood Outcomes Center

Framework Partner States StatePart C619 CaliforniaX ColoradoXX DelawareXX MaineXX MinnesotaXX New YorkX OhioXX 15 Early Childhood Outcomes Center

Some People We Appreciate California Meredith Cathcart Anne Kuschner Patty Salcedo Greg Sampson-Gruener Mary McLean Steve Lohrer Colorado Nan Vendegna Nick Ortiz Christy Scott Delaware Rosanne Griff-Cabelli Susan Campbell Jim Lesko Maine Debra Hannigan Janna Gregory Lisa Cooper Minnesota Lisa Backer Mary Hunt Shivani Pandit Hope Brandt Early Childhood Outcomes Center16 New York Donna Noyes Li Yan Amy Barzykowski Randy Carter Ohio Kim Carlson Ellen Gow Stephanie Siddens

Early Childhood Outcomes Center 17

Early Childhood Outcomes Center 18 State has articulated purpose(s) of COMS.

Early Childhood Outcomes Center 19 The state has effective procedures for collecting, storing, and transmitting data to the state.

Early Childhood Outcomes Center 20 The state agency and local programs generate outcomes questions and analyze data for accountability and program improvement.

Early Childhood Outcomes Center 21 State has developed and implemented a plan for interpreting data, reporting, and communicating information about child outcomes.

Early Childhood Outcomes Center 22 The state agency and local programs use child outcomes data for accountability and program improvement.

Early Childhood Outcomes Center 23 Evaluation of the COMS is in place to ensure relevance and timeliness of procedures.

Early Childhood Outcomes Center 24 State coordinates child outcomes measurement and data use across EC systems.

Quality Indicator Provides additional detail as to what constitutes quality implementation of the component. 18 quality indicators across the 7 components 25 Early Childhood Outcomes Center

26 The state has effective procedures for collecting, storing, and transmitting data to the state.

3 Quality Indicators for Data Collection and Transmission Data collection procedures are carried out efficiently and effectively. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. State's method for entering, transmitting, and storing data is effective and efficient. 27 Early Childhood Outcomes Center

The Self Assessment 28 Early Childhood Outcomes Center

Purpose of the Self Assessment Assist states in setting priorities for improving their COMS Provide information to assist states in advocating for resources for systems development 29 Provide guidance to states on what constitutes a high quality child outcomes measurement system. Early Childhood Outcomes Center

The Scale for the Quality Indicators 1 = No or minimal implementation of the quality indicator 3 = Some implementation 5 = Substantial implementation 7 = Full implementation 30 Early Childhood Outcomes Center

31 Early Childhood Outcomes Center

32 Early Childhood Outcomes Center

Working with the Self Assessment 33 Early Childhood Outcomes Center

Where would a state start? 34 Early Childhood Outcomes Center Questions (Analysis) Purpose

Early Childhood Outcomes Center 35 Data on Outcomes: A critical piece but only one piece of the information infrastructure WHOSERVICES COSTPERSONNEL OUTCOMES

Next steps for ECO Populate the COMS framework with resources related to the components Revise the self assessment based on feedback Develop a framework for a Family Outcomes Measurement System 36Early Childhood Outcomes Center

Feedback Self assessment is a working draft Your questions and comments will help us make it better Don’t understand something? Disagree with the scaling? We want to hear from you! comments to 37Early Childhood Outcomes Center

38

For more information For updates to the framework and the self- assessment and resources to support the quality indicators: 39 Early Childhood Outcomes Center