Update on Family Outcomes Work Don Bailey and Melissa Raspa ECO Advisory Board Meeting March 7, 2012

Goals for Today Provide update on family outcomes APR data Introduce ECO family outcomes and experiences measurement system self- assessment tool Discuss future of family outcomes Early Childhood Outcomes Center

Part C Measurement Approaches Tools –NCSEAM survey (n = 25) –ECO Family Outcomes Survey (n = 24) FOS-R (n = 3) –State-developed survey (n = 6) Methodology –Census (n = 44) –Sampling (n = 11) 3Early Childhood Outcomes Center

Part C Population Surveyed TimingAll familiesFamilies > 6 months Total Point in time At exit369 At IFSP meeting325 IFSP or exit213 Total Early Childhood Outcomes Center

Part C Response Rates Overall response rate = 36% Tool –ECO: 41% –NCSEAM: 31% –State: 31% Methodology: –Census: 37% –Sample: 34%

Part C Response Rates Distribution –In person: 46% –Multiple methods: 37% –Mail: 27% Return –In-person: 75% –Multiple methods: 36% –Mail: 27% 6Early Childhood Outcomes Center

Part C Representativeness CriteriaStates (N)States (%) Race/ethnicity4175 Geography (district, county)1731 Gender1425 Child’s age (at referral, at survey)1324 Disability/eligibility category611 Length of time in services611 Income35 7Early Childhood Outcomes Center

Part C APR Trend Data Early Childhood Outcomes Center

Part C Trend Data – Indicator 4A 9Early Childhood Outcomes Center

Part C Trend Data – Indicator 4B 10Early Childhood Outcomes Center

Part C Trend Data – Indicator 4C 11Early Childhood Outcomes Center

Part C APR by Survey Tool 12Early Childhood Outcomes Center

Part B Measurement Approaches Tools –NCSEAM (n=43) –Adapted NCSEAM or ECO (n=8) –State (n=8) Methodology –Census (n=21) –Sampling (n=35) 13Early Childhood Outcomes Center

Part B Response Rates Overall response rate = 24% Distribution –Multiple methods: 30% –Mail: 17% Return –In-person: 75% –Multiple methods: 36% –Mail: 27 14Early Childhood Outcomes Center

Part B APR Trend Data 15Early Childhood Outcomes Center

Next Steps APR analyses for FFY Examine data quality –APR by response rates Develop template for states to share family outcomes data Provide TA to states on as-needed basis 16Early Childhood Outcomes Center

FAMILY OUTCOMES AND EXPERIENCES MEASUREMENT SYSTEM SELF ASSESSMENT TOOL Early Childhood Outcomes Center 17

Purpose Provides a common language and organizing structure for discussing family outcomes and experiences Assist states in developing a measurement system that captures various aspects of family outcomes and experiences, such as –benefits that families receive from program participation –family satisfaction with services –perceived helpfulness of the services –family involvement with the service delivery system 18Early Childhood Outcomes Center

Framework and Self-Assessment F RAMEWORK –Set of components and quality indicators –Provides the structure for the self-assessment S ELF - ASSESSMENT –Scale that provides criteria for levels of implementation within each quality indicator –Rating assigned based on level of implementation within each indicator 19 Early Childhood Outcomes Center

Partner States Connecticut Illinois Minnesota Texas 20Early Childhood Outcomes Center

Framework Components –7 key areas of a measurement system Quality indicators –16 specific items that further describe each of the components Elements –Various examples of what constitute quality at the indicator level 21Early Childhood Outcomes Center

22 Purpose Data Collection and Transmission Analysis Reporting Using Data Evaluation Cross- System Coordination Components Quality Indicators Purpose 1.State has articulated purpose(s) of COMS. Data Collection and Transmission 2.Data collection procedures are carried out efficiently and effectively. 3.Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. 4.State's method for entering, transmitting, and storing data is effective and efficient. Analysis 5.State identifies accountability and program improvement questions related to child outcomes. 6.Local programs identify accountability and program improvement questions related to child outcomes. 7.State agency analyzes data in a timely manner. 8.Local programs analyze data in a timely manner. 9.State agency ensures completeness and accuracy of data. Reporting 10.State agency interprets, reports, and communicates information related to child outcomes. 11.Local programs interpret, report, and communicate information related to child outcomes. Using Data 12.State agency makes regular use of information on child outcomes to improve programs. 13.Local programs makes regular use of information on child outcomes to improve programs. Evaluation 14.State evaluates its COMS regularly. Cross-system Coordination 15.Part C and 619 coordinate child outcomes measurement. 16.Child outcomes measurement is integrated across early childhood (EC) programs statewide. 17.Child outcomes measurement is aligned with state’s early learning guidelines/standards. 18.State has a longitudinal data system to link child outcomes data from EC program participation to K–12 data. Elements a. State has… b. State has… c. State agency.. d. Representative.. e. State agency… f. State …… g. State provides… h. State has..

Self-Assessment Provides guidance to states on what constitutes a high quality family experiences and outcomes measurement system. Assists states in setting priorities for improving their measurement system Provides information to assist states in advocating for resources for systems development 23

How Does it Work? Each quality indicator has multiple elements Evidence for the extent of implementation for each element is provided Each element is rated as NY = Not Yet or Don’t know IP = In Process IF = Fully Implemented NA = Not applicable (only where applicable) The quality indicator is given a rating based on the ratings of the elements Early Childhood Outcomes Center24

Self-assessment Scoring 25 Implementation of Elements Quality Indicator Score All elements are fully implemented7 Nearly all elements are fully implemented and the rest are in process6 Most of the elements are fully implemented and the rest are in process.5 At least one element is fully implemented and the rest are in process4 All of the elements are in process3 Some of the elements are in process2 None of the elements are yet in process1

Challenges with Family Framework and Self-assessment Going beyond the APR: Family outcomes and experiences Balancing expectations: Reality vs. ideal Scoring system –Difficulty in reaching 7 –Not applicable items Early Childhood Outcomes Center26

Next Steps Finalize framework and self-assessment tool and post on ECO web site National conference call to introduce to broader audience Provide TA to states on as-needed basis Early Childhood Outcomes Center27

WHAT IS THE FUTURE OF FAMILY OUTCOMES?

Context Family-centered (FC) practices are endorsed by many professional organizations as central to high-quality practice –Institute of Medicine –American Academy of Pediatrics –American Hospital Association –American Speech-Language Hearing Association –Division for Early Childhood

We know…. Meta-analyses of research provides strong evidence that FC practices lead to many benefits for families and children

Dunst, Trivette, & Hamby (2007) Meta-analysis of 47 studies of family-centered practices in a variety of settings Higher use of FC practices was associated with: –Satisfaction with services* –Self-efficacy beliefs* –Parenting behaviors –Personal and/or family well-being –Social support* –Child behavior and functioning *Strongest associations

Pitrowski, Talavera, & Mayer (2009) Meta-analysis of 13 articles evaluating Health Steps, a pediatric care model based on – strength-based primary prevention – the assumption that child well-being and family-centered care are necessarily linked – supporting the role of parents –positive, caring, and enduring relations with parents. Parents in HS were –More satisfied with services –Exhibited more desirable parenting practices –More satisfied in their role as parents –Interacted more sensitively and appropriately with child

Broom & Enriquez (2009) Reviewed 9 randomized trials of family- centered interventions in treating children with Type 1 diabetes Family-centered practices were associated with –Improved health outcomes for children –Reduced diabetes-related conflict in families –Improved family relations

Two studies using the National Survey of Children with Special Health Care Needs When families perceived a strong partnership between the child’s family and the child’s service provider –children’s health outcomes were improved – children received more services –families were more satisfied with services

Also Our NEILS analyses and our ECO papers show that family-centered practices are related to family outcomes

Epley, Summers & Turnbull (2011)

But research also shows that FC practices are not widely implemented Medical Expenditure Panel Survey and National Survey of CSHCN children found that only about 2/3 of families reported getting family-centered care A hospital survey found that only about half used FC rounds in pediatric in-patient settings Minority families generally report fewer FC services and less satisfaction

Dunst (2012) argues that EI has not evolved in tandem with what we know

So, if….. Professional organizations widely endorse FC practices, and… Research shows that FC practices make a difference… What should we do about the future of FC practices and family outcomes in early intervention and preschool programs?

Use of Family-Centered Practices We don’t know at a state or national level the extent to which programs are actually using family- centered principles and practices We don’t know what factors predict or are associated with variability in implementation of F-C practices NEEDED: A comprehensive description of the status of F-C practices in EI and preschool programs, and a clear understanding of why some programs are more family-centered than others

Family-centered principles, when integrated across all program activities, ought to lead to family outcomes. Referral and intake Determining eligibility Child assessment Family assessment Team meetings and decision-making Service provision and coordination Parenting practices Community support Understand child’s abilities and special needs Know rights and advocate effectively Help child develop and learn Have support systems Access the community Family-Centered Principles Program ActivitiesFamily Outcomes Shared philosophy Families as partners Focus on strengths Family choice of goals and services Collaboration and coordination of service Effective communication Flexibility Community-based

What is it about F-C practices that leads to better outcomes? With the exception of the NEILS study and other smaller studies, we don’t know the full range of outcomes that F-C practices can affect The pediatric literature has far more studies of this than EI NEEDED: Sophisticated large-scale studies modeling how and which F-C practices influence a variety of outcomes

The desirability & applicability of F-C practices & family outcomes in preschool Nothing is known about F-C practices in preschool programs for children w/disabilities The ECO family outcomes have never been accepted or seriously discussed for preschool We have some evidence that family outcomes are less likely to be achieved with older children NEEDED: Stakeholder opinions and consensus regarding the desirability of FC practices & appropriate family outcomes in preschool

F-C practices and child outcomes Is helping families a good outcome in and of itself, OR must we show that this results in benefits for children? We have danced around this topic for 20 years, with no resolution NEEDED: Either we need consensus on the argument that family benefit is a worthwhile outcome OR we need to show that F-C practices and family outcomes are important to maximize child benefit

Potential Year 5 Family Activities Help states use framework to improve systems for family outcomes assessment Targeted TA to states with low family helpfulness ratings Collect, compile and analyze data from states that have FOS data that can be linked with child outcomes data Explore with one or two states or programs use of FOS in preschool programs