Quality Rating Systems: A Study of Differing Models and Methodologies The Annual Meeting of the Child Care Policy Research Consortium March 8 – 11, 2005 Baltimore, MD Midwest Child Care Research Consortium

Midwest Child Care Research Consortium o In 2001, researchers and state program partners in four states initiated the Midwest Child Care Research Consortium (MCCRC). o The four states are Iowa, Kansas, Missouri, and Nebraska (HHS Region VII). o In 2004, researchers from Mississippi State University and the National Center on Rural Early Learning Initiatives joined MCCRC.

Measurement Assessment and Data The Consortium has a rich history of conducting multi-state research studies. The Consortium’s research focuses on issues associated with: o early childhood program quality (center- based programs and family child care homes; o provider workforce issues; o parent perceptions of child care use and quality; and, o state and federal polices and programs such as child care subsidy funds.

Measurement Assessment and Data Data are analyzed across the consortium as well as state specific to identify: o significant findings; o how policy decisions impact early childhood programs and the families who use the services; and, o how state and federal policies can be changed to improve early childhood programs and the well-being of families with young children.

Measurement Assessment and Data MCCRC was awarded a grant in 2000 from the Child Care Bureau and was also funded by the Kauffman Foundation to conduct a 3 year study. o Year 1: Phone survey of providers (n=2,026) randomly selected from 4 states and 365 of those providers participated in program observations. o Year 2: Paper survey of parents (n=1325) in same sites; Phone survey of subsidy-receiving parents (n=651). o Year 3: Licensing specialists worked with researchers to gather asset data about providers (n=1499) and directors (n=186) and program observations were conducted (n=115). o Year 4: Surveys and observations were completed with family child care providers (n=390).

Measurement Assessment and Data MCCRC was funded in by the Child Care Bureau and the Kauffman Foundation to conduct a study of Family Child Care. In 2004, MCCRC received a grant from the Child Care Bureau to implement and study Quality Rating Systems across the MCCRC states. In 2004, MCCRC was awarded a grant from the National Center for Rural Early Childhood Initiatives to increase the sample size of rural early childhood programs studied and to facilitate the development of a Quality Rating System in Mississippi.

Current Work of MCCRC The current project builds on the previous work of the Consortium and creates a system for differentiating quality in five states. The current project will: o Create five models of Quality Rating Systems. o Document quality-enhancement experiences of providers across states and across levels of quality. o Investigate change over a one-year period in the Quality Rating System scores associated with provider training. o Investigate perceptions of career development among providers and perceptions of child care among subsidy-receiving parents.

Objectives of the Study 1. To operationalize and validate 5-level Quality Rating Systems being developed in five states. 2. To study the naturally-occurring training of providers and to assess changes in quality ratings associated with varying levels and types of training using a pre- post study design.

Objectives of the Study, cont. 3. To qualitatively study providers and parents to learn more about… o provider career intentions at various levels of quality. o how parents perceive quality ratings in terms of their willingness to pay for quality care. o the experiences of subsidy-receiving parents. 4. To build on the past work of the Consortium and to link new data to extant data.

Context of the Study The context of the MCCRC study is one in which there are many states and national organizations exploring various types of rating systems. MCCRC study is unique in that: o It is a five-state consortium involving university researchers working in partnership with staff from state agencies. o The sample population includes rural and urban early childhood programs and programs that are subsidy- receiving as well as non-subsidy-receiving. o It includes focus groups of parents and providers.

Context of the Study, cont. At this time, 35 states are in the process of developing Quality Rating Systems. o 20 states have 2-3 tier systems o 8 states have 4 tier systems o 7 states have 5-6 tier systems

Context of the Study, cont. Categories that are common among Quality Rating Systems include*:  Learning Environments  Staff Compensation  Parent / Family Involvement  Program Evaluation  Administrative Policies and Procedures  Professional Development (Education, Training) *Summarized from the National Child Care Information Center’s “Common Categories of Criteria Used in State Tiered Quality Strategies.”

Context of the Study, cont. NAEYC Statement * Quality rating systems (which include tiered reimbursement, rating licensing and voluntary and mandatory rating systems of programs based on indicators of program quality) should be used for: o Greater consumer awareness of quality programs; o Increasing resources to help programs improve and sustain higher quality; o Lead to system-wide improvements in the quality of all programs, including all settings and auspices and ages of children served. *Exert from two-page NAEYC document (November, 2004)

Context of the Study, cont. What is the potential benefit of a Quality Rating System for various constituents * ? o Parents o Providers o Programs o Policymakers *Adapted from the National Child Care Information Center’s, “Goals and/or Objectives of State Quality Rating Systems.”

Context of the Study, cont. The anticipated benefits of a statewide Quality Rating System:  Supports parents and families by providing access to information  Creates greater consumer demand for higher quality programs  Supports program improvement by providing benchmarks for improvement, identifying areas of need, and incentives for change  Supports advocates and policymakers by information about the status of early childhood program quality in the state

Context of the Study, cont. Challenges for a Quality Rating System: o Raising the level of program quality in the community to a new height while assuring that new and existing programs have easily identified stepping stones and supports as they move along the quality continuum. o Balancing expenditures among all aspects of the Quality Rating System, including technical assistance.

Methodology o Although each state will develop their own Quality Rating System, each state will use the same pre-post test research design and the same research instruments. o The sample population will be licensed * early childhood program (center-based and family child care homes). o The sample population will be randomly selected from lists of licensed programs. *Nebraska will also include some unlicensed programs.

Sample Population o The five-state total: Center-based classroom observations, approximately N=650 Family childcare home observations, approximately N=350 o The sample population will include subsidy-receiving and non-subsidy-receiving programs.

Quantitative Study There are five aspects to the quantitative analysis. The Consortium will: 1. Facilitate each state’s development a Quality Rating Systems. 2. Pilot the Quality Rating Systems 3. Validate the Quality Rating Systems 4. Study the naturally occurring provider choices in training, professional development activities, and program supports. 5. Study the provider’s professional development training and the program’s quality enhancement activities to determine how the efforts influence the program’s QRS rating in year 2.

Instruments used in Quantitative Analysis 1. Environmental Rating Scale o Early Childhood Environmental Rating Scale-Revised o Family Day Care Rating Scale o Infant-Toddler Environmental Rating Scale-Revised 2. Early Childhood Rating Scale Extension (ECERS-E) Four Curricular Subscales 3. Caregiver Interaction Scale (Arnett) 4. Demographic Survey 5. Director Survey of Program Attributes 6. Survey of Training, Technical Assistance, and Program Supports

Qualitative Study The objective of the provider focus groups is to learn about their perspectives on professional development and how the Quality Rating System (QRS) might impact their choices and their program. Focus groups will occur in each state with providers at two time points:  Before they receive their Quality Rating System scores.  One year later

Qualitative Study The objective of the parent focus groups is to learn about their perceptions of quality care, their perspective on paying for quality care and to see how these views vary across two groups of parents. The two groups of parents are: o Parents who receive child subsidy funds o Parents who do not receive child subsidy funds.

Five State Implementation Missouri Efforts to create a Quality Rating System began several years ago and a stakeholder group has been meeting to guide the process of development and implementation. o The Missouri model was piloted in conjunction with the Colorado EDUCARE model in a few counties. o The stakeholder group is in the process of developing Quality Rating Systems for family child care, center-based programs, and after- school programs.

Five State Implementation Iowa Following a request from the legislators in the 2003 legislative session, the State Child Care Advisory Council (SCCAC) brought together a work group to study Quality Rating Systems in other states. SCCAC then made recommendations for Iowa. Since that time: o SCCAC conducted an extensive study of existing QRS models and made recommendations for an Iowa model. o To more fully develop the recommendations, the State Public Policy Group facilitated public input on the QRS model. o The Iowa model was recently approved to be piloted as part of the MCCRC study.

Five State Implementation Nebraska & Mississippi As part of the MCCRC study of differing Quality Rating Systems, Nebraska and Mississippi researchers and state agency partners are working within their states to develop Quality Rating System models. These models will be piloted during year 2 of the study.

Five State Implementation Kansas Kansas researchers and state agency partners are using the Colorado EDUCARE model. This model is a 5-level rating system for use in programs serving children birth–age 5. The model calls for an initial rating/assessment followed by training and technical assistance. The components of the model are: o Learning Environment o Family Partnerships o Training & Education o Adult-child Ratios/ Group Size o Program Accreditation