The Relationship between Fidelity of Implementation and Classroom Quality in Early Childhood Education Katerina Sergi, M.A.1, Giorgio Carlo Cappello, Ph.D.1, Candice Pittman, Ph.D.2, Shrinidhi Ambinakudige, Ph.D.1, 3, April May, M.A.4, and Carolyn Adams-Price, Ph.D.1, 5 1National Strategic Planning and Analysis Research Center, 2Mississippi Department of Human Services, 3Department of Geosciences, 4Mississippi Building Blocks, 5Department of Psychology What is MBB? Mississippi Building Blocks (MBB) is an intervention program designed to improve early care and education of children in child care centers across Mississippi. Currently in its eighth year, the research-based intervention program is implemented in preschools with the recognition that the high poverty rate of families leaves a large percentage of young children unprepared for kindergarten at age 5. The pre-testpost-test design involves a treatment and a control group. Teachers in pre-kindergarten classrooms receive extensive individualized coaching on the implementation of a comprehensive literacy and mathematics curriculum, designed to prepare children in child care centers for kindergarten. The MBB intervention program provides linkages between early childhood teacher scores on fidelity of implementation assessment and children’s improvement on standardized tests from pre-test to post-test (Adams-Price, 2016). Significance of Study Understand the factors that determine the degree to which teachers, who are coached, become skilled at implementing a systematic literacy curriculum. Research Objective Explore the relationship between fidelity of implementation and the overall quality of early childhood classrooms. Research Questions What is the relationship between teachers’ fidelity of implementation of a new curriculum and classroom quality? Is there a correlation between faithful implementation of a curriculum and an outside assessor’s judgment of the literacy environment of the classroom? Methods Sample Pre-kindergarten, treatment classrooms of 3 and 4 year old children in years 2013-2014 and 2014-2015. We combined data from a span of two years to get an adequate sample size for our analyses. Children were students in classrooms from 30 different treatment centers. Research Design Classrooms were assessed before and after the teachers in those classrooms received extensive coaching on the implementation of a comprehensive literacy and mathematics curriculum. Following the ELLCO assessments, the teachers received materials and instructions for improving the classroom environment. Then the same teachers started receiving coaching from MBB staff on implementing the literacy and mathematics curriculum. ELLCO and Fidelity assessments were conducted in fall (before coaching started) and spring (after coaching) of a given school year. Measures Fidelity of Implementation Assessment. Fidelity measures the degree to which teacher has implemented the MBB curriculum. Teachers were assessed by supervisors associated with the MBB intervention program. Dose and duration subscale – as measured by the amount and frequency of using the MBB curriculum. Quality of delivery subscale – as measured by the teacher-child interactions (language modeling and literacy focus) and intended outcomes. Early Language & Literacy Classroom Observation (ELLCO). ELLCO measures the extent to which the classroom environment promotes increases in early literacy skills in children. The assessment was conducted by trained on the ELLCO measure independent assessors. The two subscales of the ELLCO are: General Classroom Environment – Classroom structure subscale: classroom organization, classroom contents, classroom management, personnel. Curriculum subscale: approaches to curriculum, opportunities for child choice and initiative, recognizing diversity. Language and Literacy – Language environment subscale: discourse climate, opportunities for extended conversations, efforts to build vocabulary, phonological awareness. Books and book reading subscale: organization of book area, book characteristics, books for learning, approaches to book reading, quality of book reading. Print and early writing subscale: early writing environment, support for children’s writing, environmental print. Data Analysis Quantitative analyses: Descriptive statistics of Fidelity and ELLCO scores (Tables 1 and 2). Partial correlations to measure the strength and the direction of the relationship between fidelity post-test scores and two subscales of the ELLCO post-test, while taking into account ELLCO pre-test scores (Figures 1, 2 and 3). Examine the extent to which the pre-test classroom and language environment (as measured by ELLCO) affects the relationship of the post-test fidelity and classroom and language environment. Discussion Dose/duration of implementation is significantly associated with general classroom environment, and specifically with classroom structure. Teachers who see improvements in their classroom environments also seem to display high fidelity. Dose/duration and quality of delivery of implementation are significantly associated with language environment. Teachers who faithfully implement the curriculum create a language environment conducive to children’s early language skills. Effective implementation of an intervention program can have a positive impact on teachers’ practices and the quality of classroom and language environments. Future research can examine the relationship between ELLCO scores and narratives. Results Figure 1. Partial Correlations between Post-Test Fidelity and Post-Test ELLCO Subscales, Controlling for pre-ELLCO Scores (Two Years) Table 1. Fidelity Assessment Scores in Treatment Group n=30 2013-2014 2014-2015 Pre-Test Post-Test M SD Dose / Duration Scoring: 1=inadequate; 2=minimal; 3=good 0.87 0.70 2.51 0.35 0.69 0.40 1.94 0.62 Quality of Delivery Scoring: 1=inadequate; 3=minimal; 5=good; 7=excellent 1.74 1.11 5.31 1.09 1.56 0.72 4.41 1.59 Table 2. ELLCO Assessment Scores in Treatment Group n=30 2013-2014 2014-2015 Pre-Test Post-Test M SD General Classroom Environment Scoring: 1=deficient; 2=inadequate; 3=basic; 4=strong; 5=exemplary 2.76 0.88 3.36 3.09 1.06 3.60 1.38 Language and Literacy 2.38 0.81 3.12 2.92 0.96 3.64 1.55 Figure 2. Partial Correlations between Post-Test Fidelity and Post-Test ELLCO Classroom and Curriculum Subscales, Controlling for pre-ELLCO Scores (Two Years) Figure 3. Partial Correlations between Post-Test Fidelity and Post-Test ELLCO Language, Books, and Print Subscales, Controlling for pre-ELLCO Scores (Two Years) Contact Information: Katerina Sergi ksergi@nsparc.msstate.edu