Formative Assessment and Student Achievement: Two Years of Implementation of the Keeping Learning on Track® Program Courtney Bell (ETS) Jonathan Steinberg (ETS) Dylan Wiliam (Institute of Education, University of London) Caroline Wylie (ETS)

Keeping Learning on Track ® Program Developed from research-basis around formative assessment (Crooks, 1997; Natriello, 1998; Kluger & DeNisi; 1996; Black & Wiliam, 1998c; Nyquist, 2003; Brookhart, 2005) and the power of teacher learning communities (Grossman, Wineburg, & Woolworth 2001; Thompson, & Goe, 2006) Five research-based strategies (Leahy et al., 2005) Engineering effective classroom discussions, questions, and learning tasks that elicit evidence of student learning; Clarifying and sharing learning intentions and criteria for success; Providing feedback that moves learners forward; Activating students as the owners of their own learning; Activating students as instructional resources for one another.

KLT Components 2-day introductory workshop 2-day TLC leader workshop Monthly school-based TLC meetings Structured and supported by TLC Leader modules Sufficient content for two years of monthly meetings

Working in Cleveland Work began with 10 schools (Cohort A) Nine of the original schools (1 closed) continued Additional 5 new schools (Cohort B) All Schools All in third or fourth year of failing to make Adequate Yearly Progress

Details of sample Numbers of students in each grade in CMSD, and

Data Used and Ohio Achievement Tests for Grades 3-8 Reliability Math: 0.85 to 0.90 Reading: 0.87 to 0.89 CMSD scores well below state mean 0.65 to 0.83 sd below state mean in math 0.60 to 0.71 sd below state mean in reading High student mobility 4400 of students changed school from to students were retained in grade or were promoted by more than one grade from to Analysis excluded these students No data available on: Teacher assignment Fidelity of implementation

Data analysis Univariate HLM Dependent variable: test scores Students nested in schools, schools nested in treatment scores used as a covariate No significant treatment effect p-values typically well over 0.5 Suggests weak effects rather than low power

Exploratory analyses Cleveland school data from the OH DOE website to identify matched schools Number of students Proportion of LEP students % of longevity (measure of teacher turnover) % of African American students Accountability designation in (defined by state accountability system) Year of improvement status (defined by state accountability system) % of core courses not taught by a highly qualified teacher If a tie-breaker was needed, the final category was % of special education students

Exploratory results: math KLTNon-KLT Grade1 year2 yearAll1 year2 yearp-value Model coefficients for math scores, in standard deviations (positive/negative)

Exploratory results: reading KLTNon-KLT Grade1 year2 yearAll1 year2 yearp-value Model coefficients for reading scores, in standard deviations (positive/negative)

Comparison of matched schools Grade 1 year2 year KLT schools - matched non-KLT schools (Coefficients for reading scores in standard deviations)

Discussion No statistically significant effect of KLT Exploratory analyses Math: no evidence of effect Reading: some evidence in earlier grades Limitations Fidelity of implementation Small effects Low statistical power Next steps Research designs based on “theory of action” of KLT Impact of training on teacher behaviors Impact of changes in teacher behaviors on students:  Attitudes  Outcomes

For More Information To view, download and print the papers: Conference files will be posted on the web for one month