Quality in formative assessment Symposium at the 2008 meeting of AERA, New York, NY Discussant: Dylan Wiliam www.dylanwiliam.net

Formative assessment Formative assessment involves the creation of, and the capitalization upon, moments of contingency in instruction

What gets formed? Long-cycle Span: across units, terms Length: four weeks to one year Impact: Student monitoring; curriculum alignment Medium-cycle Span: within and between teaching units Length: one to four weeks Impact: Improved, student-involved, assessment; teacher cognition about learning; before end-of-unit adjustment to instruction. Short-cycle Span: within and between lessons Length: day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours Impact: classroom practice; student engagement

For whom? Individual students Groups of students on whom data were collected Other groups of students

Theory of action Evidence of student learning is elicited (by teachers, peers, or by the learners themsleves) This evidence is intepreted in terms of a theory of learning Decisions about next steps in instruction are taken that are either better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited

Competing definitions of quality What is effective formative assessment? Data is generated for the purpose of improving instruction (intention) Data is generated that is likely to improve instruction (prediction) Under any conditions Under commonly encountered conditions Under specified conditions Data is generated that actually improves instruction (action)

Evidence collection Kinds of assessment in support of learning Monitoring Whether the required learning is taking place Diagnostic When the required learning is not taking place, what is not being learned Formative When the required learning is not taking place, what needs to be done to improve the situation Evidence collection must be driven by a theory of learning

Learning progressions “Unpacking” state standards Horizontal unpacking—operationalization for testing Vertical unpacking—operationalization for learning Learning progressions Are rarely independent of curriculum Require both empirical and theoretical rationales

The role of content knowledge Knowing what the correct answer is Knowing what incorrect answers indicate Knowing what instructional activities are likely to move learning on Knowing what issues are worth bothering about

Specific comments Herman & Choi An important existence proof A particularly well understood, and curriculum-independent, domain How much accuracy is needed? Heritage, Kim & Vendlinski Importance of teacher x principle x task effects Planning next instruction is hard Requires an “anatomy of quality” Wylie & Ciofalo Competing priorities Minimizing false positives Interpretability of true negatives