Knowing What Students Know: The Science and Design of Educational Assessment Committee on the Foundations of Assessment Board on Testing and Assessment, Center for Education National Research Council

Implications of New Foundations for Assessment Design Recursive process Begin with model of cognition and learning in the domain  Examples:  development of number sense (Case & Griffin);  subtraction bugs (Brown & Burton);  ACT -R theory (Anderson et al.);  Facets in physics (Minstrell & Hunt);  Middle School Math through Applications (Greeno et al.);  CRESST models;  Australia’s progress maps (ACER)

Design of Observations Tasks (and scoring) guided by cognition and interpretation models  Examples:  Number Knowledge Test (Case & Griffin)  CAETI simulation & expert scoring system Sometimes sets of tasks useful for diagnosis  Example:  Diagnoser (Minstrell)

Validation Should include evidence that tasks actually tap the intended knowledge and cognitive processes  methods: e.g., think alouds, analyses of errors  Examples:  cognitive complexity of science performance tasks (Baxter & Glaser);  QUASAR (Lane & Silver)  Standards for Educational and Psychological Tests. (1999)

Reporting Assessments must be designed from the start to ensure that reporting of desired types of information will be possible Results should be reported relative to model of learning  Example: Australia’s progress maps (ACER; Queensland)

A Way of Thinking About Fairness Methods of conditional inference (ways of taking into account info about students’ instructional histories) 1.Control instruction and/or task to ensure task-examinee match up 2.Obtain background info about student’s instruction 3.Allow student choice of tasks in light of what they know about themselves 4. Minimize irrelevant variance

Implications of New Foundations for Assessment Practice In the classroom: assessment should be integral part of instruction students should get information about particular qualities of their work and what they can do to improve students understand learning goals and landmark performances along the way cognitive science findings need to be made teacher-friendly

Assessment Practice In large-scale testing contexts: proposed approach holds promise for drawing more valid and fair inferences about student achievement but to fully capitalize on advances, need to relax constraints that drive current practices feasibility & scalability in this policy context should be demonstrated  some alternatives: population sampling (e.g., Maryland), curriculum-embedded (e.g., AP Art)

Assessment Practice Committee envisions systems of assessments that are:  comprehensive  coherent  continuous Shift emphasis toward classroom where learning occurs and where information support is most needed