1. 1 Assessment of Learning Outcomes: conceptual and practical challenges Paul Black Department of Education King’s College London

2. 2 Validity Problematic Users have diverse purposes and interpretations Three cases –Maths and English in school –Learning (How) To Learn –Workplace assessment UK project – ASLO –Assessment of Significant Learning Outcomes

3. 3 Case A1: Mathematics in UK Schools Conflicting aims: –Algorithms – clear criteria, easy to assess –Real problems – diverse criteria, hard to assess –Construct – what does ‘being good at maths’ mean ? –Assessment by teachers needed but not wanted –Accountability pressures – power of test agencies Conflicts reflected in measures: –Alignment with which aims ? –Assessment by teachers needed but not wanted –Accountability pressures – power of test agencies

4. 4 Case A2: English in UK Schools Teachers agree on aims –Construct implies diversity ‘horizon’ of good performance –No explicit rules - assessment holistic –A ‘community of practice’ Alignment of measures – Assessment by teachers in informal contexts – Comparability of evidence, of standards – Public trust ? – Cost

5. 5 Case B: Learning (How) To Learn LHTL Self report measures - Consistent with behaviour? Link to attainment? Attainment measures – use tasks involving new learning - Strong variation with context - No agreement on interpretation If alignment with measures problematic – - Perhaps construct not a unitary one - LHTL: a collection of diverse learning practices

6. 6 Case C: Workplace learning Diversity and complexity – within & between occupations Strongly context dependent – knowledge tacit, performance contingent Codifying in explicit schedules of skills - limited value –Out of context ‘surrogate tests’ mainly invalid –Assessment to be in workplace context Final summative picture vs. –Snapshot of performance in a learning progression

7. 7 Reflections Meaning of the construct –Agreement or conflict? Conflicts of purpose. –Explicit criteria, analytic, clear to all or: –Holistic, understood within a community Securing alignment – Measures help define the construct – Formal test situations or informal in realistic contexts ? – Assessment criteria explicit or contingent ? – Feasibility, manageability, trust, cost. Accountability. –To whom, and for what ? –Control of the measures = power over the construct Terminal picture vs. one scene in a progression