1. Once you know what they’ve learned, what do you do next? Designing curriculum and assessment for growth Dylan Wiliam Institute of Education, University of London www.dylanwiliam.net Presentation to MDSE/MARCES conference; University of Maryland, College Park, MD; October 2006

2. Outline Education reform in England and Wales Designing an assessment system to support learning Age-independent levels of achievement Distribution of achievement over time Applications to curriculum specification

3. A familiar story Education Reform Act (1988) –An early attempt to use markets to reform education Choice Diversity Standardization Information

4. Key features of ERA Basic curriculum: –Religious education (!) –Core subjects (English, Math, Science) –Non-core subjects (7 in all) Four “key stages” (5-7, 7-11, 11-14, 14-16) Core subjects assessed at end of each key stage Other subjects assessed at some key stages

5. Task Group on Assessment and Testing (TGAT) To advise the Secretary of State on the practical considerations which should govern all assessment including testing of attainment at age (approximately) 7, 11, 14 and 16, within a national curriculum; including the marking scale or scales and kinds of assessment including testing to be used, the need to differentiate so that assessment can promote learning across a range of abilities, the relative roles of informative and of diagnostic assessment, the uses to which the results of assessment should be put, the moderation requirements needed to secure credibility for assessments, and the publication and other services needed to support the system – with a view to securing assessment and testing arrangements which are simple to administer, understandable by all in and outside the education service, cost- effective, and supportive of learning in schools.

6. Task Group on Assessment and Testing (TGAT) Basic choice –Age-dependent “benchmark” assessments at each age-point –Age-independent linked system of achievement levels across ages Crucial factors –Technical feasibility –Impact on students

7. Age-dependent levels Simple to understand Familiar Significant negative impact on student motivation Encourages a notion of ability as “fixed” rather than incremental

8. Age-independent levels In psychology –Piaget (Shayer et al., 1976; Shayer & Wylam, 1978) –Pascual-Leone –Case –SOLO (Biggs & Collis, 1982) –Van Hiele –CSMS (Hart, 1981) In Education (or math education at least!) –The “Dalton Plan” (Parkhurst, 1922) –Kent Mathematics Project (Banks, 1991) –Secondary Mathematics Individualised Learning Experiment –Graded Assessment in Mathematics (Brown, 1992)

9. Preliminary evidence 6099 + 1 = ? (Foxman et al., 1980) –Correctly answered by some 7-year-olds –Incorrectly answered by some 14-year-olds The “seven year gap” (Cockcroft, 1981) Progression in measuring (Simon et al., 1995) Spread of achievement in an age cohort apparently much greater than generally assumed

10. CSMS (Hart, 1981)

11. Sequential tests of educational progress (ETS, 1957)

12. Sensitivity to instruction 1 year Distribution of attainment on an item highly sensitive to instruction

13. Sensitivity to instruction (2) 1 year Distribution of attainment on an item moderately sensitive to instruction

14. Sensitivity to instruction (3) 1 year Distribution of attainment on an item relatively insensitive to instruction

15. Sensitivity to instruction (4) TestSensitivity index IQ-type test (insensitive)0 NAEP6 TIMSS8 ETS “STEP” tests (1957)8 Completely sensitive test100

16. Insensitivity to instruction Artifact or reality? –Influenced by test construction procedures –Influenced by approaches to curriculum –Dimensions of progression Reasoning power Curriculum exposure Maturity

17. Nature of hierarchies Hierarchies are partly arbitrary –Division can precede multiplication –Integration can precede differentiation Hierarchies are partly psychological –Some learning sequences appear inevitable Writing Number skills

18. Years of secondary schooling Proportion of age cohort Graded Assessment in Mathematics Intended for all school students, aged 11 to 16 Design requirement: all students should be able to increase by one level per year Upper levels designed to be equivalent to existing national examinations

19. ITBS language usage test Grade equivalent Percentile

20. A very simple model Achievement age is normally distributed about chronological age, with a standard deviation proportional to the chronological age Constant of proportionality varies from around one-sixth to one-half, depending on the kind of curriculum and assessment

24. Standardized tests

25. The TGAT model StageAgesLevels 15-71-3 27-112-6 311-143-8 414-164-10

26. Curriculum development Curriculum developers forced to focus on “What develops?” Models of curriculum –Grade-based models (France, Germany) –Social promotion (England, Japan, Sweden) –Hybrid models (USA) Models of differentiation –Same goals, same curriculum, different speeds –Same goals, different curriculum –Different goals Models of progression –Good in math, design technology –OK in language arts, science –Poor in history Dimensions of progression –Mathematics: reasoning power –Science: curriculum exposure –English: maturity

27. Hierarchies in science 1.Know that light comes from different sources 2.Know that light passes through some materials and not others, and that when it does not, shadows may be formed 3.Know that light can be made to change direction, and that shiny surfaces can form images 4.Know that light travels in straight lines, and this can be used to explain the formation of shadows 5.Understand how light is reflected 6.Understand how prisms and lenses refract and disperse light 7.Be able to describe how simple optical devices work 8.Understand refraction as an effect of differences of velocities in different media 9. 10.Understand the processes of dispersion, interference, diffraction and polarisation of light

28. Strengths Forces a focus on progression in “big ideas” rather than coverage Supports incremental, rather than entity view of ability Supports strong value-added inferences

29. Weaknesses Some subjects fit the model better than others Some (accepted) models of curriculum become non-viable Requires careful articulation between curriculum, standards, and assessment May focus on aspects relatively insensitive to instruction