1. Embedding formative assessment Disc 1 presentation Sustaining assessment for learning with teacher learning communities

2. Overview of the day Why raising achievement is important
Why investing in teachers is the answer
Why formative assessment should be the focus
Why teacher learning communities should be the mechanism
How we can put this into practice

3. Raising achievement matters…
For individuals
Increased lifetime salary
Improved health
For society
Lower criminal justice costs
Lower health care costs
Increased economic growth

4. …now more than ever…
Source: Economic Policy Institute

5. Which of the following categories of skill is disappearing from the workplace most rapidly?
Routine manual
Non-routine manual
Routine cognitive
Complex communication
Expert thinking/problem-solving

6. …but quality matters too
Autor, Levy & Murnane, 2003

7. Where’s the solution? Structure Governance Smaller secondary schools
Larger secondary schools
Creating/getting rid of middle schools
Alignment
Curriculum reform
Textbook replacement
Governance
Specialist schools
Academies
Technology
Computers
Interactive whiteboards

8. School effectiveness Three generations of effectiveness research
Raw results approaches
Different schools get different results
Conclusion: schools make a difference
Demographic-based approaches
Demographic factors account for most of the variation
Conclusion: schools don’t make a difference
Value-added approaches
School-level differences in value-added are relatively small
Classroom-level differences in value-added are large
Conclusion: an effective school is little more than a school full of effective classrooms

9. It’s the classroom
Variability at the classroom level is up to four times greater than at school level
It’s not class size
It’s not the between-class grouping strategy
It’s not the within-class grouping strategy
It’s the teacher

10. Teacher quality A labour force issue with two solutions
Replace existing teachers with better ones?
No evidence that more pay brings in better teachers
No evidence that there are better teachers out there deterred by certification requirements
Improve the effectiveness of existing teachers
The ‘love the one you’re with’ strategy
It can be done
We know how to do it, but at scale? Quickly? Sustainably?

11. Extra months of learning per year
Cost/effect comparisons
Intervention
Extra months of learning per year
Cost per year
Class-size reduction (by 30%) 4
£20k
Increase teacher content knowledge by 2 sd 2 ?
Formative assessment/ assessment for learning 8
£2k

12. The research evidence Several major reviews of the research
Natriello (1987)
Crooks (1988)
Kluger & DeNisi (1996)
Black & Wiliam (1998)
Nyquist (2003)
All find consistent, substantial effects

13. Formative assessment (Black et al., 2002)
Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning. It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence. An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils, in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged. Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.

14. Types of formative assessment
Long-cycle
Span: across units, terms
Length: four weeks to one year
Medium-cycle
Span: within and between teaching units
Length: one to four weeks
Short-cycle
Span: within and between lessons
Length:
Day-by-day: 24 to 48 hours
Minute-by-minute: five seconds to two hours

15. Effects of formative assessment
Long-cycle
Student monitoring
Curriculum alignment
Medium-cycle
Improved, student-involved, assessment
Improved teacher cognition about learning
Short-cycle
Improved classroom practice
Improved student engagement

16. Unpacking formative assessment
Key processes
Establishing where the learners are in their learning
Establishing where they are going
Working out how to get there
Participants
Teachers
Peers
Learners

17. Where the learner is going
Aspects of formative assessment
Where the learner is going
Where the learner is
How to get there
Teacher
Clarify and share learning intentions
Engineering effective discussions, tasks and activities that elicit evidence of learning
Providing feedback that moves learners forward
Peer
Understand and share learning intentions
Activating students as learning
resources for one another
Learner
Understand learning intentions
Activating students as owners of their own learning
Adapted with permission of ETS

18. Five ‘key strategies’...
Clarifying, understanding, and sharing learning intentions
Curriculum philosophy
Engineering effective classroom discussions, tasks and activities that elicit evidence of learning
Classroom discourse, interactive whole-class teaching
Providing feedback that moves learners forward
Feedback
Activating students as learning resources for one another
Collaborative learning, reciprocal teaching, peer-assessment
Activating students as owners of their own learning
Metacognition, motivation, interest, attribution, self-assessment
Wiliam & Thompson (2007), adapted with permission of ETS

19. Use evidence about learning to adapt teaching and learning
...and one big idea
Use evidence about learning
to adapt teaching and learning
to meet student needs

20. An educational positioning system
A good teacher:
establishes where the students are in their learning
identifies the learning destination
carefully plans a route
begins the learning journey
makes regular checks on progress on the way
makes adjustments to the course as conditions dictate

21. Engineering effective classroom discussions, tasks and activities that elicit evidence of learning

22. Kinds of questions: Israel
Which fraction is the smallest?
Success rate 88%
Which fraction is the largest?
Success rate 46%; 39% choose b)
Vinner, PME conference, Lahti, Finland 1997

23. Misconceptions

24. Misconceptions
3a = 24
a + b = 16

25. Molecular structure of water?

26. Feedback that moves learning forward

27. Kinds of feedback: Israel
264 low and high ability grade 6 students in 12 classes in 4 schools; analysis of 132 students at the top and bottom of each class
Same teaching, same aims, same teachers, same classwork
Three kinds of feedback: scores, comments, scores + comments
Feedback
Gain
Attitude
Scores
None
Top
+ve
Bottom
-ve
Comments
30%
All
+ve
[Butler (1988) Br.J.Educ.Pyschol., ]

28. Responses Feedback Gain Attitude Scores None Top +ve Bottom -ve
Comments
30%
All
What do you think happened for the students given both scores and comments?
Gain: 30%; Attitude: all +ve
Gain: 30%; Attitude: top +ve, bottom –ve
Gain: 0%; Attitude: all +ve
Gain: 0%; Attitude: top +ve, bottom –ve
Something else
[Butler (1988) Br.J.Educ.Pyschol., ]

29. Kinds of feedback: Israel (2)
200 grade 5 and 6 Israeli students
Divergent thinking tasks
4 matched groups:
experimental group 1 (EG1); comments
experimental group 2 (EG2); grades
experimental group 3 (EG3); praise
control group (CG); no feedback
Achievement
EG1>(EG2≈EG3≈CG)
Ego-involvement
(EG2≈EG3)>(EG1≈CG)
[Butler (1987) J.Educ.Pyschol., ]

30. Effects of feedback Kluger & DeNisi (1996)
Review of 3000 research reports
Excluding those:
without adequate controls
with poor design
with fewer than 10 participants
where performance was not measured
without details of effect sizes
Left 131 reports, 607 effect sizes, involving individuals
Average effect size 0.4, but
effect sizes were variable
40% of effect sizes were negative

31. Feedback Formative assessment requires
data on the actual level of some measurable attribute
data on the reference level of that attribute
a mechanism for comparing the two levels and generating information about the ‘gap’ between the two levels
a mechanism by which the information can be used to alter the gap
Feedback is therefore formative only if the information fed back is actually used in closing the gap

32. Formative assessment Frequent feedback is not necessarily formative
Feedback that causes improvement is not necessarily formative
Assessment is formative only if the information fed back to the learner is used by the learner in making improvements
To be formative, assessment must include a recipe for future action

33. How do students make sense of this?
Attribution (Dweck, 2000)
Personalisation (internal v external)
Permanence (stable v unstable)
Essential that students attribute both failures and success to internal, unstable causes (it’s down to you, and you can do something about it)
Views of ‘ability’
Fixed (IQ)
Incremental (untapped potential)
Essential that teachers inculcate in their students a view that ‘ability’ is incremental rather than fixed (by working, you’re getting smarter)

34. Sharing learning intentions

35. Sharing criteria with learners
Three teachers each training four year 8 science classes in two US schools
14 week experiment
Seven two-week projects, scored 2-10
All teaching the same, except:
for a part of each week
two of each teacher’s classes discuss their likes and dislikes about the teaching (control)
the other two classes discuss how their work will be assessed
[White & Frederiksen, Cognition & Instruction, 16(1), 1998]

36. Sharing criteria with learners
Iowa Test of Basic Skills
Group
Low
Middle
High
Likes and dislikes
Reflective assessment

37. Activating students as owners of their own learning Activating students as learning resources for one another

38. Self-assessment: Portugal
Teachers studying for MA in Education
Group 1 do regular programme
Group 2 work on self-assessment for two terms (20 weeks)
Teachers matched in age, qualifications and experience using the same curriculum scheme for the same amount of time
Pupils tested at beginning of year, and again after two terms
Group 1 pupils improve by 7.8 marks
Group 2 pupils improve by 15 marks
[Fontana & Fernandez, Br.J.Educ.Pyschol., ]

39. Comments? Questions?

40. Practical techniques

41. Practical techniques: eliciting evidence
Key idea: questioning should
cause thinking
provide data that informs teaching
Improving teacher questioning
Generating questions with colleagues
Closed v open
Low-order v high-order
Appropriate wait time
Getting away from I-R-E
Basketball rather than serial table tennis
‘No hands up’ (except to ask a question)
Class polls to review current attitudes towards an issue
‘Hot seat’ questioning
In many classrooms teachers use questions as a way of directing the attention of the class, and keeping students ‘on task’. This may keep students ‘on their toes’ but rarely helps learning. It may be better, in a whole-class lesson, to have an extended exchange with a single student. This ‘hot seat’ questioning also has benefits for other students who appear to learn vicariously from the exchange. Other students may see extended exchanges between the teacher and another student as a chance to relax and go ‘off task’, but if the teacher asks them what they have learned from a particular exchange between another student and the teacher, their concentration is likely to be quite high!
How much time a teacher allows a student to respond before evaluating the response is also important. It is well known that teachers do not allow students much time to answer questions, and, if they don’t receive a response quickly, they will ‘help’ the student by providing a clue or weakening the question in some way, or even moving on to another student. However, what is not widely appreciated is that the amount of time between the student providing an answer and the teacher’s evaluation of that answer is much more important. Of course, where the question is a simple matter of factual recall, then allowing a student time to reflect and expand upon the answer is unlikely to help much. But where the question requires thought, then increasing the time between the end of the student’s answer and the teacher’s evaluation from the average ‘wait-time’ of less than a second to three seconds, produces measurable increases in learning (although increases beyond three seconds have little effect, and may cause lessons to lose pace).
In fact, questions need not always come from the teacher. There is substantial evidence that students’ learning is enhanced by getting them to generate their own questions. If, instead of writing an end-of-topic test herself, the teacher asks the students to write a test that tests the work the class has been doing, the teacher can gather useful evidence about what the students think they have been learning, which is often very different from what the teacher thinks the class has been learning. This can be a particularly effective strategy with disaffected older students, who often feel threatened by tests. Asking them to write a test for the topic they have completed, and making clear that the teacher is going to mark the question rather than the answers, can be a hugely liberating experience for many students.
All student response systems
ABCD cards, mini whiteboards, exit passes

42. Questioning in maths: discussion
Look at the following sequence
3, 7, 11, 15, 19, ...
Which is the best rule to describe the sequence? A.
n + 4 B.
3 + n C.
4n – 1 D.
4n + 3

43. Questioning in maths: diagnosis
In which of these right-angled triangles is a2 + b2 = c2 ? A a c b C E B D F

44. Questioning in science: discussion
Ice-cubes are added to a glass of water. What happens to the level of water as the ice-cubes melt? A.
The level of the water drops B.
The level of the water stays the same C.
The level of the water increases D.
You need more information to be sure

45. Questioning in science: diagnosis
The ball sitting on the table is not moving. It is not moving because: A.
no forces are pushing or pulling on the ball B.
gravity is pulling down, but the table is in the way C.
the table pushes up with the same force that gravity pulls down D.
gravity is holding it onto the table E.
there is a force inside the ball keeping it from rolling off the table
Wilson & Draney, 2004

46. Dinosaur extinction Why did dinosaurs become extinct? A.
Humans destroyed their habitat B.
Humans killed them all for food C.
There was a major change in climate

47. Save the ozone layer What can we do to preserve the ozone layer? A.
Reduce the amount of carbon dioxide produced by cars and factories B.
Reduce the greenhouse effect C.
Stop cutting down the rainforests D.
Limit the number of cars that can be used when the level of ozone is high E.
Properly dispose of air-conditioners and fridges

48. Questioning in English: discussion
Macbeth: mad or bad?

49. The dog ran across the road
Questioning in English: discussion
Where is the verb in this sentence?
The dog ran across the road A B D C

50. Questioning in English: diagnosis
Which of these is the best thesis statement? A.
The typical TV show has nine violent incidents B.
The essay I am going to write is about violence on TV C.
There is a lot of violence on TV D.
The amount of violence on TV should be reduced E.
Some programmes are more violent than others F.
Violence is included in programmes to boost ratings G.
Violence on TV is interesting H.
I don’t like the violence on TV

51. Questioning in history: discussion
In which year did World War II begin? A.
1919 B.
1937 C.
1938 D.
1939 E.
1941

52. Questioning in history: diagnosis
Why are historians concerned with bias when analysing sources? A.
People can never be trusted to tell the truth B.
People deliberately leave out important details C.
People are only able to provide meaningful information if they experienced an event first hand D.
People interpret the same event in different ways, according to their experience E.
People are unaware of the motivations for their actions F.
People get confused about sequences of events

53. Questioning in MFL: discussion
Is the verb ‘être’ regular in French?

54. Questioning in MFL: diagnosis
Which of the following is the correct translation for ‘I give the book to him’? A.
Yo lo doy el libro B.
Yo doy le el libro C.
Yo le doy el libro D.
Yo doy lo el libro E.
Yo doy el libro le F.
Yo doy el libro lo
This item is diagnostic because it has been designed so that if pupils answer incorrectly, it is easy to work out why. Response A indicates a pronoun error, responses B and E indicate placement errors, and responses D and F indicate both pronoun and placement errors.

55. Hinge-questions
A hinge-question is based on the important concept in a lesson that is critical for students to understand before you move on in the lesson
The question should fall about midway during the lesson
Every student must respond to the question within two minutes
You must be able to collect and interpret the responses from all students in 30 seconds

56. Figurative language A. Alliteration 1 He was a bull in a china shop B.
Hyperbole 2
May I have a drop of water? C.
Irony 3
This backpack weighs a ton D.
Metaphor 4
The sweetly smiling sunshine… E.
Onomatopoeia 5
He honked his horn at the cyclist F.
Personification 6
I’ve told you a million times already G.
Simile 7
The Redcoats are coming! H.
None of the above 8
‘They in the sea being burnt, they in the burnt ship drown’d’ 9
He was as tall as a house

57. Triangle Shirtwaist Factory fire, March 25,1911

58. Triangle Factory fire Which of the following sources is biased? A.
Photograph of the event B.
New York Times story on March 26, 1911 C.
Description of the fire in the textbook D.
Transcript of talk by Frances Perkins, September 30,1964

59. Practical techniques: feedback
Key idea: feedback should
cause thinking
provide guidance on how to improve
Comment-only grading
Focused grading
Explicit reference to mark-schemes and scoring guides
Suggestions on how to improve
‘Strategy cards’ - ideas for improvement
Not giving complete solutions
Re-timing assessment
(e.g. two-thirds-of-the-way-through-a-unit test)
The research evidence suggests that feedback in terms of scores, grades and levels is unlikely to improve achievement, but that feeding back in terms of comments (whether written or verbal) is. This immediately raises the question ‘what kind of comments’, and although there is no specific research evidence on this point, it seems that, to be useful, a comment should cause thinking to take place.
This feature of ‘mindfulness’ is one of the crucial features of effective formative assessment - effective learning involves having most of the students thinking most of the time. This notion of ‘mindfulness’ also gives some clues about what sort of marking is most helpful. Many teachers say that formative feedback is less useful in mathematics, because an answer is either wrong or right. But even where answers are wrong or right, we can still encourage students to think. For example, rather than marking answers right and wrong and telling the students to do corrections, teachers could, instead, feed back saying simply ‘Three of these ten questions are wrong. Find out which ones and correct them’. After all, we are often telling our students to check their work, but rarely help them develop the skills to do so.
Other strategies that are useful are focused grading – i.e. grading a particular piece of work for one aspect (such as sentence structure, expression or spelling) rather than trying to correct everything. This is particularly useful if the comments can be related directly to the assessment criteria for the work.
Of course, it is very difficult for feedback to function formatively at the end of a unit, so rather than an ‘end-of-unit test’ it may be more useful to have a ‘two-thirds-of-the-way-through-a-unit test’. Those students who have understood something can then help those who haven’t. Many teachers sometimes worry that such strategies may hold back abler students, but the research evidence suggests that it is the students who give help who benefit most from such peer-tutoring. While this may not accelerate more able students through the curriculum, it does lead to better long-term retention.

60. Practical techniques: sharing learning intentions
Explaining learning intentions at start of lesson/unit
Learning intentions
Success criteria
Intentions/criteria in students’ language
Posters of key words to talk about learning
e.g. describe, explain, evaluate
Planning/writing frames
Annotated examples of different standards to ‘flesh out’ assessment rubrics (e.g. lab reports)
Opportunities for students to design their own tests
Most of these strategies are self-explanatory. Planning and writing frames provide a structure to help students develop a response. While some teachers see such frames as constricting, for most students they provide valuable ‘scaffolding’ for their answers.

61. Practical techniques: students owning their learning and as learning resources
Students assessing their own/peers’ work
with rubrics
with exemplars
‘two stars and a wish’
Training students to pose questions/identifying group weaknesses
Self-assessment of understanding
Traffic lights
Red/green discs
End-of-lesson students’ review
Again, most of these strategies are self-explanatory. With ‘traffic lights’ the teacher identifies a small number of objectives for the lesson (perhaps only one), which are made as clear as possible to the students at the beginning of the lesson. At the end of the lesson, students are asked to indicate their understanding of each objective by a green, yellow or red circle, according to whether they feel they have achieved the objective fully, partially or not at all. This provides useful feedback to the teacher at two levels - to see if there are parts of the lesson that it would be worth re-doing with the whole class, but also to get feedback about which students would particularly benefit from individual support (one technique here has been to ask ambers and greens to work together, while the teacher works with the reds). However, the real benefit of such a system is that it forces the student to reflect on what she or he has been learning. Other teachers have used ‘smiley faces’ which have the advantage, if drawn in pencil, of being modifiable when the student is more confident about their understanding.
The other strategy that may require explanation is that of ‘end-of-lesson’ reviews. The idea here is that at the beginning of the lesson, one student is appointed as a ‘rapporteur’ for the lesson. The teacher then teaches a whole-class lesson on some topic, and finishes the lesson ten or fifteen minutes before the end of the lesson. The student rapporteur then gives a summary of the main points of the lesson, and tries to answer any remaining questions that students in the class may have. If he or she can’t answer the questions, then the rapporteur asks members of the class to help out. What is surprising is that teachers who have tried this out have found that students are queuing up to play the role of rapporteur, provided this is started at the beginning of the school year, or even better, when students are new to the school.