From Monologue to Dialogue: enhancing feedback in learning

From Monologue to Dialogue: enhancing feedback in learning
Professor David Nicol Deputy-Director Centre for Academic Practice and Learning Enhancement (CAPLE Director, REAP project ( University of Strathclyde Queen’s University Belfast, Centre for Educational Development, 20th May 2009

NSS: Assessment and feedback (2008)
No Survey Statement England Scotland Northern Ireland 5. The criteria used in marking have been clear in advance 69 6. Assessment arrangements and marking have been fair 74 73 7. Feedback on my work has been prompt 56 51 53 8. I received detailed comments on my work 61 52 9. Feedback on my work has helped clarify things I did not understand 50 22. Overall, I am satisfied with the quality of the course 82 86 83

Feedback as ‘transmission’
Elite to mass higher education Quality of the input message (what is written by teachers) versus the quality of interaction Feedback as monologue trying to carry the burden of dialogue Students as passive NSS might lead to narrow focus for interventions

Feedback as ‘dialogue’
Contingent: in response to learners’ needs. Multi-faceted: deriving from many sources (peers) and available in different formats. Catalytic: triggering inner dialogues around disciplinary concepts and ideas. Motivating: encouraging feedback seeking rather than feedback avoidance behaviours. Actionable: expressed as changes in thinking and in action

Plan Background Re-engineering Assessment Practices (REAP) project
Concepts and ideas Suggestions to enhance practice Case study of practice from REAP Guidelines for implementation

Background Research: HE Academy 2004: Literature review: 7 principles of good assessment and feedback practice in relation to development of learner self-regulation (Nicol and Macfarlane-Dick, 2004: 2006) Local Implementations: Scottish Funding Council : The Reengineering Assessment Practices (REAP) project (£1m) Policy and strategy: University of Strathclyde Assessment Policy and Practice Guidelines Synthesis: QAA Scotland 2009: First Year Experience: Assessment and Feedback publication (66 pages) (

Re-engineering Assessment Practices project
Scottish Funding Council (£1m) Strathclyde, Glasgow and Glasgow Caledonian Large 1st year classes ( students) A range of disciplines (19 modules ~6000 students) Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offline Learning quality and teaching efficiencies Assessment for learner self-regulation

First Year: The academic experience
What is important in the first year? Coping with transition Understanding what is required Engagement with academic programmes Receiving support and feedback Experiences of success Feeling in control of own learning Belief that you can succeed A sense of belonging within the academic and social culture Based on research by Yorke (UK) and Tinto (US)

Background (1) Gibbs, G. & Simpson, C (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, 1, 3-31. See: Formative Assessment in Science Teaching (FAST) project at:

Gibbs and Simpson (2004) Assessment tasks [Conditions 1-4]
Capture enough study time (in and out of class) Are spread out evenly across timeline of study Lead to productive activity (deep vs surface) Communicate clear and high expectations i.e concern here is with ‘steers’ about how much work to do

Background (2) Literature Review
Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), Nicol, D & Milligan, C. (2006), Rethinking technology-supported assessment practices in relation to the seven principles of good feedback practice. In C. Bryan & K. Clegg, Innovative assessment in higher education, Routledge. Background Student Enhanced Learning through Effective Feedback [SENLEF] project funded by HE Academy REAP project:

Rethinking assessment and feedback
1. Consider self and peers as much as the teacher as sources of assessment and feedback Tap into different qualities than teacher can provide Saves time Provides considerable learning benefits (lifelong learning) 2. Focus on every step of the cycle: Understanding the task criteria (Sadler, 1983) Applying what was learned in action 3. Not just written feedback: Also oral, computer, vicarious, formal and informal

Seven principles of good feedback
Good feedback should: Clarify what good performance is (goals, criteria, standards). Facilitate the development of reflection and self-assessment in learning Deliver high quality feedback to students: that enables them to self-correct Encourage peer and student-teacher and peer dialogue around learning Encourage positive motivational beliefs & self esteem through assessment Provide opportunities to act on feedback Provide information to teachers that can be used to help shape their teaching (making learning visible) Source: Nicol and Macfarlane-Dick (2006)

Two super principles SUPER-PRINCIPLE 1: time and effort on task (structured engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions SUPER-PRINCIPLE 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the seven principles discussed above. Case examples from REAP – applying these conditions/ principles

Discussion points

Good feedback should: Clarify what good performance is (goals, criteria, standards). Facilitate the development of reflection and self-assessment in learning Deliver high quality feedback to students: that enables them to self-correct Encourage peer and student-teacher and peer dialogue around learning Encourage positive motivational beliefs & self esteem through assessment Provide opportunities to act on feedback Provide information to teachers that can be used to help shape their teaching (making learning visible) Source: Nicol and Macfarlane-Dick (2006)

Good feedback should: 1. Clarify what good performance is (goals, criteria, standards). Teacher models expert performance Students derive criteria from exemplars Model answers Students create assessment criteria for a task

Good feedback should: 2. Facilitate the development of reflection and self-assessment in learning Time on task Students identify what was strong and weak when they hand in an assignment Explain concepts underpinning problem-solving questions Provide an abstract with an essay Respond to teacher-feedback comments Select feedback comments relevant to a task from a list

Good feedback should: 3. Delver high quality feedback comments to students: that enables them to self-correct

The nature of feedback comments
Understandable: expressed in a language that students will understand Selective: only two or three worked out comments Specific: pointing to examples in the student’s submission where the feedback applies Timely: provided in time to inform the next piece of work or happening when most useful Contextualised: framed with reference to the learning outcomes or assessment criteria Non-judgemental: descriptive rather than evaluative, focused on learning goals not just performance goals Balanced: focused on positive as well as areas in need of improvement Forward looking: suggesting how students might improve subsequent assignments Transferable: focused on processes, skills and self-regulatory abilities

Good feedback should: 3. Delver high quality feedback comments to students: that enables them to self-correct ‘Feed forward’ rather than feed back Feedback on processes and skills Feedback on students’ self-assessments Don’t give feedback – point to problem areas

Good feedback should: 4. Encourage teacher-student and peer dialogue around learning Students request feedback Students respond to feedback received Audio feedback Discussions of feedback in tutorials or one to one Peer commenting and critiquing Critical reviews Electronic voting methods Collaborative assignments

Good feedback should: 5. Encourage positive motivational beliefs and self-esteem Focus on learning goals rather than performance comparisons Withhold marks until feedback responded to Avoid overuse of praise Provide descriptive rather than evaluative feedback (reader-response feedback) Align formative and summative (e.g. MCQs)

Good feedback should: 6. Provide opportunities to use feedback Provide feedback as action points Drafts and redrafts Reward use of feedback in a new task

Good feedback should: 7. Being responsive to feedback needs Requested feedback Online testing enabling ‘just-in-time’ teaching Electronic voting methods One-minute papers Monitoring discussion boards

Discussion points

Psychology 560 first year students Mixture of psychology majors (130) and those taking psychology only for one year (430) 6 topic areas, 48 lectures, 4 tutorials, 12 practicals Assessment; 2 x MCQs (25%), tutorial attendance (4%), taking part in experiment (5%), essay exam (66%)

Problems identified No practice in writing skills but required in the exam More detail provided in lectures than mentioned in exams (not enough independent reading) No feedback except on MCQs (percent correct) Didn’t want to increase staff workload Wanted to improve overall exam marks And standard of entrant to second year

Discussion point What would you do to improve the student experience in first year psychology? You can use any technology (or combination of technologies) but you must consider costs and staff time constraint.

Psychology Redesign Discussion board in WebCT Students in 85 discussion groups of 7-8, same groups throughout year Also open discussion board for class Friday lectures dropped Students discover for themselves through collaboration what would have been presented in the Friday lecture Series of online tasks

Structure of group tasks
6 cycles of 3 weeks (one cycle x major course topic) First week: ‘light’ written task (e.g. define terms) = 7 short answers (all answer) Second week = guided reading Week three: ‘heavy’ written task: students answer guided questions and then collaborate in writing a word essay. Within each week: The Monday lecture – introducing material Immediately after lecture, task posted online – for delivery the following Monday Model answers (selected from students) posted for previous week’s task

The teaching role Participation in the discussions was compulsory but not marked (this year there is 2% mark for participation) The course leader provided general feedback to the whole class – often motivational He encouraged students to give each other feedback And he selected the model answers The group discussions were not moderated Around 8 teaching assistants monitored the discussions and reported non-participation to the teacher

Guidelines Comment constructively on each other’s work. If you think that someone has missed a detail or is confused then HELP - point out how an answer might be better. This isn’t showing off – it’s being supportive – don’t leave someone stuck with half an answer or one that’s wrong or confused just because you don’t want to seem to ‘know better’ – it might be you that needs the help next time. Build answers in your online group discussion space, i.e. ‘show your working’ online where possible so that you can keep track of your progress as a group (and also so that Teaching Assistants can check on who is doing what and who isn’t). Source: Jim Baxter, Psychology, Strathclyde University

Online Project 1 – Classical Conditioning Phenomena.
Online Project 1 – Classical Conditioning Phenomena. Each Group Member should read the Passer chapter from the beginning to at least as far the section which begins ‘Applications of Classical Conditioning’. Satisfy yourself that you can answer EACH of the questions below. Then agree as a group who will answer what. Project 1 is to answer these questions as fully as you can: 1) What type of response is susceptible to Classical Conditioning? 2) Why does Extinction occur? 3) What is Spontaneous Recovery? 4) What does the phenomenon of Spontaneous Recovery tell us about the nature of Extinction in Classical Conditioning? 5) What is Generalisation? 6) What is Discrimination? 7) What is Higher Order Conditioning?

Project 9: An example of ‘heavy’ task
The Task – 800 word essay: Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible? readings suggested questions provided – all should try and advice on how to divide task given

Topic: Online Project 9 - Online Group Projects 45
Topic: Online Project 9 - Online Group Projects 45 Date: 11 February 2007 Subject: hey?? Author: John Hey, is it OK with everyone if I post up the part of my essay, describing Freud’s theories, some of his strengths and weaknesses’

Date: 11 February 2007 Subject: Re:hey?? Author: Donna yeah cool, il describe Eysneck (is that even how its spelt’’) is any1 else gettin confused’ freuds sayin that adult personality is influenced by parenting, either excessive or lax at each of the stages that can result in regression or fixation. whereas Eysneck is saying that differences in personality are a result of differences in the level of arousal’’so there not compatible coz Freuds- emphassing the importance of childhood experiences and Ey is saying that it is biological’’but then genetic differences account for like half of the difference or sumfin’’’ n each is emphasising the importance of bio/env and not saying that that is the only influence on adult personality’’’ AND- if the ego and the superego develop to control the impulses of the id, which is innate and unconscious then how can behaviour and personality be a result of childhood experiences if the ids pesent at birth’ ahh but then i spose that ego and superego have contact with reality.... c y am conffudled!! please help dont even no if this is anything 2 do with question, who knows.xx

Topic: Online Project 9 - Online Group Projects 45 Date: 11 February 2007 Subject: Re:hey?? Author: John I think you are on the right lines. In my essay, I said that Eysenck’s theories of personality can be viewed in contrast to Freud’s theories of personality. I wrote that both their theories can be correct although they emphasised different things. Freud emphasised the significance of childhood experiences (the three main psychosexual stages) and the unconscious in influencing adult personality traits whereas Eysenck emphasised more the heritability of traits and psychological factors such as Psychoticism, Extraversion and Neuroticism as main influences on the personality of an individual. I hope that make more sense. That’s kinda what I wrote in my essay, anyway!

Topic: Online Project 9 - Online Group Projects 45 Date: 11 February 2007 Subject: Re:hey?? Author: Donna yeh that makes sense! thanks. i havent wrote the full essay yet but il put up a summary of eysencks theory just now. well in like 10mins. ta xx

Benefits Students worked exceptionally hard
Written responses of exceedingly high standard Discussions about learning and leaner responsibility High levels of motivation: atmosphere in class improved Online interactions showed powerful ‘scaffolding’ Feedback possible with 560 students: peer and self-feedback (model answers) Easy for tutors to monitor participation Improved mean exam performance (up from 51-57%, p<0.01)

Has it worked?

Online postings/interaction
24,362 messages posted by groups Average number of postings per student 44.3 1067 postings to general open discussion forum Students set up online study groups for other subjects Structured tasks online triggered important social-cognitive processes

Relation to the Gibbs & Simpson’s four assessment conditions
Tasks require significant study out of class (condition 1) Tasks are distributed across topics and weeks (condition 2) They move students progressively to deeper levels of understanding (condition 3) There are explicit goals and progressive increase in challenge (condition 4)

Relation to 7 feedback principles
Standard format and model answers provide progressive clarification of expectations (principle 1) Students encouraged to self-assess against model answer (principle 2) Course leader provides motivational and meta-level feedback and selects model answers (principle 3) Online peer discussion aimed at reaching consensus is core feature of design about response (principle 4) Focus on learning not just marks, sense of control/challenge enhanced motivation (principle5) Repeated cycle of topics and tasks provide opportunities to act on feedback (principle 6) VLE captures all interactions allowing course leader to monitor progress and adapt teaching (principle 7)

What can we learn from this case study?
Use of simple technologies (discussion board) Considerable thought gone into the learning design [which is transferable] The drivers were learning improvements rather than technology (context of use) Key finding across all REAP studies was need to balance structure and learner control An important finding was the way that the social and the academic processes were shown to be mutually supportive

Discussion points

Figure 1: Framework for Analysis of Assessment and Feedback practice
EMPOWERMENT/ SELF-REGULATION ACADEMIC EXPERIENCE SOCIAL EXPERIENCE ENGAGEMENT Figure 1: Framework for Analysis of Assessment and Feedback practice

of Good Assessment and Feedback
12 Principles of Good Assessment and Feedback Practice

Good formative assessment and feedback practices should:
Help clarify what good performance is (goals, criteria, standards) Encourage ‘time an effort’ on challenging learning tasks Deliver high quality feedback information that helps learners self-correct Provide opportunities to act on feedback Ensure that summative assessment supports formative learning processes Encourage interaction and dialogue around learning (peer, teacher-student) Facilitate the development of self-assessment and reflection in learning Give choice in the topic, method, criteria, weighting or timing of assessments. Involve students in decision making about assessment policy and practice Support the development of learning groups and communities Encourage positive motivational beliefs and self-esteem Provide information to teachers that can be used to help shape the teaching

Dynamics of Implementation
Interest in FA Summative A can undermine the value of fa e.g.

Principle 1: Clarify what good performance is
EMPOWERMENT/ SELF-REGULATION Students create criteria Students add own criteria Students identify criteria from samples of work Exemplars of different performance levels provided Students rephrase criteria in own words Provide document with criteria ACADEMIC EXPERIENCE SOCIAL EXPERIENCE ENGAGEMENT Principle 1: Clarify what good performance is

Principle 2: Encourage time and effort on challenging tasks
EMPOWERMENT/ SELF-REGULATION Students given large project and expected to structure their own learning Ask students to determine the milestones and reward them for keeping to the deadlines Set out a series of learning tasks with milestones ACADEMIC EXPERIENCE SOCIAL EXPERIENCE ENGAGEMENT Principle 2: Encourage time and effort on challenging tasks

+ 2. Encourage time & effort on challenging learning tasks
EMPOWERMENT/ SELF-REGULATION ACADEMIC EXPERIENCE SOCIAL EXPERIENCE 2. Encourage time & effort on challenging learning tasks 6.Encourage interaction and dialogue around learning (peer and teacher-student) + ENGAGEMENT

+ 3. Deliver individual written feedback on students’ work
EMPOWERMENT/ SELF-REGULATION ACADEMIC EXPERIENCE SOCIAL EXPERIENCE 3. Deliver individual written feedback on students’ work 6.Encourage group discussion of how that feedback might be used in tutorials + ENGAGEMENT

EMPOWERMENT/ SELF-REGULATION Students self-assess using MCQs and provide confidence ratings ACADEMIC EXPERIENCE SOCIAL EXPERIENCE Students self-assess own performance using online MCQs ENGAGEMENT Principle 7

MCQ and Confidence-Based Marking
Ref: Tony Gardner-Medwin (2006), Confidence-based marking: towards deeper learning and better exams. Degree of certainty C=1 Low C=2 Medium C=3 High Mark if correct 1 2 3 Penalty if wrong - 2 - 6 Scoring regime for confidence-based marking

EMPOWERMENT/ SELF-REGULATION Students create MCQs and feedback for wrong and right answers 6. Encourage interaction and dialogue around learning (peer and teacher-student) Students self-assess using MCQs and provide confidence ratings ACADEMIC EXPERIENCE + SOCIAL EXPERIENCE Students self-assess own performance using online MCQs ENGAGEMENT Principle 7 Principle 6

Guidelines for Implementation
A single principle or many? Active involvement of students Tight-loose – maintain fidelity to the principles (tight) but encourage disciplines develop their own techniques of implementation (loose) Clarify students’ responsibilities Evaluate changes (process indicators) And where ICT can add value Alternate solo and group work Share your learning designs Interest in FA Summative A can undermine the value of fa e.g.

My Publications Nicol, D (2009), Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland ( Nicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge. Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), See also for copies.

From Monologue to Dialogue: enhancing feedback in learning
Interactive Keynote Session Queen’s University Belfast, Centre for Educational Development, 20th May 2009

Feedback as ‘dialogue’
Contingent: in response to learners’ needs Multi-faceted: deriving from many sources (peers) and available in different formats Catalytic: triggering inner dialogues around disciplinary concepts and ideas Motivating: encouraging feedback seeking rather than feedback avoidance behaviours. Actionable: expressed as changes in thinking and in action

Productive use of study time
Assessment tasks should: Capture enough study time (in and out of class) Are spread out evenly across timeline of study Lead to productive activity (deep vs surface) Communicate clear and high expectations Concern here is with ‘steers’ about how much work to do Source: Gibbs and Simpson (2004)

Seven feedback principles
Good feedback practice should: Clarify what good performance is (goals, criteria, standards). Facilitate the development of reflection and self-assessment in learning Deliver high quality feedback to students: that enables them to self-correct Encourage peer and student-teacher and peer dialogue around learning Encourage positive motivational beliefs & self esteem through assessment Provide opportunities to act on feedback Provide information to teachers that can be used to help shape their teaching (making learning visible) Source: Nicol and Macfarlane-Dick (2006)

Another Perspective (1)
1-4. Students don’t engage in time on task Don’t learn to manage their time Less opportunity to identify misunderstandings Have less reason to get feedback from peers P1. Unclear goals/criteria Under-performance by students Feedback doesn’t connect Students don’t’ engage in effective self-assessment P2. Absence of formal opportunties for self-assessment Students don’t learn to make evaluative judgment about own work Have less ownership and understanding of assessment criteria P3. Poor quality teacher feedback Students become dependent on teacher Feedback does not feed-forward

Another perspective (2)
P4. Lack of peer dialogue Teacher has to provide all the feedback Students don’t scaffold each other’s learning P5. Judgemental feedback Damaged motivation Can lead to fixed mindset Students give up or leave course P6. No clear opportunities action on feedback Students don’t see the benefit Make less improvement Staff time might be wasted P7. Teacher not aware of student difficulties Less able to target feedback to student needs Over or under production of feedback

Discussion in Groups Consider your own classes/modules:
Identify any bottlenecks/issues in your own teaching How might you apply the assessment/feedback principles to address these bottlenecks and enhance student learning? Identify any questions raised in your discussion groups (e.g. What difficulties do you envisage in application of the principles? What other recipes would you suggest?) Plenary report back Be prepared to share a good idea with others in the plenary and/or A ‘question worth asking’ that might help illuminate any important issues.

Implementation strategy
The big idea: ‘learner autonomy’ The problem: assessment The principles: specified but not over-specified solutions Local implementations with a common framework Multi-professional support Evaluations of implementations Dissemination internal and external Scaling up from course teams to departments Embedding Interest in FA Summative A can undermine the value of fa e.g.

Written feedback in mass higher education
Students don’t read and/or act on comments Evidence of poor quality comments (e.g. Duncan, 2007) No guidelines on what might be effective (see Hattie and Timperley, 2007) Multiple purposes and audiences for comments Negative effects of feedback (Kluger & DeNisi, 1998) Matching feedback to individual needs (Pelligrino, Chudowsky and Glaser, 2001) Students viewed as passive (Boud, 2007) No evidence that improving feedback comments on their own improves learning! (despite Black and Wiliam, 1998)

Another Perspective (1)
1-4. Students don’t engage in time on task Don’t learn to manage their time Less opportunity to identify misunderstandings Have less reason to get feedback from peers P1. Unclear goals/criteria Under-performance by students Feedback doesn’t connect Students don’t’ engage in effective self-assessment P2. Absence of formal opportunties for self-assessment Students don’t learn to make evaluative judgment about own work More likely to be dependent on teacher for feedback Have less ownership and understanding of assessment criteria P3. Wrong kind of teacher feedback Students become dependent on teacher Students don’t believe staff interested in them

Another perspective (2)
P4. Lack of peer dialogue Teacher has to provide all the feedback Students don’t scaffold each other’s learning P5. Negative feedback Damaged motivation Can lead to fixed mindset Students give up or leave course P6. No clear opportunities action on feedback Students don’t see the benefit Staff time wasted Make less improvement P7. Teacher not aware of student difficulties Less able to target feedback to student needs Over or under production of feedback

From Monologue to Dialogue: enhancing feedback in learning

Similar presentations

Presentation on theme: "From Monologue to Dialogue: enhancing feedback in learning"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

From Monologue to Dialogue: enhancing feedback in learning

Similar presentations

Presentation on theme: "From Monologue to Dialogue: enhancing feedback in learning"— Presentation transcript:

Similar presentations

About project

Feedback