1. TMA feedback: can we do better? Mirabelle Walker Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT)

2. FAST project findings (1) Quantity and timing of feedback –Sufficient feedback is provided, often enough and in enough detail –The feedback is provided quickly enough to be useful to students Quality of feedback –Feedback focuses on students’ performance and learning, and on actions under their control –Feedback is appropriate to the purpose of the assignment and to its criteria for success –Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing Student response to feedback –Feedback is received and attended to –Feedback is acted upon by the student Gibbs & Simpson (2004–5)

3. FAST project findings (2) What students said about their TMA feedback Received plenty of it Motivated by praise and encouragement Mainly received within three weeks Read feedback but rarely acted on it

4. FAST project findings (3)

5. FAST project findings (4)

6. Idea of ‘depth’ of comment Depth 1 – acknowledges e.g. ‘more needed here’; ‘good’ Depth 2 – corrects / amplifies e.g. ‘you needed to mention xxxx’; ‘a good introduction’ Depth 3 – explains e.g. ‘you needed to mention xxxx because …’; ‘a good introduction because …’

7. FAST project findings (5)

8. FAST project findings (6) Significant weaknesses in current practice: Too much emphasis on justifying the grade Lack of shared understanding of assessment criteria (students & CT) ALs good at articulating students’ weaknesses; explaining strengths problematic Lack of holistic assessment of students’ work

9. My project Replicate some of FAST investigations in Technology – results similar? –Courses chosen: T173, T209 & T224 –Analysis of feedback on sample TMAs –Telephone interviews with students Follow-up action with ALs, monitors, CTs, STs as appropriate – and review

10. Progress so far Feedback analysed on all three courses Some results circulated to T209 & T224 ALs Some comparisons with Science made Telephone interviews conducted on T209 & T224 – T173 next month Some analysis of telephone feedback done – awaiting T173 to finalise

11. Category percentages compared – Technology & Science

12. Category percentages compared – Technology courses & Science

13. Category percentages compared – Technology (not T209) & Science

14. Type of content comment compared – Technology & Science

15. Type of content comment compared – Technology courses & Science

16. Type of content comment compared within T173

17. Depth percentages compared – Technology & Science

18. Depth percentages compared – Technology courses & Science

19. Further comparison of depths

20. Number of comments per student MeanMedianLowestHighest T1732926456 T20936359 (twice)81 T22417158 (twice)48 Science39

21. Action taken on T209 and T224 Document ‘Using TMA comments to good effect’ prepared for each course –Explains ideas of ‘depth’ and ‘feed forward’ –Contains course-specific examples of depth 2 & depth 3 comments Sent out in first tutor mailing (2006) with commendation from Course Chair Monitors briefed accordingly

22. Telephone surveys T209 and T224 complete – carried out immediately after end of course Examined –student’s perception of usefulness/helpfulness of feedback –whether (& how) student had used the feedback in any future TMA / the ECA / the exam –student’s preferences regarding placing of comments: on PT3, script, (pro-forma)

23. Preliminary findings (1) Students are eager to receive their marked TMAs and do read the feedback … … but they do not necessarily use the feedback again in the rest of the course (approx 20% said they never used it) T209 students were likely to make more use of the feedback later in the course than T224 students T209 students particularly mentioned using skills development feedback

24. Preliminary findings (2) Not all students found that tackling the TMA, and the feedback they subsequently received, encouraged them to study the rest of the course A small number of students said they were disappointed with the quality of the feedback (but some were surprisingly accepting)

25. Preliminary findings (3) Overwhelming majority of students value comments on the script the most: –‘where lost marks made clear’ –‘tells me exactly where the mistake is’ –‘very specific’ –‘more evidenced against actual text’ –‘easier seeing my work with comments relating to it’

26. Aside about feedback and eTMAs We can’t assume that students read the PT3 first – or even at all We can’t assume that students find and read a separate marking document sent back with the marked TMA Some (most?) students find juggling documents on the screen awkward/difficult It’s easy for tutors to place comments exactly where they apply Turn-around times longer: symptom of a problem?

27. Action taken on T209 and T224 T209 and T224: Students reminded to look for and read the PT3 T209: Separate pro-forma dropped for eTMAs; tutors asked to copy and paste grids at end of questions

28. FAST project findings in Technology? Significant weaknesses in current practice: Too much emphasis on justifying the grade  Lack of shared understanding of assessment criteria (students & CT) varied ALs good at articulating students’ weaknesses; explaining strengths problematic  Lack of holistic assessment of students’ work 

29. Good feedback in Technology? Quantity and timing of feedback –Sufficient feedback is provided, often enough and in enough detail In general –The feedback is provided quickly enough to be useful to students In general Quality of feedback –Feedback focuses on students’ performance and learning Too much biased towards performance on this TMA –Feedback is appropriate to the purpose of the assignment and to its criteria Purpose and criteria often implicit –Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing Sometimes, but more explanations would be helpful Student response to feedback –Feedback is received by students, and attended to Yes –Feedback is acted upon by the student Not sufficiently

30. Implications for Course Teams Different sorts of questions and criteria elicit different types of feedback (or) To elicit particular feedback, write the question and criteria accordingly – maybe even write the course material accordingly

31. More implications for CTs Be explicit with ourselves and with ALs and students about what an assignment’s purpose and criteria are Don’t assume that ALs will instinctively know what sort of feedback we’re hoping for – be explicit in the marking guide (or elsewhere)

32. Implications for Associate Lecturers Need for shift towards emphasis on supporting student’s learning and progress through course, rather than just explaining what was wrong in this particular TMA That implies more student-centred feedback – and more holistic feedback on PT3s It may also imply giving as much (more?) emphasis to feedback as to marks

33. Implications for monitors Need to shift emphasis from ‘Was the mark OK?’ towards ‘Was the feedback OK?’ Need to encourage appropriate forms of feedback (and discourage non-appropriate ones?)

34. References FAST presentations given at Open University, 10 February 2005 Gibbs, G & Simpson, C (2004–5) ‘Conditions under which assessment supports students’ learning’ Learning and teaching in higher education 1(1) pp 3–31; available via http://www.open.ac.uk/science/fdtl/pub.htm

35. Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT) The Open University Walton Hall Milton Keynes MK7 6AA http://cetl.open.ac.uk/colmsct