Matching student expectations and staff practices in producing and engaging with high quality feedback Jason Bohan, Maxine Swingler, Heather Woods School of Psychology, University of Glasgow

Introduction Feedback promotes student engagement with deeper learning and positively impacts on future learning (Biggs, 1999) Effective feedback aids transition from supported school/college environments to independent learning in HE (Poulous & Mahoney, 2008) UK NSS scores reveal low satisfaction for feedback Staff perception is that they provide good quality feedback, but student’s fail to utilise this (Careless, 2006) Q. Given the importance of feedback, why is there a difference between student and staff experiences of the feedback process and can we align the process more tightly?

Method Data Collection – 3 Phases Student Survey Student Survey Focus groups Staff Survey 59 (12%) Level 1 psychology students responded to a survey posted on our VLE (Moodle). Low response but representative of the class: Average age = 21 years 78% Female Honours students 79% New to psychology - 59% Native language – 63% Compared to Bohan & Stack survey – numbers similar. (proportionally higher honours and age here) 26% of the class: Average age = 19 76% females Honours 62% New to psychology 61% Native language 63%

Method Data Collection – 3 Phases Student Survey – 6 themes Focus groups Staff Survey What is feedback? What is good feedback? Expectations of assessments and experience of feedback How do you use feedback? How do you want to receive feedback? How does the feedback you receive differ from before university?

Method Data Collection – 3 Phases Focus Groups Student Survey Staff Survey Honours student volunteers ran one focus group 4 students participated

Method Data Collection – 3 Phases Staff Survey Student Survey Focus groups Staff Survey 10 staff members all involved in assessing Level 1 feedback were asked: What they believed was important in coursework What type of comments they provided on coursework What type of additional feedback students requested How they liked to give feedback

1. What is feedback? 10 sources of feedback to be rated: Written comments Talking to teachers Exam feedback Feedback sheets – written comments Feedback sheets – generic comments Email from tutors Lab quiz (automated) VLE (Moodle) comments from staff VLE (Moodle) comments from student Peer discussion

1. What is feedback? Written comments on coursework and feedback sheets rated top- 95% and 93% agreed Peer communication rated lowest- 19% rated Moodle comments from other students as feedback 69% of students agreed that staff comments on Moodle were feedback

1. What is feedback? Compared what students considered to be feedback with what they found helpful 3 significant differences (Mann-Whitney) between…. Student comments on Moodle (p<0.001) 19% agreed as feedback, 47% rated as helpful Comments written on coursework (p<0.001) 95% agreed as feedback, 76% rated as helpful Automated feedback from lab quizzes (p<0.05) 74% agreed as feedback, 61% rated as helpful

1. What is feedback? So, differences between what students see as feedback and what they find helpful. Peer communication is useful but not viewed as feedback – possibly because less reliable information source? Written comments seen as feedback, but many don’t find helpful Automated feedback (shown correct answer and given score) 39% do not find helpful 24% don’t rate written comments as helpful

What is Feedback? Digging Deeper… Student led focus group discussion (Level 1 psychology students). What is Feedback? Written comments yeah, it’s I think what we draw most information from in terms of where to improve and where we’re doing quite well.” Staff-student Forums “..good idea but people didn’t really think about things before they posted their questions! Student discussion on Forums “how would they have more of an idea than I do?” Participants tended to see written comments about coursework as the most important type of feedback. Comments were reported as the most detailed source of feedback they received on strengths and areas for improvement. Reflect the survey findings of written comments being the top rated source of feedback.. Students had opportunity to ask questions on forums about coursework in general and in the run up to the hand in of their lab reports (forums were particularly busy then). Positive comments about the staff responding quickly and found answers helpful, but the number of posts was overwhelming. Moodle system means everytime a Q is asked or responded to –students get an email-often similar questions were posted repeatedly which led to confusion –as 1 P said “ I wish they would just read!” Student posts on forums were seen as helpful in the survey, but not classed as feedback- and this was reflected in the FG. Students liked discussion with peers, but were not confident in using this information as the basis for their coursework.

2. What is good feedback? Improvements for next piece of coursework Comments on structure Comments on argument Comments on the content Comments on critical analysis Comments that provide encouragement Clearly relates to grade Acknowledges effort Further resources Markers overall impression Points out errors, e.g. spelling

2. What is good feedback? Everything! Direct comments on coursework and suggestions on improvement rated as most important- 98% Points on errors e.g. spelling 14% rated as unimportant but 70% still rated as important Comments on effort and providing encouragement also important How important is it to you for the marker to provide feedback on the following points? 10 points listed and students to respond as unimportant, unsure or important. Re suggestions for improvement- interesting when tied in with how use feedback- discussed later- 66% highlighted that that helped later when understaood and 37% didn’t help later Nothing lower than 69% Interesting that students are unsure whether feedback on critical analysis, further resources, markers overall impression- focus on what I did and tell me how to do it better?

What is Good Feedback? Timely Break down of assesment grades “Psychology was in the end the subject where I got most of the feedback from.” It’s very transparent […] that’s definitely something that I really appreciate!” Timely Break down of assesment grades “I think feedback should not be only negative points but also showing you ‘this was well done, if you keep that going, it’s a good thing for the future.’” Positive comments “explained in a way that you can read it and go ‘OK, I know what I’m supposed to be doing next time to avoid this happening again’” How to improve

3. Expectations of assessments and experience of feedback Research skills Knowledge/comprehension Structure Critical analysis Writing style What should be included in a good answer and what they have received feedback on. Compared with matched question in staff survey Research skills, knowledge/comprehension, structure, critical analysis, writing style

3. Expectations of assessments and experience of feedback No respondents, staff or student, disagreed that these factors should be in a essay/report Different when asked if the students have received feedback on these factors or staff had provided feedback….

3. Expectations of assessments and experience of feedback Knowledge and comprehension 100% of staff provided this feedback 32% of students report not getting it Writing style 80% of staff commented on this 46% of students did not receive it 100% of staff agreed that they had provided feedback on knowledge and comprehension 68% of students agreed that they had received feedback 80% of staff agree that they had provided feedback on writing style vs 54% of students agreeing they had received it

4. How do you use feedback? Feedback is useful Clarification for understanding Grade focus Pre-submission support and advice Reflection when new assessment issued Reaction to poor grade Marker meeting for clarification Reaction to negative comments Grade and feedback connection Remarking Marking criteria Understanding comments Detailed comments

4. How do you use feedback? 97% of students rated the feedback received as useful 36% only look at the grade 54% look back at comments with new coursework (15% unsure) 49% are unsure how their feedback and grade are related 53% disagreed that feedback helped in understanding 22% agreed that they did not understand the comments on their work (17% unsure)

Barriers to understanding How do you use Feedback? Consistency My friend will be marked up for something and it’s something I’ll get marked down for and it’s just really inconsistent.” Clarity “Sometimes they only write so short bullet points” One to one “She was able to see the person who marked it and get..direct feedback from that.” Barriers to understanding Previous exam feedback “..they published the feedback from last year’s exam….and I did look at that and it helped me I think..” Criteria in advance “‘see, this is what first year was, but those are the points we’re looking for now in second year’.” Feedforward FG discussion provides some clues as to what they don’t understand and why..perceived inconsistency in marking-actually the grading process and criteria are the same-but perhaps the markers comments are different? Students see a comment as a direct reflection of the grade-whereas a C grade or a B grade could both receive a comment on how to improve on structure-important for reflecting on our own assessment practice. Sometimes difficult to read handwriting or understand short comments. Many students unaware they can meet the marker if they want further clarification of comments. Possibly reading comments outwith the context of the feedback sheet (which provides further explanation). Students found the general feedback from the previous exam to be very useful (points on what was good, what to improve) and wanted something similar for their coursework-and the criteria to be clear in advance of handing in the coursework-this is available as a report/essay writing guide on moodle-but students seem to be unaware of it.

5. How do you want to receive feedback? 83% students want comments on coursework but 55% have difficulty using and understanding them 73% of students said they would be happy with online feedback only Staff are split, 50% staff said they would prefer online only and 50% combination of written comments on scripts and structured feedback Staff rated an online feedback sheet as 1st choice for provision of feedback

How do you want to receive feedback? Structured “….you could just take the feedback from from first semester and see ‘oh those are the points they’re looking for. Those are the points I need to work on.’” Direct comments on coursework “Particular comments towards a particular paragraph or sentence or whatever but then also the kind of an overview.” Highlights another contradiction-students want a structured breakdown of their feedback-what was good, what could be improved etc which is useful when looking back at previous feedback. This suggests a structured feedback sheet, yet they also want direct comments on the text of the course work itself (not on a feedback sheet!) How do we reconcile these 2 demands? –second quote sums this up nicely.

6. How does the feedback you receive differ from before university? When I understood the feedback it helped with later work I paid close attention to the feedback I received I received feedback on drafts Feedback was provided fast enough to be useful I used feedback from my friends to improve my work The feedback for one assignment didn’t help later ones The feedback I received didn’t help me much

6. How does the feedback you receive differ from before university? Significant differences found between pre-university and university experiences of feedback 90% agreed that understanding feedback helped with later work at school- 66% at uni 68% agreed that they received drafts at school compared to 15% at uni 83% agreed that feedback was provided fast enough to be useful compared to 44% at uni More uncertainty around paying close attention to feedback at uni compared to school Level of agreement decreases in uni with feedback helping later work and feedback being returned fast enough More uncertainty regarding paying close attention to feedback- school more explicit about how they use feedback? Receiving feedback on drafts- median increases by 1 point for uni- PAL?

Differences between school and university Class size “Obviously the whole situation at university, there’s more people, it’s more anonymous in a way…that’s something that maybe stands in the way of individual feedback….” Formative feedback …opportunity to write an essay that is not graded but where you can get feedback…Like on a voluntary basis.” Students expectations of feedback at university seem different, given the large class sizes, and the feeling of anonymity. They would like more opportunities for formative feedback on drafts of essays/reports.

Conclusions & Moving Forward Broadly large agreement between staff and students on what constitutes feedback, differences in experiences Many students say they don’t receive comments on content (knowledge and comp) Style of writing Match structured staff feedback to student expectations more explicitly Marking criteria should clearly define the differing assessment criteria (knowledge, critical, writing style etc) that markers use

Conclusions & Moving Forward Students view written comments as important feedback but many do not find useful, e.g. don’t understand the comments and/or how they relate to the overall grade Many students don’t view peer-interaction as feedback but report finding it useful Help students use feedback more when attempting future assignments, e.g. reflective exercises requiring students to return to prior feedback before attempting current assignments, peer discussions…

Conclusions & Moving Forward University feedback (compared to school/college) is viewed as: Less helpful for future work Slower (mirrors NSS results) More anonymous (i.e. less connection with staff) Ways of delivering feedback Students like written comments but are agreeable to online comments Staff are split between preferences for written and online delivery Online feedback could improve the speed of delivery. But, can we provide high-quality feedback without written comments on coursework? Technology like teleform would allow us to produce structure feedback sheet sent directly to student without comments on scripts. Would substantially reduce admin time, but the danger is that by reducing the time between submission and feedback we degrade the quality of the feedback we provide.

Thank you for listening!