Feedback and Assessment: Could Do Better Kate Little Quality & Student Engagement Consultant National Union of Students

Session objectives Understand the national picture around feedback and assessment Understand the sources of evidence available to back up your arguments Benchmark your course/institution against NUS’s ten principles of good practice Begin planning a campaign to improve feedback and assessment

A national issue: NSS 09-13 Assessment and feedback is the lowest scoring area for satisfaction amongst students nationally. As you can see, scores in all these areas have increased in the last five years, but the red line of feedback and assessment remains the lowest and is still an issue for students across the UK. This data is from a large survey of final year undergraduates called the National Student Survey, which I will talk about in more detail later.

Satisfaction by subject Assessment and feedback 5. The criteria used in marking have been clear in advance. 6. Assessment arrangements and marking have been fair. 7. Feedback on my work has been prompt. 8. I have received detailed comments on my work. 9. Feedback on my work has helped me clarify things I did not understand. Agriculture and related subjects 78 79 62 74 66 Architecture, Building and Planning 72 71 64 68 Biological Sciences 76 77 65 63 Business and Administrative studies 67 Combined 84 86 83 88 Computer Science Creative Arts and Design 75 70 Education 82 Engineering and Technology 61 Geographical Studies 73 Historical and Philosophical studies Initial Teacher Training Languages 80 Law Mass Communications and Documentation Mathematical Sciences Medicine and Dentistry 69 57 48 50 Physical Sciences 81 Social studies Subjects allied to Medicine Veterinary Science 53 42 UK average There are clear differences in satisfaction by subject for each of the five questions around assessment and feedback. The first two questions, focusing on assessment, score much higher than the following three on feedback. You can see some variation, however: for example, veterinary science scores poorly in every area except the fairness of assessment arrangements, combined subjects score consistently high across the board, and agriculture shows more of a mixed picture. It’s worth going into a bit of depth to see what the specific low areas are for your course.

What is the NSS? The National Student Survey is a survey of academic experience aimed at final year undergraduates at HE and FE institutions across the UK. Over 400,000 students this year were asked to complete the 5 minute survey. So we’ve looked at some of the national data around feedback and assessment: it’s worth stepping back a little and seeing where that data has come from.

What is the NSS? Teaching Assessment and feedback The survey is composed of 23 tick-box questions focusing on these areas: Teaching Assessment and feedback Organisation and management Academic support Learning resources Personal development The students’ union Overall satisfaction These are the areas that the survey asks questions about. There are also optional additional banks of questions that institutions can choose to add on, and institutions or students’ unions can also create up to two customised questions for their institutions. There is also a box for free-text comments where students can say anything they want about their experience.

What is the NSS? Run by an independent organisation, Ipsos-MORI Three stages: Online Telephone Postal Students can opt-out at any point 50% response rate is required! HEFCE commissions an independent organisation, Ipsos-MORI, to run the survey. It is run between January and April and and institutions can choose their start date within a 6 week window, based on whatever works for that institution. There are three stages to the survey. First there is an online stage. Students can either go proactively to the website, or they will receive an email inviting them to participate. Then if they haven’t filled it in, Ipsos-MORI will contact the student by phone and give them the option to do the survey over the phone – they will contact them up to 3 times. And finally they’ll send a questionnaire in the post. At any point during this process students can opt out, of the survey; if they do this then they obviously won’t be receiving phone calls or receiving the postal questionnaire. One of the really important things to know is that in order for results to be published, there needs to be a 50% response rate. This applies nationally, it applies to every institution, and it also applies to different courses within institutions. There also need to be at least 23 individual responses as well. This is why promoting the NSS is such a big deal, because if the threshold isn’t reached, you won’t be able to find out the results.

Why is the NSS important? Unparalleled amount of robust data on student opinion Puts student voice at the centre of dialogue around academic quality Useful to identify areas for improvement Institutions take the results seriously The NSS is really important because it means that students’ opinions about the quality of their academic experience are central to the discussion about academic quality. It can help identify areas within a university that need to be improved, and institutions tend to take the results seriously because there is such a high response rate and the survey is run by a professional market research agency. So it really gives a huge amount of information on student opinion, more than you can get really in any other way.

Being evidence-led Evidence (definition) The available body of facts or information indicating whether a belief or proposition is true or valid For students’ unions and course reps, being evidence led is increasingly important to ensure the institution takes your arguments seriously Ensuring that your campaigns and communications with your institution are based on solid evidence will help you do a better job of representing your students

What evidence is available? Course reps GOAT NSS Quantitative PRES/PTES Qualitative Module evaluation Survey free text comments (NSS) University internal surveys ISB NUS Briefings QAA/HEA work Government agencies Part time officers Charity Briefings Policy Union committee minutes University committee minutes

Benchmarking tool Principles Levels of Practice An example of a national resource you can use to support feedback and assessment campaigns is NUS’s benchmarking tool. We produced this in November based on ten principles of good practice we took from NSS data and conversations with students, and mapped each principle against five levels of practice ranging from first steps to outstanding. This allows you to build a more nuanced picture of feedback and assessment at your institution, and some areas you might like to focus improvement efforts on. Principles Levels of Practice

Outstanding practice: Partnership Each of the “outstanding” practices involve staff and students working in partnership. This partnership needs to be meaningful in order to work, which means that both groups must listen and be willing to compromise. Some of the principles may be mutually incompatible in some institutions: for example, it may not be possible to achieve “outstanding” in both feedback timeliness and feedback quality if the institution cannot afford more staff time for marking. Have honest conversations with institutional staff about what is and isn’t possible.

Benchmark Your Course Which box best describes your course’s approach to each principle? Where are the lowest areas? Where is the best practice? Does this tally with what you already know? 10 minute activity: give them each a benchmarking tool and ask them to benchmark their course. Encourage conversations on tables about the questions on the chart.

Planning Your Campaign Things to bear in mind: You may not be able to achieve “outstanding” in everything at once. What are your priorities? You will only achieve your goals by working in partnership with staff. How will you approach them? Who do you need to talk to? You (or the students on your course) may not agree with all the principles. That’s OK – make them work for you. What is the impact on postgraduate students who teach? We’re going to look at a quick campaign plan for you to take back with you, but before we do that there are some things to bear in mind when using a national resource as evidence for a campaign. You might not be able to achieve outstanding in everything at once – be realistic. What are the biggest areas of concern? Can you offer further evidence, such as NSS, to support your case? You’ll only achieve your goals by working in partnership with staff, so it’s worth thinking about what your approach will be. Collect your evidence, build your case and consider who actually makes the decisions that you’re trying to affect. It’s worth noting that because this is a national resource it doesn’t have the local context that you can provide: this means that you may not agree with all the principles or level of practice. That’s fine – there will be differences according to subject, institution and demographics – it’s about getting the most that you can out of the resource you have. If there are only three principles that apply, take those and use them as your evidence alongside local context provided by NSS and the views of your coursemates. A final consideration is the impact your campaign may have on postgrads who teach. A lot of the work you submit is likely to be marked by postgrads, who already struggle with being paid for the work they do. Improving turnaround times or requiring a certain amount of detail increases their workload: you may wish to talk to some postgrad reps about your campaign, and maybe work together to submit joint proposals that help both groups of students.

Planning Your Campaign Write down your aim – and make it SMART (specific, measurable, achievable, realistic, time-defined) Who do you need to talk to? (other students, module leader, head of department) What evidence do you have? What are you going to do?

Questions? @katelittle nss@nus.org.uk