2. Results of the Students’ Unions survey on the Student Evaluation of Teaching (SET) and Student Evaluation of Modules (SEM) Undergraduate and PGT results
Dasha Karzunina Education Officer

3. Background
The following presentation will deal with the results of the Students’ Union’s research into proposed changes to the SET and SEM programmes in place across the University of Nottingham; it is particularly concerned with changes proposed through the Evaluate Programme. The following results deal only with Undergraduates and PGT students – PGR students are dealt with separately (as they answered different questions). Data-collection was carried out through a Web-based survey, promoted through Union communication channels – data-collection ended on the 19th February 2014.

4. Respondents answered a survey with a series of questions designed to measure:
The experience that they have of the current SET and SEM process (e.g. complexity)
How valuable they find the SET and SEM process
Responses to proposals that are coming from the Evaluate Programme
Decision-making factors when choosing modules
New data-sets that could be used to aid the decision making process

5. Turn-out and Demographics
Full survey turn-out of 1019 responses, with optional questions on module choice receiving a smaller turn-out (n=306)
The following demographic data were collected about respondents:
Year and level of study
Fee status/domicile
Lead School and department (this has been re-coded to Faculty to aid analysis).
Most are included for reference only, although Faculty has been used as an analysis tool as well as a reference point.

6. Level/Year of Study
Please note that these are included for information only and have not been used in cross-tabulation.

7. Fee status/Domicile
Please note that these are included for information only, and have not been used for cross-tabulation

8. Faculty
There is a low turn-out in the Faculties of Science and Engineering: all Faculty analyses for these should be handled with some care.

9. Section 1 - Current experiences of SET and SEM
Turn-out n=1019
Respondents were asked to give their responses to a series of seven statements designed to explore:
Their current experience of SET and SEM (such as the complexity)
How valuable they find SET and SEM
How they perceive that the department values their feedback and responds to it.
Responses were on 10-point scale from 1 (Least agree) to 10 (most agree): for the purposes of analysis, 5-6 are classed as apathetic responses.

10. Level of agreement: SET and SEM statements

11. Faculty Means – SET and SEM statements

12. Results…
Most respondents agree that they find the completion of SET and SEM to be valuable to their academic experience, and value the chance to be able to evaluate their teaching
Few find the process complicated, although a majority would like to see the process given an online alternative.
Over 25% of respondents feel pressurised to give positive scores, although this is not the normal, and the majority do not feel this way.
Most respondents feel that their departments value their feedback and act upon it – many, however, do not feel this way.

13. Results continued Feelings of pressure
Open text comments also reflect some of the results shown above
Feelings of pressure
“I find that evaluating by filling forms in class is quite stressful and daunting as the teacher knows who said what… For me that sort of defeats the purpose of it being ‘anonymous’” – 2nd Year, Arts
“Can be awkward as the class sizes are small and the module lecturer boasts that she knows everyone’s handwriting. [It] makes [giving] anything but positive feedback very difficult.” – Foundation, Medicine and Health Sciences

14. Lack of resolution of feedback
“It really feels that there is little change to the course regardless of feedback” - 4th Year, Arts
“All [feedback] seems to be ignored” - 2nd Year, Arts
“Comments never make a difference as the medical school always seems to justify our criticism to suit them” -4th Year, Medicine and Health Sciences

15. Perceived inefficiency of the current system
“We submit different feedback in many different forms about different aspects of the teaching, school, and so on. It’d be nice if there was one consistent way, instead of a thousand surveys, forms and so on.” - 2nd Year, Science
“It’d be better if it was online – it seems time-consuming and a waste of paper to do teaching reviews on paper.” -2nd Year, Medicine and Human Sciences
“It might be better to do it online, or in some way outside of the classroom, in case a student has to miss a class for any reason and so miss the chance to give an evaluation” -4th Year, Social Sciences

16. Section 2 – Module Choice
Lower turn-out n=306 These questions were optional, and those who indicated that they had no module choice were opted out. 45% of those eligible answered the follow-ups
Respondents were asked to give their responses to a series of questions regarding:
Their decision making process when selecting modules
Current use of information when selecting modules.
Their level of choice
Responses were on 10-point scale from 1 (Least agree) to 10 (most agree): for the purposes of analysis, 5-6 are classed as apathetic responses.

17. Level of choice
Those who had no choice were not directed to follow-up questions on decision making. Follow-up questions were optional, leading to a lower turn-out

18. Current use of information

19. Current use of information (contd).
Open-texts also suggest that other data are used “The method of assessment is also a deciding factor” – 3rd Year, Social Sciences “The teaching period (either during semester one or two) and module credits also affects my decision to choose the module in order to avoid too much work per semester.” - PGT, Social Sciences

20. Module Choice: Agreement with statements

21. Module Choice: Faculty Means

22. Open-text comments
“[Module selection is] heavily dependent on word of mouth from previous years’ students” – 3rd Year, Medicine and Health Sciences
“[We] don’t really get any information on modules except for the course description (or not made clear where you find additional information)” – 1st year, Medicine and Health Sciences.

23. Results
Most respondents use all of the information available to them when choosing their modules (over 70%), although over 75% also suggest that they would like more information.
A large minority (46%) indicate that they’ve regretted a decision when they’ve made a module choice, although there is a large spread in the level of agreement with this statement – some feel regret strongly, and many no sense of regret at all.
Most indicate that they choose their modules mainly because of academic interests, and whilst several indicate that they may use new data in a way that will potentially influence or change their choice, most won’t.
Most respondents (72.5%) would value the ability to see the scores of the previous cohorts’ evaluation of teaching and modules when making module choices.

24. Conclusions and Recommendations
Students are broadly satisfied with the current arrangements for SET and SEM, however, there is a large demand for a more convenient process that moves away from the current paper-based norm. This for a couple of reasons:
Respondents indicate that they have worries about the anonymity within the current process, and we feel that a move to a position with less teacher intervention would be advisable.
Respondents also feel that there may be a slight amount of inefficiency in the current system, and that this can be simplified with an online tool.
Some students feel that there is a lack of transparency in the current SET and SEM arrangements, with many feeling that their feedback isn’t acted upon – this isn’t wholly the case, but is felt by a large enough amount of students that it can be seen to be a worry.

25. Conclusions 2
More transparent publication of SET and SEM scores could alleviate the sense that there isn’t a great amount of transparency in the system, and the School will be more equipped to close the feedback loop for students by open publication.
As can be seen a large amount of UG and PGT students have a desire to see more information when picking their modules. SET and SEM scores aren’t the only data that can contribute to this, with such things as course reading lists also playing a part.
Whilst the primary factor in any student choosing a module will always be academic interest and curiosity, or career goals, many would value the ability to make a more informed choice: this won’t necessarily change the choice that the student makes, but would lead to more of a sense that the choice made was a correct one.

26. Questions?
These data were prepared by Samuel Nichols, Education Network Researcher – the full-text and recommendations will be circulated following this meeting.
For questions on methodology, and for more information, then please contact Sam on: