1. Keeping the students satisfied: a longitudinal, comparative institutional analysis of survey free- text comments Elena Zaitseva, Clare Milsom, Martyn Stewart Academic Enhancement Unit Liverpool John Moores University

2. HEIs are concerned with meeting expectations of current and prospective students Potential of the NSS for enhancement of the student learning experience has been considered by the HEA and HEFCE LJMU introduced ‘Mirror’ surveys for Years 1 and 2 Enhancement is only possible when you know what the issues are Student voice and satisfaction

3. NSS/Mirror surveys open text comments An extensive source of authentic student voice Analysis is often confined within departments/schools and not much cross- institutional sharing is taking place (various reasons for it) Institutional level analysis is time and resource consuming, but is there a value to it?

4. An ‘alternative’ approach to analysing student comments Using semantic analysis software Leximancer Concept map shows a ‘landscape’ of student satisfaction/dissatisfaction Sentiment analysis helps to establish an ‘institutional attitude’ to certain aspects of student experience Longitudinal study undertaken – looking at the NSS and Mirror surveys results over two years (to identify changes and impact of interventions)

5. Year 1 comparative analysis 20112012

6. Year 1 comparative analysis Feedback (fav) changed its location on the map: in 2012 it is closely positioned to helpful and tutor Assignments: 2011 (unfav), positioned closely to lecture and communication(both unfav) ; in 2012 – favor concept, close proximity to tutor(s) and helpful Learning is a new concept for 2012, as well as practical – both favourable and closely positioned on the concept map Coursework changed sentiment to unfavourable in 2012 (group based coursework, coursework on joint degrees)

7. Year 2 comparative analysis 20112012

8. Year 2 comparative analysis Feedback is perceived more positively by year 2 students in 2012 Course becomes unfavourable concept for 2012 cohort (result of a major curriculum change – move to 24 credit modules) Modules, being a favourable concept in 2011, change sentiment in 2012 (perceived lack of flexibility, limited optional module choice) Practical – a new and favourably rated concept for 2 nd years in 2012

9. NSS concept map 2011

10. NSS concept map 2012

11. NSS comparative analysis Students are more concerned with employability and relevance of their degree to the world of work (course is connected to skills and knowledge in 2012) Library changed position and became a favorable concept in 2012 (result of institutional actions taken) Feedback still unfav but likelyhood is less then in 2012 (13 and 04 accordingly) Degree changed sentiment to unfavorable in 2012 (students are more consented with relevance and practical side of the degree)

12. Conclusions Student satisfaction is a dynamic process Analysing comments longitudinally can help to identify changes in students’ attitude and see the impact of interventions at institutional level Quick data turn around allows timely academic development activities Presenting open text data as concept maps is a more engaging way of discussing some of the issues with academics

13. 'the practice of science happens at the border between the known and the unknown…..there is no certainty; there is simply the best description we have of the Universe....'. (Cox 2011)