Dr Elena Zaitseva Teaching and Learning Academy Postgraduate, Taught, and ‘On Their Own’: identifying needs of PGT cohort via survey data triangulation Dr Elena Zaitseva Teaching and Learning Academy HEA Surveys Conference, 8 May 2019, Bristol

Introduction Context: Drop in PGT student satisfaction in 2018 Skills Development, Career Orientation, Feedback and Overall Satisfaction were affected Aims: Identifying changes in PGT cohort profile Understanding challenges they face (expectations, motivations and needs) Informing/prioritising institutional enhancement initiatives

Methodology Data triangulation and analysis of three institutional data sets: demographic profile of PTES respondents (2014-2018) PTES 2018 free text comments module evaluation comments from the same cohort of students Demographic profile: empirical exploration (descriptive stats) Demographic profile of the survey respondents is very close to that of the HESA population (Soilemetzidis et al. 2014) Fairly representative response rate (33%+) PTES comments - course level student voice Module evaluation data - granularity and context

Methodology Comments: semi-automated semantic analysis, followed by thematic analysis to find consistent themes within the underperforming areas PTES 2018 - 39 300 words, 73 pages of A4, single-spaced) Level 7 module evaluations - 111 205 words, 283 pages of A4, single- spaced) Semantic analysis tool Leximancer (https://www.heacademy.ac.uk/knowledge-hub/their-own-words) Findings from PTES 2017 comments analysis were revisited for comparative purposes.

Changes in demographics More mature students (46 y.o. and older) - 8% of responses in 2014- 16 and 14% in 2018 Percentage of students entering PGT study with another or previous postgraduate degree has been increasing.

Changes in demographics

Demographics – other observations Overall Satisfaction declined across all age groups in 2018, while satisfaction with Skills Development declined mainly in students younger than 35 years of age Students with disabilities rated Skills Development particularly low in 2018 Slightly more students were considering leaving the course in 2018 than in 2017. The highest figures were among 25 years old or younger and 56 years old and older.

Comments: concept maps 2017 2018 Ff

Skills: main themes PTES: favourable concept (07%) in 2017 equal probability of being mentioned in favourable or unfavourable context (04%) in 2018 Positive comments: gaining useful practical experience and developing skills in labs and other practical settings; ability to critically assess own skills and to achieve a higher level of competence. Critical comments: skills support being too generic; need for enhanced study skills sessions for distance learning students; better timetabling of the core skills sessions.

Skills: main themes Module evaluation comments provided more contextual information: More opportunities to practice skills development, earlier access to specific resources, especially for students coming from different subject areas : … it would be good to have a section on report writing skills (writing style, layout, conventions for numbering and referencing equations, how to present graphs properly, etc). My undergraduate degree didn't cover these things and they would be really useful things to learn about. Repetition of the content from undergrad degree and the need to gain/practice more advance content/skills: Try to differ from the research skills programme in undergrad sport science. Some of the things we were learning about were identical to what I have already done in second year. …it is patronising as a post grad to be told how to structure an essay. There are extra curricular study sessions for those who do not know how to do this.

Career: main themes Favourable concept in 2017 (04%) Unfavourable concept (06%) in 2018 PGT students are not getting enough support with choosing future career/further study directions and are ‘on their own after the programme’: There is no specific assistance in terms of jobs/career post MA. This is particularly worrying for me as I only came back to study because I was out of work … It's as … we're on our own after the program, with little attention given to find out what we intend to do after the program. University should play a greater role in boosting their employability chances: Opportunity to work on live projects once uni is finished - under a university ’umbrella’ Utilise staff contacts and engage in voluntary applied experiences – to help with career planning… positive aspects included: variety of skills and knowledge learned to enable graduates in future career; hearing from guest lecturers; opportunities to attend career events to prepare for employment; being able to apply learning to work life situations; relevance of topics to future careers and some other.

Career : main themes More focus on career/employability in the curriculum: More career driven assignments… I would like to get involved in engineering teamwork related to the future career. Students realise that lack of industry experience would limit their career prospects after the graduation and suggested to talk more about entry level career options: The opportunities have been highlighted for careers but most of these will not be available to us upon graduation as we lack years of experience in the industry. Entry level career prospects would feel more appropriate to be highlighted. … it will be difficult to go to interviews and answer questions on industry experience when the course doesn't provide us with any…

Feedback: main themes Feedback related comments are getting more prominent: 348 hits in 2017 (instances that met relevance threshold) 458 hits in 2018, located within a different conceptual basis Opportunities to receive feedback were limited in Semester 1: There hasn't yet been much chance for feedback on my work… …most of the markers didn't provide feedback I know lecturers are busy … but not prioritising PG students is disappointing. Poor quality feedback hindered progression: Feedback and grades were late by 6-8 weeks and the feedback was unhelpful and didn't allow me to advance in any way.

Feedback: main themes Role that useful, constructive feedback plays in boosting confidence, willingness to learn and sense of belonging : …It gives you a bit of a boost when you receive feedback and realise you’re on the right track. Feedback on my written work would challenge me further to produce my best work. There are times when I have felt the workload to be unmanageable particularly this year due to not receiving sufficient guidance…   ….my tutor has made me feel less confident with their not very constructive feedback. I understand negative feedback is vital, however [the tutor] done it in such a way it made me feel as though I didn't belong on the course. [feedback] helps me personally to concentrate on the next assignment when I know what my marks are.

What the data is telling us? The demographic profile of PGT students’ cohort has changed with more part time, mature and educated to Masters level students entering PG education. These students expect better value for money and more individualised study experience at PGT level than at undergraduate level. More students coming from different subject areas are undertaking postgraduate studies. These students ask for more 'orientation' type of resources and reading lists uploaded on Canvas prior to start of the course. Implications for skill support workshops - library to play an important role. A skills dialog needs to be embedded into to the curriculum and reinforced throughout the academic year. Role of feedback in student progressing, belonging and confidence building. More visible/proactive career support is required (both careers and programme teams should be involved).

Enhancement initiatives PGT experience task and finish group Emphasis on induction/orientation/managing expectations Library skills session review Career support - esp. for students on non –vocational programmes; support that takes into account motivational factors and experience of students Emphasis on feedback (PGCert, staff development sessions).

Questions or comments? References Soilemetzidis, I., Bennett, P. and Leman, J. (2014) The Postgraduate Taught Experience Survey 2014. York: Higher Education Academy [Internet]. Available from: http://www.improvingthestudentexperience.com/library/PG_documents/PTES_2014_Report.pdf Zaitseva, E. and Milsom, C. (2015) In Their Own Words: Analysing Students’ Comments from the Postgraduate Taught Experience Survey. York: Higher Education Academy. Available from: https://www.heacademy.ac.uk/knowledge-hub/their-own-words